arXiv subjects

Yuji Nakatsukasa

Publications and source records attributed to Yuji Nakatsukasa.

At least 73 records · Page 4Linked to original sources

arXiv2021

Solving two-parameter eigenvalue problems using an alternating method

We present a new approach to compute selected eigenvalues and eigenvectors of the two-parameter eigenvalue problem. Our method requires computing generalized eigenvalue problems of the same size as the matrices of the initial two-parameter eigenvalue problem. The method is applicable for right definite problems, possibly after performing an affine transformation. This includes a class of Helmholtz equations when separation of variables is applied. We provide a convergence proof for extremal eigenvalues and empirical evidence along with a local convergence proof for other eigenvalues.

math.NA↗

arXiv2021

Stable and Efficient Computation of Generalized Polar Decompositions

We present methods for computing the generalized polar decomposition of a matrix based on the dynamically weighted Halley (DWH) iteration. This method is well established for computing the standard polar decomposition. A stable implementation is available, where matrix inversion is avoided and QR decompositions are used instead. We establish a natural generalization of this approach for computing generalized polar decompositions with respect to signature matrices. Again the inverse can be avoided by using a generalized QR decomposition called hyperbolic QR decomposition. However, this decomposition does not show the same favorable stability properties as its orthogonal counterpart. We overcome the numerical difficulties by generalizing the CholeskyQR2 method. This method computes the standard QR decomposition in a stable way via two successive Cholesky factorizations. An even better numerical stability is achieved by employing permuted graph bases, yielding residuals of order $10^{-14}$ even for badly conditioned matrices, where other methods fail.

math.NA↗

arXiv2020

Twice is enough for dangerous eigenvalues

We analyze the stability of a class of eigensolvers that target interior eigenvalues with rational filters. We show that subspace iteration with a rational filter is robust even when an eigenvalue is near a filter's pole. These dangerous eigenvalues contribute to large round-off errors in the first iteration, but are self-correcting in later iterations. For matrices with orthogonal eigenvectors (e.g., real-symmetric or complex Hermitian), two iterations is enough to reduce round-off errors to the order of the unit-round off. In contrast, Krylov methods accelerated by rational filters with fixed poles typically fail to converge to unit round-off accuracy when an eigenvalue is close to a pole. In the context of Arnoldi with shift-and-invert enhancement, we demonstrate a simple restart strategy that recovers full precision in the target eigenpairs.

math.NA↗

arXiv2020

Zolotarev's fifth and sixth problems

In an influential 1877 paper, Zolotarev asked and answered four questions about polynomial and rational approximation. We ask and answer two questions: what are the best rational approximants $r$ and $s$ to $\sqrt{z}$ and $\mbox{sign}(z)$ on the unit circle (excluding certain arcs near the discontinuities), with the property that $|r(z)|=|s(z)|=1$ for $|z|=1$? We show that the solutions to these problems are related to Zolotarev's third and fourth problems in a nontrivial manner.

math.CA↗

arXiv2020

Reciprocal-log approximation and planar PDE solvers

This article is about both approximation theory and the numerical solution of partial differential equations (PDEs). First we introduce the notion of {\em reciprocal-log} or {\em log-lightning approximation} of analytic functions with branch point singularities at points $\{z_k\}$ by functions of the form $g(z) = \sum_k c_k /(\log(z-z_k) - s_k)$, which have $N$ poles potentially distributed along a Riemann surface. We prove that the errors of best reciprocal-log approximations decrease exponentially with respect to $N$ and that exponential or near-exponential convergence (i.e., at a rate $O(\exp(-C N / \log N))$) also holds for near-best approximations with preassigned singularities constructed by linear least-squares fitting on the boundary. We then apply these results to derive a "log-lightning method" for numerical solution of Laplace and related PDEs in two-dimensional domains with corner singularities. The convergence is near-exponential, in contrast to the root-exponential convergence for the original lightning methods based on rational functions.

math.NA↗

arXiv2020

Rational neural networks

We consider neural networks with rational activation functions. The choice of the nonlinear activation function in deep learning architectures is crucial and heavily impacts the performance of a neural network. We establish optimal bounds in terms of network complexity and prove that rational neural networks approximate smooth functions more efficiently than ReLU networks with exponentially smaller depth. The flexibility and smoothness of rational activation functions make them an attractive alternative to ReLU, as we demonstrate with numerical experiments.

cs.NE↗

arXiv2020

Fast and stable randomized low-rank matrix approximation

Randomized SVD has become an extremely successful approach for efficiently computing a low-rank approximation of matrices. In particular the paper by Halko, Martinsson, and Tropp (SIREV 2011) contains extensive analysis, and has made it a very popular method. The typical complexity for a rank-$r$ approximation of $m\times n$ matrices is $O(mn\log n+(m+n)r^2)$ for dense matrices. The classical Nystr{ö}m method is much faster, but applicable only to positive semidefinite matrices. This work studies a generalization of Nystr{ö}m method applicable to general matrices, and shows that (i) it has near-optimal approximation quality comparable to competing methods, (ii) the computational cost is the near-optimal $O(mn\log n+r^3)$ for dense matrices, with small hidden constants, and (iii) crucially, it can be implemented in a numerically stable fashion despite the presence of an ill-conditioned pseudoinverse. Numerical experiments illustrate that generalized Nystr{ö}m can significantly outperform state-of-the-art methods, especially when $r\gg 1$, achieving up to a 10-fold speedup. The method is also well suited to updating and downdating the matrix.

math.NA↗

arXiv2020

Exponential node clustering at singularities for rational approximation, quadrature, and PDEs

Rational approximations of functions with singularities can converge at a root-exponential rate if the poles are exponentially clustered. We begin by reviewing this effect in minimax, least-squares, and AAA approximations on intervals and complex domains, conformal mapping, and the numerical solution of Laplace, Helmholtz, and biharmonic equations by the "lightning" method. Extensive and wide-ranging numerical experiments are involved. We then present further experiments showing that in all of these applications, it is advantageous to use exponential clustering whose density on a logarithmic scale is not uniform but tapers off linearly to zero near the singularity. We give a theoretical explanation of the tapering effect based on the Hermite contour integral and potential theory, showing that tapering doubles the rate of convergence. Finally we show that related mathematics applies to the relationship between exponential (not tapered) and doubly exponential (tapered) quadrature formulas. Here it is the Gauss--Takahasi--Mori contour integral that comes into play.

math.NA↗

arXiv2020

Error localization of best L1 polynomial approximants

An important observation in compressed sensing is that the $\ell_0$ minimizer of an underdetermined linear system is equal to the $\ell_1$ minimizer when there exists a sparse solution vector and a certain restricted isometry property holds. Here, we develop a continuous analogue of this observation and show that the best $L_0$ and $L_1$ polynomial approximants of a polynomial that is corrupted on a set of small measure are nearly equal. We go on to demonstrate an error localization property of best $L_1$ polynomial approximants and use our observations to develop an improved algorithm for computing best $L_1$ polynomial approximants to continuous functions.

math.NA↗

arXiv2020

Fast Graph Sampling Set Selection Using Gershgorin Disc Alignment

Graph sampling set selection, where a subset of nodes are chosen to collect samples to reconstruct a smooth graph signal, is a fundamental problem in graph signal processing (GSP). Previous works employ an unbiased least-squares (LS) signal reconstruction scheme and select samples via expensive extreme eigenvector computation. Instead, we assume a biased graph Laplacian regularization (GLR) based scheme that solves a system of linear equations for reconstruction. We then choose samples to minimize the condition number of the coefficient matrix---specifically, maximize the smallest eigenvalue $λ_{\min}$. Circumventing explicit eigenvalue computation, we maximize instead the lower bound of $λ_{\min}$, designated by the smallest left-end of all Gershgorin discs of the matrix. To achieve this efficiently, we first convert the optimization to a dual problem, where we minimize the number of samples needed to align all Gershgorin disc left-ends at a chosen lower-bound target $T$. Algebraically, the dual problem amounts to optimizing two disc operations: i) shifting of disc centers due to sampling, and ii) scaling of disc radii due to a similarity transformation of the matrix. We further reinterpret the dual as an intuitive disc coverage problem bearing strong resemblance to the famous NP-hard set cover (SC) problem. The reinterpretation enables us to derive a fast approximation scheme from a known SC error-bounded approximation algorithm. We find an appropriate target $T$ efficiently via binary search. Extensive simulation experiments show that our disc-based sampling algorithm runs substantially faster than existing sampling schemes and outperforms other eigen-decomposition-free sampling schemes in reconstruction error.

eess.SP↗

arXiv2019

Sharp error bounds for Ritz vectors and approximate singular vectors

We derive sharp bounds for the accuracy of approximate eigenvectors (Ritz vectors) obtained by the Rayleigh-Ritz process for symmetric eigenvalue problems. Using information that is available or easy to estimate, our bounds improve the classical Davis-Kahan $\sinθ$ theorem by a factor that can be arbitrarily large, and can give nontrivial information even when the $\sinθ$ theorem suggests that a Ritz vector might have no accuracy at all. We also present extensions in three directions, deriving error bounds for invariant subspaces, singular vectors and subspaces computed by a (Petrov-Galerkin) projection SVD method, and eigenvectors of self-adjoint operators on a Hilbert space.

math.NA↗

arXiv2019

Classification of chaotic time series with deep learning

We use standard deep neural networks to classify univariate time series generated by discrete and continuous dynamical systems based on their chaotic or non-chaotic behaviour. Our approach to circumvent the lack of precise models for some of the most challenging real-life applications is to train different neural networks on a data set from a dynamical system with a basic or low-dimensional phase space and then use these networks to classify univariate time series of a dynamical system with more intricate or high-dimensional phase space. We illustrate this generalisation approach using the logistic map, the sine-circle map, the Lorenz system, and the Kuramoto--Sivashinsky equation. We observe that a convolutional neural network without batch normalization layers outperforms state-of-the-art neural networks for time series classification and is able to generalise and classify time series as chaotic or not with high accuracy.

eess.SP↗

arXiv2019

Vandermonde with Arnoldi

Vandermonde matrices are exponentially ill-conditioned, rendering the familiar "polyval(polyfit)" algorithm for polynomial interpolation and least-squares fitting ineffective at higher degrees. We show that Arnoldi orthogonalization fixes the problem.

math.NA↗

arXiv2019

Weighted Monte Carlo with least squares and randomized extended Kaczmarz for option pricing

We propose a methodology for computing single and multi-asset European option prices, and more generally expectations of scalar functions of (multivariate) random variables. This new approach combines the ability of Monte Carlo simulation to handle high-dimensional problems with the efficiency of function approximation. Specifically, we first generalize the recently developed method for multivariate integration in [arXiv:1806.05492] to integration with respect to probability measures. The method is based on the principle "approximate and integrate" in three steps i) sample the integrand at points in the integration domain, ii) approximate the integrand by solving a least-squares problem, iii) integrate the approximate function. In high-dimensional applications we face memory limitations due to large storage requirements in step ii). Combining weighted sampling and the randomized extended Kaczmarz algorithm we obtain a new efficient approach to solve large-scale least-squares problems. Our convergence and cost analysis along with numerical experiments show the effectiveness of the method in both low and high dimensions, and under the assumption of a limited number of available simulations.

q-fin.CP↗

arXiv2019

An algorithm for real and complex rational minimax approximation

Rational minimax approximation of real functions on real intervals is an established topic, but when it comes to complex functions or domains, there appear to be no algorithms currently in use. Such a method is introduced here, the {\em AAA-Lawson algorithm,} available in Chebfun. The new algorithm solves a wide range of problems on arbitrary domains in a fraction of a second of laptop time by a procedure consisting of two steps. First, the standard AAA algorithm is run to obtain a near-best approximation and a set of support points for a barycentric representation of the rational approximant. Then a "Lawson phase" of iteratively reweighted least-squares adjustment of the barycentric coefficients is carried out to improve the approximation to minimax.

math.NA↗

arXiv2019

Accuracy of approximate projection to the semidefinite cone

When a projection of a symmetric or Hermitian matrix to the positive semidefinite cone is computed approximately (or to working precision on a computer), a natural question is to quantify its accuracy. A straightforward bound invoking standard eigenvalue perturbation theory (e.g. Davis-Kahan and Weyl bounds) suggests that the accuracy would be inversely proportional to the spectral gap, implying it can be poor in the presence of small eigenvalues. This work shows that a small gap is not a concern for projection onto the semidefinite cone, by deriving error bounds that are gap-independent.

math.NA↗

arXiv2019

Approximating the pth Root by Composite Rational Functions

A landmark result from rational approximation theory states that $x^{1/p}$ on $[0,1]$ can be approximated by a type-$(n,n)$ rational function with root-exponential accuracy. Motivated by the recursive optimality property of Zolotarev functions (for the square root and sign functions), we investigate approximating $x^{1/p}$ by composite rational functions of the form $r_k(x, r_{k-1}(x, r_{k-2}( \cdots (x,r_1(x,1)) )))$. While this class of rational functions ceases to contain the minimax (best) approximant for $p\geq 3$, we show that it achieves approximately $p$th-root exponential convergence with respect to the degree. Moreover, crucially, the convergence is doubly exponential with respect to the number of degrees of freedom, suggesting that composite rational functions are able to approximate $x^{1/p}$ and related functions (such as $|x|$ and the sector function) with exceptional efficiency.

math.NA↗

arXiv2019

The low-rank eigenvalue problem

The nonzero eigenvalues of $AB$ are equal to those of $BA$: an identity that holds as long as the products are square, even when $A,B$ are rectangular. This fact naturally suggests an efficient algorithm for computing eigenvalues and eigenvectors of a low-rank matrix $X= AB$ with $A,B^T\in\mathbb{C}^{N\times r}, N\gg r$: form the small $r\times r$ matrix $BA$ and find its eigenvalues and eigenvectors. For nonzero eigenvalues, the eigenvectors are related by $ ABv = λv \Leftrightarrow BAw = λw $ with $w=Bv$, and the same holds for Jordan vectors. For zero eigenvalues, the Jordan blocks can change sizes between $AB$ and $BA$, and we characterize this behavior.

math.NA↗