arXiv subjects

Yuji Nakatsukasa

Publications and source records attributed to Yuji Nakatsukasa.

90 records · Page 5Linked to original sources

arXiv2018

Shifted CholeskyQR for computing the QR factorization of ill-conditioned matrices

The Cholesky QR algorithm is an efficient communication-minimizing algorithm for computing the QR factorization of a tall-skinny matrix. Unfortunately it has the inherent numerical instability and breakdown when the matrix is ill-conditioned. A recent work establishes that the instability can be cured by repeating the algorithm twice (called CholeskyQR2). However, the applicability of CholeskyQR2 is still limited by the requirement that the Cholesky factorization of the Gram matrix runs to completion, which means it does not always work for matrices $X$ with $κ_2(X)\gtrsim {\bf u}^{-\frac{1}{2}}$ where ${\bf u}$ is the unit roundoff. In this work we extend the applicability to $κ_2(X)=\mathcal{O}({\bf u}^{-1})$ by introducing a shift to the computed Gram matrix so as to guarantee the Cholesky factorization $R^TR= A^TA+sI$ succeeds numerically. We show that the computed $AR^{-1}$ has reduced condition number $\leq {\bf u}^{-\frac{1}{2}}$, for which CholeskyQR2 safely computes the QR factorization, yielding a computed $Q$ of orthogonality $\|Q^TQ-I\|_2$ and residual $\|A-QR\|_F/\|A\|_F$ both $\mathcal{O}({\bf u})$. Thus we obtain the required QR factorization by essentially running Cholesky QR thrice. We extensively analyze the resulting algorithm shiftedCholeskyQR to reveal its excellent numerical stability. shiftedCholeskyQR is also highly parallelizable, and applicable and effective also when working in an oblique inner product space. We illustrate our findings through experiments, in which we achieve significant (up to x40) speedup over alternative methods.

math.NA↗

arXiv2018

Approximate and integrate: Variance reduction in Monte Carlo integration via function approximation

Classical algorithms in numerical analysis for numerical integration (quadrature/cubature) follow the principle of approximate and integrate: the integrand is approximated by a simple function (e.g. a polynomial), which is then integrated exactly. In high-dimensional integration, such methods quickly become infeasible due to the curse of dimensionality. A common alternative is the Monte Carlo method (MC), which simply takes the average of random samples, improving the estimate as more and more samples are taken. The main issue with MC is its slow (though dimension-independent) convergence, and various techniques have been proposed to reduce the variance. In this work we suggest a numerical analyst's interpretation of MC: it approximates the integrand with a constant function, and integrates that constant exactly. This observation leads naturally to MC-like methods where the approximant is a non-constant function, for example low-degree polynomials, sparse grids or low-rank functions. We show that these methods have the same $O(1/\sqrt{N})$ asymptotic convergence as in MC, but with reduced variance, equal to the quality of the underlying function approximation. We also discuss methods that improve the approximation quality as more samples are taken, and thus can converge faster than $O(1/\sqrt{N})$. The main message is that techniques in high-dimensional approximation theory can be combined with Monte Carlo integration to accelerate convergence.

math.NA↗

arXiv2018

Rational minimax approximation via adaptive barycentric representations

Computing rational minimax approximations can be very challenging when there are singularities on or near the interval of approximation - precisely the case where rational functions outperform polynomials by a landslide. We show that far more robust algorithms than previously available can be developed by making use of rational barycentric representations whose support points are chosen in an adaptive fashion as the approximant is computed. Three variants of this barycentric strategy are all shown to be powerful: (1) a classical Remez algorithm, (2) a "AAA-Lawson" method of iteratively reweighted least-squares, and (3) a differential correction algorithm. Our preferred combination, implemented in the Chebfun MINIMAX code, is to use (2) in an initial phase and then switch to (1) for generically quadratic convergence. By such methods we can calculate approximations up to type (80, 80) of $|x|$ on $[-1, 1]$ in standard 16-digit floating point arithmetic, a problem for which Varga, Ruttan, and Carpenter required 200-digit extended precision.

math.NA↗

arXiv2018

A Backward Stable Algorithm for Computing the CS Decomposition via the Polar Decomposition

We introduce a backward stable algorithm for computing the CS decomposition of a partitioned $2n \times n$ matrix with orthonormal columns, or a rank-deficient partial isometry. The algorithm computes two $n \times n$ polar decompositions (which can be carried out in parallel) followed by an eigendecomposition of a judiciously crafted $n \times n$ Hermitian matrix. We prove that the algorithm is backward stable whenever the aforementioned decompositions are computed in a backward stable way. Since the polar decomposition and the symmetric eigendecomposition are highly amenable to parallelization, the algorithm inherits this feature. We illustrate this fact by invoking recently developed algorithms for the polar decomposition and symmetric eigendecomposition that leverage Zolotarev's best rational approximations of the sign function. Numerical examples demonstrate that the resulting algorithm for computing the CS decomposition enjoys excellent numerical stability.

math.NA↗

arXiv2018

On orthogonal tensors and best rank-one approximation ratio

As is well known, the smallest possible ratio between the spectral norm and the Frobenius norm of an $m \times n$ matrix with $m \le n$ is $1/\sqrt{m}$ and is (up to scalar scaling) attained only by matrices having pairwise orthonormal rows. In the present paper, the smallest possible ratio between spectral and Frobenius norms of $n_1 \times \dots \times n_d$ tensors of order $d$, also called the best rank-one approximation ratio in the literature, is investigated. The exact value is not known for most configurations of $n_1 \le \dots \le n_d$. Using a natural definition of orthogonal tensors over the real field (resp., unitary tensors over the complex field), it is shown that the obvious lower bound $1/\sqrt{n_1 \cdots n_{d-1}}$ is attained if and only if a tensor is orthogonal (resp., unitary) up to scaling. Whether or not orthogonal or unitary tensors exist depends on the dimensions $n_1,\dots,n_d$ and the field. A connection between the (non)existence of real orthogonal tensors of order three and the classical Hurwitz problem on composition algebras can be established: existence of orthogonal tensors of size $\ell \times m \times n$ is equivalent to the admissibility of the triple $[\ell,m,n]$ to the Hurwitz problem. Some implications for higher-order tensors are then given. For instance, real orthogonal $n \times \dots \times n$ tensors of order $d \ge 3$ do exist, but only when $n = 1,2,4,8$. In the complex case, the situation is more drastic: unitary tensors of size $\ell \times m \times n$ with $\ell \le m \le n$ exist only when $\ell m \le n$. Finally, some numerical illustrations for spectral norm computation are presented.

math.NA↗

arXiv2018

Rational approximation of $x^n$

Let $E_{kk}^{(n)}$ denote the minimax (i.e., best supremum norm) error in approximation of $x^n$ on $[\kern .3pt 0,1]$ by rational functions of type $(k,k)$ with $k<n$. We show that in an appropriate limit $E_{kk}^{(n)} \sim 2\kern .3pt H^{k+1/2}$ independently of $n$, where $H \approx 1/9.28903$ is Halphen's constant. This is the same formula as for minimax approximation of $e^x$ on $(-\infty,0\kern .3pt]$.

math.NA↗

arXiv2017

Fourth-order time-stepping for stiff PDEs on the sphere

We present in this paper algorithms for solving stiff PDEs on the unit sphere with spectral accuracy in space and fourth-order accuracy in time. These are based on a variant of the double Fourier sphere method in coefficient space with multiplication matrices that differ from the usual ones, and implicit-explicit time-stepping schemes. Operating in coefficient space with these new matrices allows one to use a sparse direct solver, avoids the coordinate singularity and maintains smoothness at the poles, while implicit-explicit schemes circumvent severe restrictions on the time-steps due to stiffness. A comparison is made against exponential integrators and it is found that implicit-explicit schemes perform best. Implementations in MATLAB and Chebfun make it possible to compute the solution of many PDEs to high accuracy in a very convenient fashion.

math.NA↗

arXiv2017

Inertia laws and localization of real eigenvalues for generalized indefinite eigenvalue problems

Sylvester's law of inertia states that the number of positive, negative and zero eigenvalues of Hermitian matrices is preserved under congruence transformations. The same is true of generalized Hermitian definite eigenvalue problems, in which the two matrices are allowed to undergo different congruence transformations, but not for the indefinite case. In this paper we investigate the possible change in inertia under congruence for generalized Hermitian indefinite eigenproblems, and derive sharp bounds that show the inertia of the two individual matrices often still provides useful information about the eigenvalues of the pencil, especially when one of the matrices is almost definite. A prominent application of the original Sylvester's law is in finding the number of eigenvalues in an interval. Our results can be used for estimating the number of real eigenvalues in an interval for generalized indefinite and nonlinear eigenvalue problems.

math.NA↗

arXiv2017

The AAA algorithm for rational approximation

We introduce a new algorithm for approximation by rational functions on a real or complex set of points, implementable in 40 lines of Matlab and requiring no user input parameters. Even on a disk or interval the algorithm may outperform existing methods, and on more complicated domains it is especially competitive. The core ideas are (1) representation of the rational approximant in barycentric form with interpolation at certain support points and (2) greedy selection of the support points to avoid exponential instabilities. The name AAA stands for "adaptive Antoulas--Anderson" in honor of the authors who introduced a scheme based on (1). We present the core algorithm with a Matlab code and nine applications and describe variants targeted at problems of different kinds. Comparisons are made with vector fitting, RKFIT, and other existing methods for rational approximation.

math.NA↗

arXiv2016

Vector spaces of linearizations for matrix polynomials: a bivariate polynomial approach

We revisit the landmark paper [D. S. Mackey, N. Mackey, C. Mehl, and V. Mehrmann, SIAM J. Matrix Anal. Appl., 28 (2006), pp.~971--1004] and, by viewing matrices as coefficients for bivariate polynomials, we provide concise proofs for key properties of linearizations for matrix polynomials. We also show that every pencil in the double ansatz space is intrinsically connected to a Bézout matrix, which we use to prove the eigenvalue exclusion theorem. In addition our exposition allows for any polynomial basis and for any field. The new viewpoint also leads to new results. We generalize the double ansatz space by exploiting its algebraic interpretation as a space of Bézout pencils to derive new linearizations with potential applications in the theory of structured matrix polynomials. Moreover, we analyze the conditioning of double ansatz space linearizations in the important practical case of a Chebyshev basis.

math.NA↗

arXiv2016

Finding a low-rank basis in a matrix subspace

For a given matrix subspace, how can we find a basis that consists of low-rank matrices? This is a generalization of the sparse vector problem. It turns out that when the subspace is spanned by rank-1 matrices, the matrices can be obtained by the tensor CP decomposition. For the higher rank case, the situation is not as straightforward. In this work we present an algorithm based on a greedy process applicable to higher rank problems. Our algorithm first estimates the minimum rank by applying soft singular value thresholding to a nuclear norm relaxation, and then computes a matrix with that rank using the method of alternating projections. We provide local convergence results, and compare our algorithm with several alternative approaches. Applications include data compression beyond the classical truncated SVD, computing accurate eigenvectors of a near-multiple eigenvalue, image separation and graph Laplacian eigenproblems.

math.NA↗

arXiv2014

A logarithmic minimization property of the unitary polar factor in the spectral norm and the Frobenius matrix norm

The unitary polar factor $Q=U$ in the polar decomposition of the matrix $Z=UH$ is the minimizer for both $\| \mathrm{Log}(Q^* Z)\|^2$ and its Hermitian part $\| \mathrm{sym Log}(Q^* Z)\|^2$ over both $\mathbb{R}$ and $\mathbb{C}$, for any given invertible matrix $Z$ in $\mathbb{C}^{n\times n}$ and any matrix logarithm $\mathrm{Log}$, not necessarily the principal logarithm $\mathrm{log}$. We prove this for the spectral matrix norm in any dimension and for the Frobenius matrix norm in two and three dimensions. The result shows that the unitary polar factor is the nearest orthogonal matrix to $Z$ not only in the normwise sense, but also in a geodesic distance. The derivation is based on Bhatia's generalization of Bernstein's trace inequality for the matrix exponential and a new sum of squared logarithms inequality.

math.CA↗

arXiv2013

The minimization of matrix logarithms - on a fundamental property of the unitary polar factor

We show that the unitary factor U in the polar decomposition of a nonsingular matrix Z = U H is the minimizer for both ||Log(Q^* Z)|| and ||sym (Log(Q^*Z))|| over unitary Q, for any given invertible complex n-times-n matrix Z, for any unitarily invariant norm and any n. We prove that U is the unique matrix with this property. As important tools we use a generalized Bernstein trace inequality and the theory of majorization.

math.CA↗

arXiv2012

Mysteries around the graph Laplacian eigenvalue 4

We describe our current understanding on the phase transition phenomenon of the graph Laplacian eigenvectors constructed on a certain type of unweighted trees, which we previously observed through our numerical experiments. The eigenvalue distribution for such a tree is a smooth bell-shaped curve starting from the eigenvalue 0 up to 4. Then, at the eigenvalue 4, there is a sudden jump. Interestingly, the eigenvectors corresponding to the eigenvalues below 4 are semi-global oscillations (like Fourier modes) over the entire tree or one of the branches; on the other hand, those corresponding to the eigenvalues above 4 are much more localized and concentrated (like wavelets) around junctions/branching vertices. For a special class of trees called starlike trees, we obtain a complete understanding of such phase transition phenomenon. For a general graph, we prove the number of the eigenvalues larger than 4 is bounded from above by the number of vertices whose degrees is strictly higher than 2. Moreover, we also prove that if a graph contains a branching path, then the magnitudes of the components of any eigenvector corresponding to the eigenvalue greater than 4 decay exponentially from the branching vertex toward the leaf of that branch.

math.NA↗

arXiv2011

On the condition numbers of a multiple generalized eigenvalue

For standard eigenvalue problems, a closed-form expression for the condition numbers of a multiple eigenvalue is known. In particular, they are uniformly 1 in the Hermitian case, and generally take different values in the non-Hermitian case. We consider the generalized eigenvalue problem and identify the condition numbers of a multiple eigenvalue. Our main result is that a multiple eigenvalue generally has multiple condition numbers, even in the Hermitian definite case. The condition numbers are characterized in terms of the singular values of the outer product of the corresponding left and right eigenvectors.

math.NA↗

arXiv2010

Quadratic perturbation bounds for generalized eigenvalue problems

We prove quadratic eigenvalue perturbation bounds for generalized Hermitian eigenvalue problems. The bounds are proportional to the square of the norm of the perturbation matrices divided by the gap between the spectrums. Using the results we provide a simple derivation of the first-order perturbation expansion of a multiple eigenvalue, whose trailing term is tighter than known results. We also present quadratic bounds for the non-Hermitian case.

math.NA↗

arXiv2010

Perturbation bounds of eigenvalues of Hermitian matrices with block structures

We derive new perturbation bounds for eigenvalues of Hermitian matrices with block structures. The structures we consider range from a standard 2-by-2 block form to block tridiagonal and tridigaonal forms. The main idea is the observation that an eigenvalue is insensitive to componentwise perturbations if the corresponding eigenvector components are small. We show that the same idea can be used to explain two well-known phenomena, one concerning extremal eigenvalues of Wilkinson's matrices and another concerning the efficiency of aggressive early deflation applied to the symmetric tridiagonal QR algorithm.

math.NA↗

arXiv2010

Gerschgorin's theorem for generalized eigenvalue problems in the Euclidean metric

We present Gerschgorin-type eigenvalue inclusion sets applicable to generalized eigenvalue problems.Our sets are defined by circles in the complex plane in the standard Euclidean metric, and are easier to compute than known similar results.As one application we use our results to provide a forward error analysis for a computed eigenvalue of a diagonalizable pencil.

math.NA↗