Searcharxiv⌕ Search

arXiv subjects

Yuji Nakatsukasa

Publications and source records attributed to Yuji Nakatsukasa.

At least 19 recordsLinked to original sources
arXiv2026

Nystr\"om method for symmetric indefinite matrices

The Nystr\"om method approximates $A\approx A(\,:\,,I)A(I,I)^{\dagger} A(\,:\,,I)^{\top}=CA(I,I)^{\dagger} C^{\top}$, where $C:=A(:,I)\in\mathbb{R}^{n\times r}$ is a column subset matrix of $A$. When applied to symmetric but indefinite matrices, the Nystr\"om method can fail because the core matrix $A(I,I)$ may severely underestimate the eigenvalues of $A$ and may become (nearly) singular. We address this issue by developing and analyzing an algorithm that carefully chooses $\widehat{M}\in\mathbb{R}^{r\times r}$ in place of $A(I,I)^\dagger$ by solving the two-sided sketched least-squares problem $\min_{M}\|X(A-CMC^{\top})X^{\top}\|_F$, where $X\in\mathbb{R}^{t\times n}$ is a random sketch matrix. We study in detail the cases where $X$ is a Gaussian or a leverage score sampling (LSS) matrix, and show that with oversampling $t>r$ the residual $\|A-C\widehat{M}C^{\top}\|_*$ is comparable to $\min_{M}\|A-CMC^{\top}\|_*$. For the Gaussian sketch, we require $t=\mathcal{O}(r)$ samples; for LSS, we show that $t=\mathcal{O}(r \log r)$ samples suffice for the theoretical guarantee, with the LSS approach carrying the advantage that once a set of $t$ row indices is identified, the approximation requires only $t^{2}$ matrix-entry evaluations to find $\widehat{M}$, given $C$. We illustrate our results with synthetic examples and applications to kernel methods.

math.NA↗
arXiv2026

Accelerating preconditioned Jacobi methods via perturbation-inspired pivoting

Perturbation theory for symmetric matrices shows that eigenvalues with small spectral gaps are more sensitive to off-diagonal perturbation, implying that different entries affect the eigenvalues unevenly. Building on this insight, we incorporate spectral gap information into the Jacobi method for symmetric eigenvalue problems and propose a new pivoting strategy, which is completely different from classical ones governed solely by the magnitude of the off-diagonal entries. When combined with a mixed-precision preconditioner that diagonalizes the matrix to low precision, numerical experiments demonstrate that the resulting strategy can significantly outperform the classical greedy approach when the original matrix has clustered eigenvalues.

math.NA↗
arXiv2026

Convergence analysis of a nonlinear eigensolver based on rational approximation of the resolvent

Given a holomorphic matrix-valued function, the poles of its sketched resolvent are generically its eigenvalues. Once a good rational approximation of the sketched resolvent is obtained, the poles of this rational approximation typically lie close to those eigenvalues, thus providing a flexible framework for solving both linear and nonlinear eigenvalue problems. However, the accuracy of the computed eigenvalues is limited and remains poorly understood. This paper analyzes the convergence of this approach and demonstrates the effectiveness of two techniques to improve accuracy: block probing and zooming in. We also establish the backward and forward stability of polefinding for a barycentric rational form via a generalized eigenproblem. Numerical experiments demonstrate the sharpness of our theoretical results.

math.NA↗
arXiv2026

A multilevel sketch-and-solve method for overdetermined least squares problems

Sketch-and-solve (SAS) is a very successful method to efficiently estimate the solution of heavily overdetermined large linear least squares problems. It uses random sketching to reduce the size of the problem, hence reducing the computational cost. Several authors have shown that averaging several solutions from SAS further improves the accuracy, which is measured by the residual associated to the approximate solution. Going further, we combine solutions from sketch-and-solve in a multilevel manner, such that the approximate solution is a combination of SAS samples obtained from small sketches and more accurate correction terms obtained from larger sketches. We first consider the variance of the estimator, which depends on the variance of the coarse samples and the correction terms. We show that the variance of the correction terms on each level follows a trend and decreases faster than the variance of the simple SAS estimator. However, we then show that the overall computational cost of our multilevel framework is slightly higher than that of the simple average estimator, so a naive application of multilevel methods appears unattractive for least squares problems.

math.NA↗
arXiv2026

Towards Universal Convergence of Backward Error in Linear System Solvers

The quest for an algorithm that solves an $n\times n$ linear system in $O(n^2)$ time complexity, or $O(n^2 \text{poly}(1/ε))$ when solving up to $ε$ relative error, is a long-standing open problem in numerical linear algebra and theoretical computer science. There are two predominant paradigms for measuring relative error: forward error (i.e., distance from the output to the optimum solution) and backward error (i.e., distance to the nearest problem solved by the output). In most prior studies, convergence of iterative linear system solvers is measured via various notions of forward error, and as a result, depends heavily on the conditioning of the input. Yet, the numerical analysis literature has long advocated for backward error as the more practically relevant notion of approximation. In this work, we show that -- surprisingly -- the classical and simple Richardson iteration incurs at most $1/k$ (relative) backward error after $k$ iterations on any positive semidefinite (PSD) linear system, irrespective of its condition number. This universal convergence rate implies an $O(n^2/ε)$ complexity algorithm for solving a PSD linear system to $ε$ backward error, and we establish similar or better complexity when using a variety of Krylov solvers beyond Richardson. Then, by directly minimizing backward error over a Krylov subspace, we attain an even faster $O(1/k^2)$ universal rate, and we turn this into an efficient algorithm, MINBERR, with complexity $O(n^2/\sqrtε)$. Finally, we extend this approach via normal equations to solving general linear systems in $O(n^2\log(n)/ε)$ time complexity. We report strong numerical performance of our algorithms on benchmark problems.

math.NA↗
arXiv2026

Finding accurate eigenvalues and eigenvectors of positive semi-definite matrices given a subspace

We revisit a classical problem in numerical linear algebra: given an $k$-dimensional subspace $\mathcal{Q}$ that approximates the leading eigenspace of an $n\times n$ positive semi-definite matrix $A$, the goal is to extract high-accuracy eigenvalues. The Rayleigh-Ritz (RR) method is the standard algorithm for the task, which has been shown to be optimal in several ways (when $A$ is symmetric, not necessarily positive semi-definite $A\succeq 0$). In this paper, we show that when $A \succeq 0$, alternative methods can outperform RR, while having the same computational complexity, that is, the main cost is in computing $AQ$, plus an $O(nk^2)$ term. In particular, we advocate the use of Nystr{ö}m's method, showing that the approximate eigenvalues always have higher accuracy than RR, and the improvement can be arbitrarily large. The difference is significant, especially when $A$ has a fast-decaying spectrum. A similar improvement is numerically observed for the purpose of approximating the leading eigenvectors. In contrast, when the target eigenvalues are the trailing ones, the situation is reversed, and the Nystr{ö}m method performs poorly; we suggest a remedy for this situation.

math.NA↗
arXiv2026

Adaptive LSQR Preconditioning from One Small Sketch

We propose APLICUR, an adaptive preconditioning framework for large-scale linear least-squares (LLS) problems. Using a single small sketch computed once at initialization, APLICUR incrementally refines a CUR-based preconditioner throughout the Krylov solve, interleaving preconditioning with iteration. This enables early convergence without the need to construct a costly high-quality preconditioner upfront. With a modest sketch dimension (typically 5 - 250), largely independent of both the problem size and numerical rank, APLICUR achieves convergence guarantees that are likewise independent of the sketch size. The method is applicable to general matrices without structural assumptions (e.g. need not be heavily overdetermined) and is well suited to large, sparse, or numerically low-rank problems. We conduct extensive numerical studies to examine the behavior of the proposed framework and guide the effective algorithmic design choices. Across a range of test problems, \mainalg{} achieves competitive or improved time-to-accuracy performance compared with established randomized preconditioners, including Blendenpik and Nyström PCG, while maintaining low setup cost and robustness across problem regimes.

math.NA↗
arXiv2026

Approximating Sparse Matrices and their Functions using Matrix-vector products

The computation of a matrix function $f(A)$ is an important task in scientific computing appearing in machine learning, network analysis and the solution of partial differential equations. In this work, we use only matrix-vector products $x\mapsto Ax$ to approximate functions of sparse matrices and matrices with similar structures such as sparse matrices $A$ themselves or matrices that have a similar decay property as matrix functions. We show that when $A$ is a sparse matrix with an unknown sparsity pattern, techniques from compressed sensing can be used under natural assumptions. Moreover, if $A$ is a banded matrix then certain deterministic matrix-vector products can efficiently recover the large entries of $f(A)$. We describe an algorithm for each of the two cases and give error analysis based on the decay bound for the entries of $f(A)$. We finish with numerical experiments showing the accuracy of our algorithms.

math.NA↗
arXiv2026

Matrix Perturbation Theory in the Tangent Space of Isospectral Matrices

Eigenvalue and eigenvector perturbation theory is a fundamental topic in several disciplines, including numerical linear algebra, quantum physics, and related fields. The central problem is to understand how the eigenvalues and eigenvectors of a matrix $A \in \mathbb{C}^{n \times n}$ change under the addition of a perturbation matrix $E \in \mathbb{C}^{n \times n}$. Much of the existing literature focuses on structured perturbations. For example, in [C.-K. Li and R.-C. Li, Linear Algebra Appl. 2005], the matrix $A$ is assumed to be Hermitian and block diagonal, while the perturbation $E$ is Hermitian and block off-diagonal. In this work, we investigate a different structured setting in which the perturbation has the commutator form $E = AB - BA$ for some matrix $B$, which we show to be a generalization of the block diagonal structure considered by Li and Li. First, we extend their main result by showing that the perturbation of the $i$-th eigenvalue of $A$, denoted by $λ_i$, is of order $\|E\|^2 / η_i$, where $η_i = \min_{j \neq i} |λ_i - λ_j|$ is the spectral gap associated with $λ_i$. Second, we provide a detailed analysis of the role played by the matrix $B$ in the perturbation of the eigenvectors. This analysis is further generalized to the case of block-diagonal matrices with multiple eigenvalues, as well as to perturbed singular values and eigenvalues of Jordan blocks.

math.NA↗
arXiv2026

Fast, High-Accuracy, Randomized Nullspace Computations for Tall Matrices

In this paper, we develop RLOBPCG, an efficient method for computing a small number of singular triplets corresponding to the smallest singular values of large, tall matrices. The algorithm combines randomized preconditioner from the sketch-and-precondition techniques with the LOBPCG eigensolver: a small sketch is used to construct a high-quality preconditioner, and LOBPCG is run on the Gram matrix to refine the singular vector. Under the standard subspace embedding assumption and a modest singular value gap between the two smallest singular values, we prove that RLOBPCG converges geometrically to the minimum singular vector. In numerical experiments, RLOBPCG achieves near-optimal accuracy on matrices with up to $10^6$ rows, outperforming classical LOBPCG and Lanczos methods by a speedup of up to $12\times$ and maintaining robustness when other iterative methods fail to converge.

math.NA↗
arXiv2026

Linear Systems and Eigenvalue Problems: Open Questions from a Simons Workshop

This document presents a series of open questions arising in matrix computations, i.e., the numerical solution of linear algebra problems. It is a result of working groups at the workshop Linear Systems and Eigenvalue Problems, which was organized at the Simons Institute for the Theory of Computing program on Complexity and Linear Algebra in Fall 2025. The complexity and numerical solution of linear algebra problems is a crosscutting area between theoretical computer science and numerical analysis. The value of the particular problem formulations here is that they were produced via discussions between researchers from both groups. The open questions are organized in five categories: iterative solvers for linear systems, eigenvalue computation, low-rank approximation, randomized sketching, and other areas including tensors, quantum systems, and matrix functions. (Updated to reflect the status of the open problems as of August 20, 2026.)

math.NA↗
arXiv2026

Matrix perturbation analysis of methods for extracting singular values from approximate singular subspaces

Given (orthonormal) approximations $\tilde{U}$ and $\tilde{V}$ to the left and right subspaces spanned by the leading singular vectors of a matrix $A$, we discuss methods to approximate the leading singular values of $A$ and study their accuracy. In particular, we focus our analysis on the generalized Nyström approximation, as surprisingly, it is able to obtain significantly better accuracy than classical methods, namely Rayleigh-Ritz and (one-sided) projected SVD. A key idea of the analysis is to view the methods as finding the exact singular values of a perturbation of $A$. In this context, we derive a matrix perturbation result that exploits the structure of such $2\times2$ block matrix perturbation. Furthermore, we extend it to block tridiagonal matrices. We then obtain bounds on the accuracy of the extracted singular values. This leads to sharp bounds that predict well the approximation error trends and explain the difference in the behavior of these methods. Finally, we present an approach to derive an a-posteriori version of those bounds, which are more amenable to computation in practice.

math.NA↗
arXiv2025

Numerical Stability of the Nyström Method

The Nyström method is a widely used technique for improving the scalability of kernel-based algorithms, including kernel ridge regression, spectral clustering, and Gaussian processes. Despite its popularity, the numerical stability of the method has remained largely an unresolved problem. In particular, the pseudo-inversion of the submatrix involved in the Nyström method may pose stability issues as the submatrix is likely to be ill-conditioned, resulting in numerically poor approximation. In this work, we establish conditions under which the Nyström method is numerically stable. We show that stability can be achieved through an appropriate choice of column subsets and a careful implementation of the pseudoinverse. Our results and experiments provide theoretical justification and practical guidance for the stable application of the Nyström method in large-scale kernel computations.

math.NA↗
arXiv2025

Randomized flexible Krylov methods for $\ell_p$ regularization

The computation of sparse solutions of large-scale linear discrete ill-posed problems remains a computationally demanding task. A powerful framework in this context is the use of iteratively reweighted schemes, which are based on constructing a sequence of quadratic tangent majorants of the $\ell_2$-$\ell_1$ regularization functional (with additional smoothing to ensure differentiability at the origin), and solving them successively. Recently, flexible Krylov-Tikhonov methods have been used to partially solve each problem in the sequence efficiently. However, in order to guarantee convergence, the complexity of the algorithm at each iteration increases with respect to more traditional methods. We propose a randomized flexible Krylov method to alleviate the increase of complexity, which leverages the adaptability of the flexible Krylov subspaces with the efficiency of `sketch-and-solve' methods. A possible caveat of the mentioned methods is their memory requirements. In this case, one needs to rely instead on inner-outer schemes. In these scenarios, we propose a `sketch-to-precondition' method to speed up the convergence of each of the subproblems in the sequence. The performance of these algorithms is shown through a variety of numerical examples.

math.NA↗
arXiv2025

SubApSnap: Solving parameter-dependent linear systems with a snapshot and subsampling

A growing number of problems in computational mathematics can be reduced to the solution of many linear systems that are related, often depending smoothly or slowly on a parameter $p$, that is, $A(p)x(p)=b(p)$. We introduce an efficient algorithm for solving such parameter-dependent linear systems for many values of $p$. The algorithm, which we call SubApSnap (for \emph{Sub}sampled $A(p)$ times \emph{Snap}shot), is based on combining ideas from model order reduction and randomised linear algebra: namely, taking a snapshot matrix, and solving the resulting tall-skinny least-squares problems using a subsampling-based dimension-reduction approach. We show that SubApSnap is a strict generalisation of the popular DEIM algorithm in nonlinear model order reduction. SubApSnap is a sublinear-time algorithm, and once the snapshot and subsampling are determined, it solves $A(p_*)x(p_*)=b(p_*)$ for a new value of $p_*$ at a dramatically improved speed: it does not even need to read the whole matrix $A(p_*)$ to solve the linear system for a new value of $p_*$. We prove under natural assumptions that, given a good subsampling and snapshot, SubApSnap yields solutions with small residual for all parameter values of interest. We illustrate the efficiency and performance of the algorithm with problems arising in PDEs, model reduction, and kernel ridge regression, where SubApSnap achieves speedups of many orders of magnitude over a standard solution; for example over $20,000\times$ for a $10^7\times 10^7$ problem, while providing good accuracy.

math.NA↗
arXiv2025

Sharp error bounds for approximate eigenvalues and singular values from subspace methods

Subspace methods are commonly used for finding approximate eigenvalues and singular values of large-scale matrices. Once a subspace is found, the Rayleigh-Ritz method (for symmetric eigenvalue problems) and Petrov-Galerkin projection (for singular values) are the de facto method for extraction of eigenvalues and singular values. In this work we derive quadratic error bounds for approximate eigenvalues of symmetric matrices obtained via the Rayleigh-Ritz process. Our bounds take advantage of the fact that extremal eigenpairs tend to converge faster than the rest, hence having smaller residuals $\|A\widehat x_i-θ_i\widehat x_i\|_2$, where $(θ_i,\widehat x_i)$ is a Ritz pair (approximate eigenpair). The proof uses the structure of the perturbation matrix underlying the Rayleigh-Ritz method to bound the components of its eigenvectors. In this way, we obtain a bound of the form $c\frac{\|A\widehat x_i-θ_i\widehat x_i\|_2^2}{\mbox{Gap}_i}$, where $\mbox{Gap}_i$ is roughly the gap between the $i$th Ritz value and the eigenvalues that are not approximated by the Ritz process, and $c> 1$ is a modest scalar. Our bound is adapted to each Ritz value and is robust to clustered Ritz values, which is a key improvement over existing results. We further show that the bound is asymptotically sharp, and generalize it to singular values of arbitrary real matrices. Finally, we apply these bounds to several methods for computing eigenvalues and singular values, and illustrate the sharpness of our bounds in a number of computational settings, including Krylov methods and randomized algorithms.

math.NA↗
arXiv2025

Instability of the Sherman-Morrison formula and stabilization by iterative refinement

Owing to its simplicity and efficiency, the Sherman-Morrison (SM) formula has seen widespread use across various scientific and engineering applications for solving rank-one perturbed linear systems of the form $(A+uv^T)x = b$. Although the formula dates back at least to 1944, its numerical stability properties have remained an open question and continue to be a topic of current research. We analyze the backward stability of the SM, demonstrate its instability in a scenario increasingly common in scientific computing and address an open question posed by Nick Higham on the proportionality of the backward error bound to the condition number of $A$. We then incorporate fixed-precision iterative refinement into the SM framework reusing the previously computed decompositions and prove that, under reasonable assumptions, it achieves backward stability without sacrificing the efficiency of the SM formula. While our theory does not prove the SM formula with iterative refinement always outputs a backward stable solution, empirically it is observed to eventually produce a backward stable solution in all our numerical experiments. We conjecture that with iterative refinement, the SM formula yields a backward stable solution provided that $κ_2(A), κ_2(A+uv^T)$ are both bounded safely away from $ε_M^{-1}$, where $ε_M$ is the unit roundoff.

math.NA↗