arXiv subjects

Ethan N. Epperly

Publications and source records attributed to Ethan N. Epperly.

27 records · Page 2Linked to original sources

arXiv2024

The ESPRIT algorithm under high noise: Optimal error scaling and noisy super-resolution

Subspace-based signal processing techniques, such as the Estimation of Signal Parameters via Rotational Invariant Techniques (ESPRIT) algorithm, are popular methods for spectral estimation. These algorithms can achieve the so-called super-resolution scaling under low noise conditions, surpassing the well-known Nyquist limit. However, the performance of these algorithms under high-noise conditions is not as well understood. Existing state-of-the-art analysis indicates that ESPRIT and related algorithms can be resilient even for signals where each observation is corrupted by statistically independent, mean-zero noise of size $\mathcal{O}(1)$, but these analyses only show that the error $ε$ decays at a slow rate $ε=\mathcal{\tilde{O}}(n^{-1/2})$ with respect to the cutoff frequency $n$ (i.e., the maximum frequency of the measurements). In this work, we prove that under certain assumptions, the ESPRIT algorithm can attain a significantly improved error scaling $ε= \mathcal{\tilde{O}}(n^{-3/2})$, exhibiting noisy super-resolution scaling beyond the Nyquist limit $ε= \mathcal{O}(n^{-1})$ given by the Nyquist-Shannon sampling theorem. We further establish a theoretical lower bound and show that this scaling is optimal. Our analysis introduces novel matrix perturbation results, which could be of independent interest.

cs.IT↗

arXiv2024

Randomly pivoted Cholesky: Practical approximation of a kernel matrix with few entry evaluations

The randomly pivoted partial Cholesky algorithm (RPCholesky) computes a factorized rank-k approximation of an N x N positive-semidefinite (psd) matrix. RPCholesky requires only (k + 1) N entry evaluations and O(k^2 N) additional arithmetic operations, and it can be implemented with just a few lines of code. The method is particularly useful for approximating a kernel matrix. This paper offers a thorough new investigation of the empirical and theoretical behavior of this fundamental algorithm. For matrix approximation problems that arise in scientific machine learning, experiments show that RPCholesky matches or beats the performance of alternative algorithms. Moreover, RPCholesky provably returns low-rank approximations that are nearly optimal. The simplicity, effectiveness, and robustness of RPCholesky strongly support its use in scientific computing and machine learning applications.

math.NA↗

arXiv2024

Efficient error and variance estimation for randomized matrix computations

Randomized matrix algorithms have become workhorse tools in scientific computing and machine learning. To use these algorithms safely in applications, they should be coupled with posterior error estimates to assess the quality of the output. To meet this need, this paper proposes two diagnostics: a leave-one-out error estimator for randomized low-rank approximations and a jackknife resampling method to estimate the variance of the output of a randomized matrix computation. Both of these diagnostics are rapid to compute for randomized low-rank approximation algorithms such as the randomized SVD and randomized Nyström approximation, and they provide useful information that can be used to assess the quality of the computed output and guide algorithmic parameter choices.

math.NA↗

arXiv2024

Fast and forward stable randomized algorithms for linear least-squares problems

Iterative sketching and sketch-and-precondition are randomized algorithms used for solving overdetermined linear least-squares problems. When implemented in exact arithmetic, these algorithms produce high-accuracy solutions to least-squares problems faster than standard direct methods based on QR factorization. Recently, Meier, Nakatsukasa, Townsend, and Webb demonstrated numerical instabilities in a version of sketch-and-precondition in floating point arithmetic (arXiv:2302.07202). The work of Meier et al. raises the question: Is there a randomized least-squares solver that is both fast and stable? This paper resolves this question in the affirmative by proving that iterative sketching, appropriately implemented, is forward stable. Numerical experiments confirm the theoretical findings, demonstrating that iterative sketching is stable and faster than QR-based solvers for large problem instances.

math.NA↗

arXiv2024

XTrace: Making the most of every sample in stochastic trace estimation

The implicit trace estimation problem asks for an approximation of the trace of a square matrix, accessed via matrix-vector products (matvecs). This paper designs new randomized algorithms, XTrace and XNysTrace, for the trace estimation problem by exploiting both variance reduction and the exchangeability principle. For a fixed budget of matvecs, numerical experiments show that the new methods can achieve errors that are orders of magnitude smaller than existing algorithms, such as the Girard-Hutchinson estimator or the Hutch++ estimator. A theoretical analysis confirms the benefits by offering a precise description of the performance of these algorithms as a function of the spectrum of the input matrix. The paper also develops an exchangeable estimator, XDiag, for approximating the diagonal of a square matrix using matvecs.

math.NA↗

arXiv2023

Kernel quadrature with randomly pivoted Cholesky

This paper presents new quadrature rules for functions in a reproducing kernel Hilbert space using nodes drawn by a sampling algorithm known as randomly pivoted Cholesky. The resulting computational procedure compares favorably to previous kernel quadrature methods, which either achieve low accuracy or require solving a computationally challenging sampling problem. Theoretical and numerical results show that randomly pivoted Cholesky is fast and achieves comparable quadrature error rates to more computationally expensive quadrature schemes based on continuous volume sampling, thinning, and recombination. Randomly pivoted Cholesky is easily adapted to complicated geometries with arbitrary kernels, unlocking new potential for kernel quadrature.

math.NA↗

arXiv2023

A theory of quantum subspace diagonalization

Quantum subspace diagonalization methods are an exciting new class of algorithms for solving large\rev{-}scale eigenvalue problems using quantum computers. Unfortunately, these methods require the solution of an ill-conditioned generalized eigenvalue problem, with a matrix pair corrupted by a non-negligible amount of noise that is far above the machine precision. Despite pessimistic predictions from classical \rev{worst-case} perturbation theories, these methods can perform reliably well if the generalized eigenvalue problem is solved using a standard truncation strategy. By leveraging and advancing classical results in matrix perturbation theory, we provide a theoretical analysis of this surprising phenomenon, proving that under certain natural conditions, a quantum subspace diagonalization algorithm can accurately compute the smallest eigenvalue of a large Hermitian matrix. We give numerical experiments demonstrating the effectiveness of the theory and providing practical guidance for the choice of truncation level. Our new results can also be of independent interest to solving eigenvalue problems outside the context of quantum computation.

quant-ph↗

arXiv2021

Graph-Induced Rank Structures and their Representations

A new framework is proposed to study rank-structured matrices arising from discretizations of 2D and 3D elliptic operators. In particular, we introduce the notion of a graph-induced rank structure (GIRS) which aims to capture the fine low rank structures which appear in sparse matrices and their inverses in terms of the adjacency graph $\mathbb{G}$. We show that the GIRS property is invariant under inversion, and hence any effective representation of the inverse of GIRS matrices would lead to effective solvers. Starting with the observation that sequentially semi-separable (SSS) matrices form a good candidate for representing GIRS matrices on the line graph, we propose two extensions of SSS matrices to arbitrary graphs: Dewilde-van der Veen (DV) representations and $\mathbb{G}$-semi-separable ($\mathbb{G}$-SS) representations. It is shown that both these representations come naturally equipped with fast solvers where the solve complexity is commensurate to fast sparse Gaussian elimination on the graph $\mathbb{G}$, and $\mathbb{G}$-SS representations have a linear time multiplication algorithm. We show the construction of these representations to be highly nontrivial by determining the minimal $\mathbb{G}$-SS representation for the cycle graph $\mathbb{G}$. To obtain a minimal representation, we solve an exotic variant of a low-rank completion problem.

math.NA↗

arXiv2021

Minimal Rank Completions for Overlapping Blocks

We consider the multi-objective optimization problem of choosing the bottom left block-entry of a block lower triangular matrix to minimize the ranks of all block sub-matrices. We provide a proof that there exists a simultaneous rank-minimizer by constructing the complete set of all minimizers.

math.OC↗