Edouard Timsit | Searcharxiv

arXiv2025

Randomized orthogonalization and Krylov subspace methods: principles and algorithms

We present an overview of randomized orthogonalization techniques that construct a well-conditioned basis whose sketch is orthonormal. Randomized orthogonalization has recently emerged as a powerful paradigm for reducing the computational and communication cost of state-of-the-art orthogonalization procedures on parallel architectures, while preserving, and in some cases improving, their numerical stability. This approach can be employed within Krylov subspace methods to mitigate the cost of orthogonalization, yielding a randomized Arnoldi relation. We review the main variants of the randomized Gram--Schmidt and Householder QR algorithms, and discuss their application to Krylov methods for the solution of large-scale linear algebra problems, such as linear systems of equations, eigenvalue problems, the evaluation of matrix functions, and matrix equations.

math.NA↗

arXiv2024

Randomized Householder QR

This paper introduces a randomized Householder QR factorization (RHQR). This factorization can be used to obtain a well conditioned basis of a vector space and thus can be employed in a variety of applications. The RHQR factorization of the input matrix $W$ is equivalent to the standard Householder QR factorization of matrix $\Psi W$, where $\Psi$ is a sketching matrix that can be obtained from any subspace embedding technique. For this reason, the RHQR factorization can also be reconstructed from the Householder QR factorization of the sketched problem, yielding a single-synchronization randomized QR factorization (recRHQR). In most contexts, left-looking RHQR requires a single synchronization per iteration, with half the computational cost of Householder QR, and a similar cost to Randomized Gram-Schmidt (RGS) overall. We discuss the usage of RHQR factorization in the Arnoldi process and then in GMRES, showing thus how it can be used in Krylov subspace methods to solve systems of linear equations. Based on Charles Sheffield's connection between Householder QR and Modified Gram-Schmidt (MGS), a BLAS2-RGS is also derived. A finite precision analysis shows that, under mild probabilistic assumptions, the RHQR factorization of the input matrix $W$ inherits the stability of the Householder QR factorization, producing a well-conditioned basis and a columnwise backward stable factorization, all independently of the condition number of the input $W$, and with the accuracy of the sketching step. We study the subsampled randomized Hadamard transform (SRHT) as a very stable sketching technique. Numerical experiments show that RHQR produces a well conditioned basis whose sketch is numerically orthogonal and an accurate factorization, even for the most difficult inputs and with high-dimensional operations made in half-precision.

math.NA↗

arXiv2023

Randomized Orthogonal Projection Methods for Krylov Subspace Solvers

Randomized orthogonal projection methods (ROPMs) can be used to speed up the computation of Krylov subspace methods in various contexts. Through a theoretical and numerical investigation, we establish that these methods produce quasi-optimal approximations over the Krylov subspace. Our numerical experiments outline the convergence of ROPMs for all matrices in our test set, with occasional spikes, but overall with a convergence rate similar to that of standard OPMs.

math.NA↗