arXiv subjects

Per-Gunnar Martinsson

Publications and source records attributed to Per-Gunnar Martinsson.

At least 19 recordsLinked to original sources

arXiv2026

A stable and fast method for solving multibody scattering problems via the method of fundamental solutions

The paper describes a numerical method for solving acoustic multibody scattering problems in two and three dimensions. The idea is to compute a highly accurate approximation to the scattering operator for each body through a local computation, and then use these scattering matrices to form a global linear system. The resulting coefficient matrix is relatively well-conditioned, even for problems involving a very large number of scatterers. The linear system is amenable to iterative solvers, and can readily be accelerated via fast algorithms for the matrix-vector multiplication such as the fast multipole method. The key point of the work is that the local scattering matrices can be constructed using potentially ill-conditioned techniques such as the method of fundamental solutions (MFS), while still maintaining scalability and numerical stability of the global solver. The resulting algorithm is simple, as the MFS is far simpler to implement than alternative techniques based on discretizing boundary integral equations using Nyström or Galerkin.

math.NA↗

arXiv2025

Randomized Algorithms for Low-Rank Matrix and Tensor Decompositions

This paper surveys randomized algorithms in numerical linear algebra for low-rank decompositions of matrices and tensors. The survey begins with a review of classical matrix algorithms that can be accelerated by randomized dimensionality reduction, such as the singular value decomposition (SVD) or interpolative (ID) and CUR decompositions. Recent advances in randomized dimensionality reduction are discussed, including new methods of fast matrix sketching and sampling techniques, which are incorporated into classical matrix algorithms for fast low-rank matrix approximations. The extension of randomized matrix algorithms to tensors is then explored for several low-rank tensor decompositions in the CP and Tucker formats, including the higher-order SVD, ID, and CUR decomposition.

math.NA↗

arXiv2025

Randomized Block Low-Rank Matrix Compression by Tagging

In this work, we present randomized compression algorithms for flat rank-structured matrices with shared bases, termed uniform Block Low-Rank (BLR) matrices. Our main contribution is a technique called tagging, which improves upon the efficiency of existing algorithms for basis matrix computation while preserving accuracy. Tagging operates on the matrix using matrix-vector products of the matrix and its adjoint, making it suitable for black-box environments where accessing individual matrix entries is computationally expensive or infeasible. We show tagging requires a constant number of matrix-vector products coupled with linear post-processing; crucially, the asymptotic pre-factors in tagging depend only on the rank parameter and the underlying problem geometry. We also establish a theoretical connection between the optimal construction of tagging matrices and projective varieties in algebraic geometry, suggesting a hybrid numeric-symbolic avenue of future work. To validate our approach, we apply tagging to compress uniform BLR matrices arising from the discretization of integral and partial differential equations. Empirical results show that tagging outperforms alternative compression techniques, significantly reducing both the number of required matrix-vector products and overall computational time. These findings highlight the practicality and scalability of tagging as an efficient method for flat rank-structured matrices in scientific computing.

math.NA↗

arXiv2025

Randomized Strong Recursive Skeletonization: Simultaneous Compression and LU Factorization of Hierarchical Matrices using Matrix-Vector Products

The hierarchical matrix framework partitions matrices into subblocks that are either small or of low numerical rank, enabling linear storage complexity and efficient matrix-vector multiplication. This work focuses on the $H^2$-matrix format constructed under the strong admissibility condition, which has two key properties: (1) a compressed representation that approximates far-field interactions with low-rank blocks while near-field interactions are stored densely, and (2) a nested basis structure that reuses basis matrices across hierarchy levels. Although these matrices support fast Cholesky and LU factorizations, implementing them - especially for 3D PDE discretizations - remains challenging due to the nested recursions and recompressions involved. This paper introduces an algorithm that simultaneously compresses and factorizes a general ${H}^{2}$ matrix, using only the action of the matrix and its adjoint on vectors. The number of required matrix-vector products is independent of the matrix size, and depends only on the problem geometry and a rank parameter. The resulting LU factorization is invertible and can serve as an approximate direct solver, with accuracy influenced by the spectral properties of the matrix. To achieve competitive sample complexity, the method employs dense Gaussian test matrices without explicitly encoding structured sparsity. Samples are drawn only once at the start of the algorithm; as the factorization proceeds, structure is dynamically introduced into the test matrices through efficient linear algebraic operations. Numerical experiments demonstrate robustness to indefiniteness and ill-conditioning, as well as the efficiency of the method for integral and differential equations in 2D and 3D.

math.NA↗

arXiv2025

Fast Direct Solvers

This survey describes a class of methods known as "fast direct solvers". These algorithms address the problem of solving a system of linear equations $\boldsymbol{Ax}=\boldsymbol{b}$ arising from the discretization of either an elliptic PDE or of an associated integral equation. The matrix $\boldsymbol{A}$ will be sparse when the PDE is discretized directly, and dense when an integral equation formulation is used. In either case, industry practice for large scale problems has for decades been to use iterative solvers such as multigrid, GMRES, or conjugate gradients. A direct solver, in contrast, builds an approximation to the inverse of $\boldsymbol{A}$, or alternatively, an easily invertible factorization (e.g. LU or Cholesky). A major development in numerical analysis in the last couple of decades has been the emergence of algorithms for constructing such factorizations or performing such inversions in linear or close to linear time. Such methods must necessarily exploit that the matrix $\boldsymbol{A}^{-1}$ is "data-sparse", typically in the sense that it can be tessellated into blocks that have low numerical rank. This survey provides a unifying context to both sparse and dense fast direct solvers, introduces key concepts with a minimum of notational overhead, and provides guidance to help a user determine the best method to use for a given application.

math.NA↗

arXiv2025

An overlapping domain decomposition method based on solution-transfer operators

An overlapping domain decomposition method is described for variable-coefficient elliptic boundary value problems on domains that can be decomposed into slabs or shells. The method represents the global solution through its traces on internal interfaces, coupled by local Dirichlet solution transfer operators posed on overlapping double slab domains. The key observation is that these interface maps act between separated interfaces, and as such can be written as smooth-kernel integral operators. The resulting global equilibrium system is Fredholm second kind and, unlike non-overlapping formulations, requires no same-interface Dirichlet-to-Neumann or other interface maps with singular kernels. This makes its off-diagonal blocks highly amenable to hierarchical low-rank compression. The formulation admits complementary continuum and discrete interpretations. At fixed slab width, the method is stable under discretization, sufficiently accurate local solves and compression. At the discrete level, the system can be interpreted as a block Jacobi preconditioned Schur complement system. For compatible SPD discretizations satisfying a standard stable-splitting assumption, the symmetrically scaled interface matrix satisfies an energy-norm condition-number bound that depends on the slab width but is uniform with respect to the local resolution. The formulation is implemented using high-order local solvers and hierarchical compression based on randomized sampling. Numerical experiments report iteration counts, accuracy, and compressibility for 2D and 3D elliptic, nonsymmetric, and oscillatory problems with as many as 28 million degrees of freedom.

math.NA↗

arXiv2025

Fast Rank Adaptive CUR via a Recycled Small Sketch

The computation of accurate low-rank matrix approximations is central to improving the scalability of various techniques in machine learning, uncertainty quantification, and control. Traditionally, low-rank approximations are constructed using SVD-based approaches such as truncated SVD or RandomizedSVD. Although these SVD approaches -- especially RandomizedSVD -- have proven to be very computationally efficient, other low-rank approximation methods can offer even greater performance. One such approach is the CUR decomposition, which forms a low-rank approximation using direct row and column subsets of a matrix. Because CUR uses direct matrix subsets, it is also often better able to preserve native matrix structures like sparsity or non-negativity than SVD-based approaches and can facilitate data interpretation in many contexts. This paper introduces IterativeCUR, which draws on previous work in randomized numerical linear algebra to build a new algorithm that is highly competitive compared to prior work: (1) It is adaptive in the sense that it takes as an input parameter the desired tolerance, rather than an a priori guess of the numerical rank. (2) It typically runs significantly faster than both existing CUR algorithms and techniques such as RandomizedSVD, in particular when these methods are run in an adaptive rank mode. Its asymptotic complexity is $\mathcal{O}(mn + (m+n)r^2 + r^3)$ for an $m\times n$ matrix of numerical rank $r$. (3) It relies on a single small sketch from the matrix that is successively downdated as the algorithm proceeds. We demonstrate through extensive experiments that IterativeCUR achieves up to $4\times$ speed-up over state-of-the-art pivoting-on-sketch approaches with no loss of accuracy, and up to $40\times$ speed-up over rank-adaptive randomized SVD approaches.

math.NA↗

arXiv2025

A Two-Level Direct Solver for the Hierarchical Poincaré-Steklov Method

We introduce a two-level direct solver for the Hierarchical Poincaré-Steklov (HPS) method for solving linear elliptic PDEs. HPS combines multidomain spectral collocation with a direct solver, enabling high-order discretizations for highly oscillatory solutions while preserving computational efficiency. Our method employs batched linear algebra routines with GPU acceleration to reduce the problem to subdomain interfaces, yielding a block-sparse linear system. This system is then factorized using a sparse direct solver that employs pivoting to achieve better numerical stability than the original HPS scheme. For a discretization of local order $p$ involving a total of $N$ degrees of freedom, the initial reduction step has asymptotic complexity $O(N p^6)$ in three dimensions. Nevertheless, the high efficiency of batched GPU routines makes the overall cost for practical purposes independent of polynomial order (for order $p=20$ or even higher). Additionally, the cost of the sparse direct solver is independent of the polynomial order. We present a description and justification of our method, along with numerical experiments on three-dimensional problems to evaluate its accuracy and performance.

math.NA↗

arXiv2025

SlabLU: A Two-Level Sparse Direct Solver for Elliptic PDEs

The paper describes a sparse direct solver for the linear systems that arise from the discretization of an elliptic PDE on a two dimensional domain. The scheme decomposes the domain into thin subdomains, or ``slabs'' and uses a two-level approach that is designed with parallelization in mind. The scheme takes advantage of $\mathcal H^2$-matrix structure emerging during factorization and utilizes randomized algorithms to efficiently recover this structure. As opposed to multi-level nested dissection schemes that incorporate the use of $\mathcal H$ or $\mathcal H^2$ matrices for a hierarchy of front sizes, SlabLU is a two-level scheme which only uses $\mathcal H^2$-matrix algebra for fronts of roughly the same size. The simplicity allows the scheme to be easily tuned for performance on modern architectures and GPUs. The solver described is compatible with a range of different local discretizations, and numerical experiments demonstrate its performance for regular discretizations of rectangular and curved geometries. The technique becomes particularly efficient when combined with very high-order accurate multi-domain spectral collocation schemes. With this discretization, a Helmholtz problem on a domain of size $1000 λ\times 1000 λ$ (for which $N=100 \rm{M}$) is solved in 15 minutes to 6 correct digits on a high-powered desktop with GPU acceleration.

math.NA↗

arXiv2025

A Simplified Fast Multipole Method Based on Strong Recursive Skeletonization

This work introduces a kernel-independent, multilevel, adaptive algorithm for efficiently evaluating a discrete convolution kernel with a given source distribution. The method is based on linear algebraic tools such as low rank approximation and ``skeleton representations'' to approximate far-field interactions. While this work is related to previous linear algebraic formulations of the fast multipole method, the proposed algorithm is distinguished by relying on simpler data structures. The proposed algorithm eliminates the need for explicit interaction lists by restructuring computations to operate exclusively on the near-neighbor list at each level of the tree, thereby simplifying both implementation and data structures. This work also introduces novel translation operators that significantly simplify the handling of adaptive point distributions. As a kernel-independent approach, it only requires evaluation of the kernel function, making it easily adaptable to a variety of kernels. By using operations on the neighbor list (of size at most 27 in 3D) rather than the interaction list (of size up to 189 in 3D), the algorithm is particularly well-suited for parallel implementation on modern hardware. Numerical experiments on uniform and non-uniform point distributions in 2D and 3D demonstrate the effectiveness of the proposed parallel algorithm for Laplace and (low-frequency) Helmholtz kernels. The algorithm constructs a tailored skeleton representation for the given geometry during a precomputation stage. After precomputation, the fast summation achieves high efficiency on the GPU using batched linear algebra operations.

math.NA↗

arXiv2025

GPU Optimizations for the Hierarchical Poincaré-Steklov Scheme

This manuscript presents GPU optimizations for the 2D Hierarchical Poincaré-Steklov (HPS) discretization scheme. HPS is a multi-domain spectral collocation method that combines high-order discretizations with direct solvers to accurately resolve highly oscillatory solutions. The domain decomposition approach of HPS connects domains directly via a sparse direct solver. The proposed optimizations exploit batched linear algebra on modern hybrid architectures, are straightforward to implement, and improve the solver's practical speed. The manuscript demonstrates that GPU optimizations can significantly reduce the traditionally high cost of performing local static condensation for discretizations with very high local order $p$. Numerical experiments for the Helmholtz equation with high wavenumbers on curved and rectangular domains confirm the high accuracy achieved by the HPS discretization and the significant reduction in computation time achieved with GPU optimizations.

math.NA↗

arXiv2025

Accelerating a restarted Krylov method for matrix functions with randomization

Many scientific applications require the evaluation of the action of a matrix function on a vector, and Krylov subspace methods are the most common ones for this task. Since the orthogonalization cost and the memory requirements can quickly become overwhelming as the basis grows, the Krylov method is often restarted after a few iterations. This paper proposes a new acceleration technique for restarted Krylov methods based on randomization. The numerical experiments show that the randomized method greatly outperforms the classical approach at the same level of accuracy. In fact, randomization can actually improve the convergence rate of restarted methods in some cases. The paper also compares the performance and stability of several existing randomized methods for solving very large, ill-conditioned problems, complementing the numerical analyses from previous studies.

math.NA↗

arXiv2024

Robust Blockwise Random Pivoting: Fast and Accurate Adaptive Interpolative Decomposition

The interpolative decomposition (ID) aims to construct a low-rank approximation formed by a basis consisting of row/column skeletons in the original matrix and a corresponding interpolation matrix. This work explores fast and accurate ID algorithms from comprehensive perspectives for empirical performance, including accuracy in both skeleton selection and interpolation matrix construction, efficiency in terms of asymptotic complexity and hardware efficiency, as well as rank adaptiveness. While many algorithms have been developed to optimize some of these aspects, practical ID algorithms proficient in all aspects remain absent. To fill in the gap, we introduce robust blockwise random pivoting (RBRP) that is asymptotically fast, hardware-efficient, and rank-adaptive, providing accurate skeletons and interpolation matrices comparable to the best existing ID algorithms in practice. Through extensive numerical experiments on various synthetic and natural datasets, we demonstrate the appealing empirical performance of RBRP from the aforementioned perspectives, as well as the robustness of RBRP to adversarial inputs.

math.NA↗

arXiv2024

Solving Large Rank-Deficient Linear Least-Squares Problems on Shared-Memory CPU Architectures and GPU Architectures

Solving very large linear systems of equations is a key computational task in science and technology. In many cases, the coefficient matrix of the linear system is rank-deficient, leading to systems that may be underdetermined, inconsistent, or both. In such cases, one generally seeks to compute the least squares solution that minimizes the residual of the problem, which can be further defined as the solution with smallest norm in cases where the coefficient matrix has a nontrivial nullspace. This work presents several new techniques for solving least squares problems involving coefficient matrices that are so large that they do not fit in main memory. The implementations include both CPU and GPU variants. All techniques rely on complete orthogonal decompositions that guarantee that both conditions of a least squares solution are met, regardless of the rank properties of the matrix. Specifically, they rely on the recently proposed "randUTV" algorithm that is particularly effective in strongly communication-constrained environments. A detailed precision and performance study reveals that the new methods, that operate on data stored on disk, are competitive with state-of-the-art methods that store all data in main memory.

cs.DC↗

arXiv2024

Efficient Bounds and Estimates for Canonical Angles in Randomized Subspace Approximations

Randomized subspace approximation with "matrix sketching" is an effective approach for constructing approximate partial singular value decompositions (SVDs) of large matrices. The performance of such techniques has been extensively analyzed, and very precise estimates on the distribution of the residual errors have been derived. However, our understanding of the accuracy of the computed singular vectors (measured in terms of the canonical angles between the spaces spanned by the exact and the computed singular vectors, respectively) remains relatively limited. In this work, we present practical bounds and estimates for canonical angles of randomized subspace approximation that can be computed efficiently either a priori or a posteriori, without assuming prior knowledge of the true singular subspaces. Under moderate oversampling in the randomized SVD, our prior probabilistic bounds are asymptotically tight and can be computed efficiently, while bringing a clear insight into the balance between oversampling and power iterations given a fixed budget on the number of matrix-vector multiplications. The numerical experiments demonstrate the empirical effectiveness of these canonical angle bounds and estimates on different matrices under various algorithmic choices for the randomized SVD.

math.NA↗

arXiv2024

Randomized Compression of Rank-Structured Matrices Accelerated with Graph Coloring

A randomized algorithm for computing a data sparse representation of a given rank structured matrix $A$ (a.k.a. an $H$-matrix) is presented. The algorithm draws on the randomized singular value decomposition (RSVD), and operates under the assumption that algorithms for rapidly applying $A$ and $A^{*}$ to vectors are available. The algorithm analyzes the hierarchical tree that defines the rank structure using graph coloring algorithms to generate a set of random test vectors. The matrix is then applied to the test vectors, and in a final step the matrix itself is reconstructed by the observed input-output pairs. The method presented is an evolution of the "peeling algorithm" of L. Lin, J. Lu, and L. Ying, "Fast construction of hierarchical matrix representation from matrix-vector multiplication," JCP, 230(10), 2011. For the case of uniform trees, the new method substantially reduces the pre-factor of the original peeling algorithm. More significantly, the new technique leads to dramatic acceleration for many non-uniform trees since it constructs sample vectors that are optimized for a given tree. The algorithm is particularly effective for kernel matrices involving a set of points restricted to a lower dimensional object than the ambient space, such as a boundary integral equation defined on a surface in three dimensions.

math.NA↗

arXiv2024

Linear-Complexity Black-Box Randomized Compression of Rank-Structured Matrices

A randomized algorithm for computing a compressed representation of a given rank-structured matrix $A \in \mathbb{R}^{N\times N}$ is presented. The algorithm interacts with $A$ only through its action on vectors. Specifically, it draws two tall thin matrices $Ω,\,Ψ\in \mathbb{R}^{N\times s}$ from a suitable distribution, and then reconstructs $A$ from the information contained in the set $\{AΩ,\,Ω,\,A^{*}Ψ,\,Ψ\}$. For the specific case of a "Hierarchically Block Separable (HBS)" matrix (a.k.a. Hierarchically Semi-Separable matrix) of block rank $k$, the number of samples $s$ required satisfies $s = O(k)$, with $s \approx 3k$ being representative. While a number of randomized algorithms for compressing rank-structured matrices have previously been published, the current algorithm appears to be the first that is both of truly linear complexity (no $N\log(N)$ factors in the complexity bound) and fully "black box" in the sense that no matrix entry evaluation is required. Further, all samples can be extracted in parallel, enabling the algorithm to work in a "streaming" or "single view" mode.

math.NA↗

arXiv2024

Fast and high-order approximation of parabolic equations using hierarchical direct solvers and implicit Runge-Kutta methods

An additive Runge-Kutta method is used for the time stepping, which integrates the linear stiff terms by an explicit singly diagonally implicit Runge-Kutta (ESDIRK) method and the nonlinear terms by an explicit Runge-Kutta (ERK) method. In each time step, the implicit solve is performed by the recently developed Hierarchical Poincaré-Steklov (HPS) method. This is a fast direct solver for elliptic equations that decomposes the space domain into a hierarchical tree of subdomains and builds spectral collocation solvers locally on the subdomains. These ideas are naturally combined in the presented method since the singly diagonal coefficient in ESDIRK and a fixed time-step ensures that the coefficient matrix in the implicit solve of HPS remains the same for all time stages. This means that the precomputed inverse can be efficiently reused, leading to a scheme with complexity (in two dimensions) $\mathcal{O}(N^{1.5})$ for the precomputation where the solution operator to the elliptic problems is built, and then $\mathcal{O}(N \log N)$ for the solve in each time step. The stability of the method is proved for first order in time and any order in space, and numerical evidence substantiates a claim of stability for a much broader class of time discretization methods. Numerical experiments supporting the accuracy of efficiency of the method in one and two dimensions are presented.

math.NA↗