arXiv subjects

Per-Gunnar Martinsson

Publications and source records attributed to Per-Gunnar Martinsson.

At least 37 records · Page 2Linked to original sources

arXiv2024

An O(N) distributed-memory parallel direct solver for planar integral equations

Boundary value problems involving elliptic PDEs such as the Laplace and the Helmholtz equations are ubiquitous in mathematical physics and engineering. Many such problems can be alternatively formulated as integral equations that are mathematically more tractable. However, an integral-equation formulation poses a significant computational challenge: solving large dense linear systems that arise upon discretization. In cases where iterative methods converge rapidly, existing methods that draw on fast summation schemes such as the Fast Multipole Method are highly efficient and well-established. More recently, linear complexity direct solvers that sidestep convergence issues by directly computing an invertible factorization have been developed. However, storage and computation costs are high, which limits their ability to solve large-scale problems in practice. In this work, we introduce a distributed-memory parallel algorithm based on an existing direct solver named ``strong recursive skeletonization factorization.'' Specifically, we apply low-rank compression to certain off-diagonal matrix blocks in a way that minimizes computation and data movement. Compared to iterative algorithms, our method is particularly suitable for problems involving ill-conditioned matrices or multiple right-hand sides. Large-scale numerical experiments are presented to show the performance of our Julia implementation.

math.NA↗

arXiv2023

Adaptive Parallelizable Algorithms for Interpolative Decompositions via Partially Pivoted LU

Interpolative and CUR decompositions involve "natural bases" of row and column subsets, or skeletons, of a given matrix that approximately span its row and column spaces. These low-rank decompositions preserve properties such as sparsity or non-negativity, and are easily interpretable in the context of the original data. For large-scale problems, randomized sketching to sample the row or column spaces with a random matrix can serve as an effective initial step in skeleton selection to reduce computational cost. A by now well-established approach has been to extract a randomized sketch, followed by column-pivoted QR CPQR) on the sketch matrix. This manuscript describes an alternative approach where CPQR is replaced by LU with partial pivoting (LUPP). While LUPP by itself is not rank-revealing, it is demonstrated that when used in a randomized setting, LUPP not only reveals the numerical rank, but also allows the estimation of the residual error as the factorization is built. The resulting algorithm is both adaptive and parallelizable, and attains much higher practical speed due to the lower communication requirements of LUPP over CPQR. The method has been implemented for both CPUs and GPUs, and the resulting software has been made publicly available.

math.NA↗

arXiv2023

Efficient algorithms for computing rank-revealing factorizations on a GPU

Standard rank-revealing factorizations such as the singular value decomposition and column pivoted QR factorization are challenging to implement efficiently on a GPU. A major difficulty in this regard is the inability of standard algorithms to cast most operations in terms of the Level-3 BLAS. This paper presents two alternative algorithms for computing a rank-revealing factorization of the form $A = U T V^*$, where $U$ and $V$ are orthogonal and $T$ is triangular. Both algorithms use randomized projection techniques to cast most of the flops in terms of matrix-matrix multiplication, which is exceptionally efficient on the GPU. Numerical experiments illustrate that these algorithms achieve an order of magnitude acceleration over finely tuned GPU implementations of the SVD while providing low-rank approximation errors close to that of the SVD.

math.NA↗

arXiv2022

A Unified Trapezoidal Quadrature Method for Singular and Hypersingular Boundary Integral Operators on Curved Surfaces

This paper describes a trapezoidal quadrature method for the discretization of singular and hypersingular boundary integral operators (BIOs) that arise in solving boundary value problems for elliptic partial differential equations. The quadrature is based on a uniform grid in parameter space coupled with the standard punctured Trapezoidal rule. A key observation is that the error incurred by the singularity in the kernel can be expressed exactly using generalized Euler-Maclaurin formulae that involve the Riemann zeta function in 2D and the Epstein zeta functions in 3D. These expansions are exploited to correct the errors via local stencils at the singular point using a novel systematic moment-fitting approach. This new method provides a unified treatment of all common BIOs (Laplace, Helmholtz, Stokes, etc.). We present numerical examples that show convergence of up to 32nd-order in 2D and 9th-order in 3D with respect to the mesh size.

math.NA↗

arXiv2022

Solving Linear Systems on a GPU with Hierarchically Off-Diagonal Low-Rank Approximations

We are interested in solving linear systems arising from three applications: (1) kernel methods in machine learning, (2) discretization of boundary integral equations from mathematical physics, and (3) Schur complements formed in the factorization of many large sparse matrices. The coefficient matrices are often data-sparse in the sense that their off-diagonal blocks have low numerical ranks; specifically, we focus on "hierarchically off-diagonal low-rank (HODLR)" matrices. We introduce algorithms for factorizing HODLR matrices and for applying the factorizations on a GPU. The algorithms leverage the efficiency of batched dense linear algebra, and they scale nearly linearly with the matrix size when the numerical ranks are fixed. The accuracy of the HODLR-matrix approximation is a tunable parameter, so we can construct high-accuracy fast direct solvers or low-accuracy robust preconditioners. Numerical results show that we can solve problems with several millions of unknowns in a couple of seconds on a single GPU.

math.NA↗

arXiv2022

Simpler is better: A comparative study of randomized algorithms for computing the CUR decomposition

The CUR decomposition is a technique for low-rank approximation that selects small subsets of the columns and rows of a given matrix to use as bases for its column and rowspaces. It has recently attracted much interest, as it has several advantages over traditional low rank decompositions based on orthonormal bases. These include the preservation of properties such as sparsity or non-negativity, the ability to interpret data, and reduced storage requirements. The problem of finding the skeleton sets that minimize the norm of the residual error is known to be NP-hard, but classical pivoting schemes such as column pivoted QR work tend to work well in practice. When combined with randomized dimension reduction techniques, classical pivoting based methods become particularly effective, and have proven capable of very rapidly computing approximate CUR decompositions of large, potentially sparse, matrices. Another class of popular algorithms for computing CUR de-compositions are based on drawing the columns and rows randomly from the full index sets, using specialized probability distributions based on leverage scores. Such sampling based techniques are particularly appealing for very large scale problems, and are well supported by theoretical performance guarantees. This manuscript provides a comparative study of the various randomized algorithms for computing CUR decompositions that have recently been proposed. Additionally, it proposes some modifications and simplifications to the existing algorithms that leads to faster execution times.

math.NA↗

arXiv2022

Randomized Algorithms for Scientific Computing (RASC)

Randomized algorithms have propelled advances in artificial intelligence and represent a foundational research area in advancing AI for Science. Future advancements in DOE Office of Science priority areas such as climate science, astrophysics, fusion, advanced materials, combustion, and quantum computing all require randomized algorithms for surmounting challenges of complexity, robustness, and scalability. This report summarizes the outcomes of that workshop, "Randomized Algorithms for Scientific Computing (RASC)," held virtually across four days in December 2020 and January 2021.

cs.AI↗

arXiv2021

Zeta Correction: A New Approach to Constructing Corrected Trapezoidal Quadrature Rules for Singular Integral Operators

A high-order accurate quadrature rule for the discretization of boundary integral equations (BIEs) on closed smooth contours in the plane is introduced. This quadrature can be viewed as a hybrid of the spectral quadrature of Kress (1991) and the locally corrected trapezoidal quadrature of Kapur and Rokhlin (1997). The new technique combines the strengths of both methods, and attains high-order convergence, numerical stability, ease of implementation, and compatibility with the "fast" algorithms (such as the Fast Multipole Method or Fast Direct Solvers). Important connections between the punctured trapezoidal rule and the Riemann zeta function are introduced, which enable a complete convergence analysis and lead to remarkably simple procedures for constructing the quadrature corrections. The paper reports a detailed comparison between the new method and the methods of Kress, of Kapur and Rokhlin, and of Alpert (1999).

math.NA↗

arXiv2021

Randomized Numerical Linear Algebra: Foundations & Algorithms

This survey describes probabilistic algorithms for linear algebra computations, such as factorizing matrices and solving linear systems. It focuses on techniques that have a proven track record for real-world problem instances. The paper treats both the theoretical foundations of the subject and the practical computational issues. Topics covered include norm estimation; matrix approximation by sampling; structured and unstructured random embeddings; linear regression problems; low-rank approximation; subspace iteration and Krylov methods; error estimation and adaptivity; interpolatory and CUR factorizations; Nyström approximation of positive-semidefinite matrices; single view ("streaming") algorithms; full rank-revealing factorizations; solvers for linear systems; and approximation of kernel matrices that arise in machine learning and in scientific computing.

math.NA↗

arXiv2020

An accelerated, high-order accurate direct solver for the Lippmann-Schwinger equation for acoustic scattering in the plane

An efficient direct solver for solving the Lippmann-Schwinger integral equation modeling acoustic scattering in the plane is presented. For a problem with $N$ degrees of freedom, the solver constructs an approximate inverse in $\mathcal{O}(N^{3/2})$ operations and then, given an incident field, can compute the scattered field in $\mathcal{O}(N \log N)$ operations. The solver is based on a previously published direct solver for integral equations that relies on rank-deficiencies in the off-diagonal blocks; specifically, the so-called Hierarchically Block Separable format is used. The particular solver described here has been reformulated in a way that improves numerical stability and robustness, and exploits the particular structure of the kernel in the Lippmann-Schwinger equation to accelerate the computation of an approximate inverse. The solver is coupled with a Nyström discretization on a regular square grid, using a quadrature method developed by Ran Duan and Vladimir Rokhlin that attains high-order accuracy despite the singularity in the kernel of the integral equation. A particularly efficient solver is obtained when the direct solver is run at four digits of accuracy, and is used as a preconditioner to GMRES, with each forwards application of the integral operators accelerated by the FFT. Extensive numerical experiments are presented that illustrate the high performance of the method in challenging environments. Using the $10^{\rm th}$-order accurate version of the Duan-Rokhlin quadrature rule, the scheme is capable of solving problems on domains that are over 500 wavelengths wide to residual error below $10^{-10}$ in a couple of hours on a workstation, using 26M degrees of freedom.

math.NA↗

arXiv2020

Corrected Trapezoidal Rules for Boundary Integral Equations in Three Dimensions

The manuscript describes a quadrature rule that is designed for the high order discretization of boundary integral equations (BIEs) using the Nyström method. The technique is designed for surfaces that can naturally be parameterized using a uniform grid on a rectangle, such as deformed tori, or channels with periodic boundary conditions. When a BIE on such a geometry is discretized using the Nyström method based on the Trapezoidal quadrature rule, the resulting scheme tends to converge only slowly, due to the singularity in the kernel function. The key finding of the manuscript is that the convergence order can be greatly improved by modifying only a very small number of elements in the coefficient matrix. Specifically, it is demonstrated that by correcting only the diagonal entries in the coefficient matrix, $O(h^{3})$ convergence can be attained for the single and double layer potentials associated with both the Laplace and the Helmholtz kernels. A nine-point correction stencil leads to an $O(h^5)$ scheme. The method proposed can be viewed as a generalization of the quadrature rule of Duan and Rokhlin, which was designed for the 2D Lippmann-Schwinger equation in the plane. The techniques proposed are supported by a rigorous error analysis that relies on Wigner-type limits involving the Epstein zeta function and its parametric derivatives.

math.NA↗

arXiv2020

Computing rank-revealing factorizations of matrices stored out-of-core

This paper describes efficient algorithms for computing rank-revealing factorizations of matrices that are too large to fit in RAM, and must instead be stored on slow external memory devices such as solid-state or spinning disk hard drives (out-of-core or out-of-memory). Traditional algorithms for computing rank revealing factorizations, such as the column pivoted QR factorization, or techniques for computing a full singular value decomposition of a matrix, are very communication intensive. They are naturally expressed as a sequence of matrix-vector operations, which become prohibitively expensive when data is not available in main memory. Randomization allows these methods to be reformulated so that large contiguous blocks of the matrix can be processed in bulk. The paper describes two distinct methods. The first is a blocked version of column pivoted Householder QR, organized as a "left-looking" method to minimize the number of write operations (which are more expensive than read operations on a spinning disk drive). The second method results in a so called UTV factorization which expresses a matrix $A$ as $A = U T V^*$ where $U$ and $V$ are unitary, and $T$ is triangular. This method is organized as an algorithm-by-blocks, in which floating point operations overlap read and write operations. The second method incorporates power iterations, and is exceptionally good at revealing the numerical rank; it can often be used as a substitute for a full singular value decomposition. Numerical experiments demonstrate that the new algorithms are almost as fast when processing data stored on a hard drive as traditional algorithms are for data stored in main memory. To be precise, the computational time for fully factorizing an $n\times n$ matrix scales as $cn^{3}$, with a scaling constant $c$ that is only marginally larger when the matrix is stored out of core.

cs.MS↗

arXiv2019

Efficient nuclear norm approximation via the randomized UTV algorithm

The recently introduced algorithm randUTV provides a highly efficient technique for computing accurate approximations to all the singular values of a given matrix $A$. The original version of randUTV was designed to compute a full factorization of the matrix in the form $A = UTV^*$ where $U$ and $V$ are orthogonal matrices, and $T$ is upper triangular. The estimates to the singular values of $A$ appear along the diagonal of $T$. This manuscript describes how the randUTV algorithm can be modified when the only quantity of interest being sought is the vector of approximate singular values. The resulting method is particularly effective for computing the nuclear norm of $A$, or more generally, other Schatten-$p$ norms. The report also describes how to compute an estimate of the errors incurred, at essentially negligible cost.

math.NA↗

arXiv2019

Randomized methods for matrix computations

The purpose of this text is to provide an accessible introduction to a set of recently developed algorithms for factorizing matrices. These new algorithms attain high practical speed by reducing the dimensionality of intermediate computations using randomized projections. The algorithms are particularly powerful for computing low-rank approximations to very large matrices, but they can also be used to accelerate algorithms for computing full factorizations of matrices. A key competitive advantage of the algorithms described is that they require less communication than traditional deterministic methods.

math.NA↗

arXiv2018

The PowerURV algorithm for computing rank-revealing full factorizations

Many applications in scientific computing and data science require the computation of a rank-revealing factorization of a large matrix. In many of these instances the classical algorithms for computing the singular value decomposition are prohibitively computationally expensive. The randomized singular value decomposition can often be helpful, but is not effective unless the numerical rank of the matrix is substantially smaller than the dimensions of the matrix. We introduce a new randomized algorithm for producing rank-revealing factorizations based on existing work by Demmel, Dumitriu and Holtz [Numerische Mathematik, 108(1), 2007] that excels in this regime. The method is exceptionally easy to implement, and results in close-to optimal low-rank approximations to a given matrix. The vast majority of floating point operations are executed in level-3 BLAS, which leads to high computational speeds. The performance of the method is illustrated via several numerical experiments that directly compare it to alternative techniques such as the column pivoted QR factorization, or the QLP method by Stewart.

math.NA↗

arXiv2018

HPS Accelerated Spectral Solvers for Time Dependent Problems

A high-order convergent numerical method for solving linear and non-linear parabolic PDEs is presented. The time-stepping is done via an explicit, singly diagonally implicit Runge-Kutta (ESDIRK) method of order 4 or 5, and for the implicit solve, we use the recently developed "Hierarchial Poincare-Steklov (HPS)" method. The HPS method combines a multidomain spectral collocation discretization technique (a "patching method") with a nested-dissection type direct solver. In the context under consideration, the elliptic solve required in each time-step involves the same coefficient matrix, which makes the use of a direct solver particularly effective. The manuscript describes the methodology and presents numerical experiments.

math.NA↗

arXiv2017

randUTV: A blocked randomized algorithm for computing a rank-revealing UTV factorization

This manuscript describes the randomized algorithm randUTV for computing a so called UTV factorization efficiently. Given a matrix $A$, the algorithm computes a factorization $A = UTV^{*}$, where $U$ and $V$ have orthonormal columns, and $T$ is triangular (either upper or lower, whichever is preferred). The algorithm randUTV is developed primarily to be a fast and easily parallelized alternative to algorithms for computing the Singular Value Decomposition (SVD). randUTV provides accuracy very close to that of the SVD for problems such as low-rank approximation, solving ill-conditioned linear systems, determining bases for various subspaces associated with the matrix, etc. Moreover, randUTV produces highly accurate approximations to the singular values of $A$. Unlike the SVD, the randomized algorithm proposed builds a UTV factorization in an incremental, single-stage, and non-iterative way, making it possible to halt the factorization process once a specified tolerance has been met. Numerical experiments comparing the accuracy and speed of randUTV to the SVD are presented. These experiments demonstrate that in comparison to column pivoted QR, which is another factorization that is often used as a relatively economic alternative to the SVD, randUTV compares favorably in terms of speed while providing far higher accuracy.

math.NA↗

arXiv2016

An accelerated Poisson solver based on multidomain spectral discretization

This paper presents a numerical method for variable coefficient elliptic PDEs with mostly smooth solutions on two dimensional domains. The PDE is discretized via a multi-domain spectral collocation method of high local order (order 30 and higher have been tested and work well). Local mesh refinement results in highly accurate solutions even in the presence of local irregular behavior due to corner singularities, localized loads, etc. The system of linear equations attained upon discretization is solved using a direct (as opposed to iterative) solver with $O(N^{1.5})$ complexity for the factorization stage and $O(N \log N)$ complexity for the solve. The scheme is ideally suited for executing the elliptic solve required when parabolic problems are discretized via time-implicit techniques. In situations where the geometry remains unchanged between time-steps, very fast execution speeds are obtained since the solution operator for each implicit solve can be pre-computed.

math.NA↗