arXiv subjects

Sivaram Ambikasaran

Publications and source records attributed to Sivaram Ambikasaran.

At least 19 recordsLinked to original sources

arXiv2026

LoCCA: Localized Chebyshev Cross Approximation for Kernel Matrix Factorization via Nodal Perturbation Stability

We propose Local Chebyshev Cross-Approximation (LoCCA), a data-driven framework that bridges smooth polynomial interpolation with flexible matrix factorizations. LoCCA dynamically maps ideal grid nodes to their nearest physical neighbors within unstructured point sets. We support this framework with a new perturbation theory, proving that displacing Chebyshev nodes onto physical data points retains mathematical stability and accuracy without explosive error growth. Numerically, LoCCA provides strict, reliable error control even on irregular geometries where standard grid-based methods fail, achieving substantial speedups and near-optimal matrix compression.

math.NA↗

arXiv2026

Continuous Cross Approximation of Matrices Arising Out of Kernel Functions

We propose a residual energy-based framework for constructing low-rank approximations of kernel matrices arising from continuous kernel functions. The method operates in a continuous setting and is based on the adaptive selection of pivot nodes, referred to as \emph{optimal nodes}, which are chosen to minimize the residual energy at each step. This leads to a sequence of rank-$1$ updates of the residual kernel and admits a natural interpretation as a continuous analog of Adaptive Cross Approximation (ACA). From a theoretical perspective, we show that the residual kernels remain in the class of compact operators and that the approximation error is exactly characterized by the residual energy. We provide convergence guarantees showing that the method yields monotonic error reduction under an alignment condition and achieves geometric decay under practically motivated assumptions. Extensive numerical experiments demonstrate that the proposed method achieves approximation errors close to those of the truncated singular value decomposition across a range of kernel functions. The method exhibits strong robustness with respect to sampling and maintains stable performance across different discretizations. Furthermore, the close agreement between the continuous residual energy and the discrete approximation error highlights the consistency of the formulation. These results establish the proposed approach as a theoretically grounded, practically effective continuous counterpart to classical cross-approximation techniques.

math.NA↗

arXiv2025

Rank of Matrices Arising out of Singular Kernel Functions

Kernel functions are frequently encountered in differential equations and machine learning applications. In this work, we study the rank of matrices arising out of the kernel function $K: X \times Y \mapsto \mathbb{R}$, where the sets $X, Y \in \mathbb{R}^d$ are hypercubes that share a boundary. The main contribution of this work is the analysis of the rank of such matrices where the particles (sources/targets) are arbitrarily distributed within these hypercubes. To our knowledge, this is the first work to formally investigate the rank of such matrices for an arbitrary distribution of particles. We model the arbitrary distribution of particles to arise from an underlying random distribution and obtain bounds on the expected rank and variance of the rank of the kernel matrix corresponding to various neighbor interactions. These bounds are useful for understanding the performance and complexity of hierarchical matrix algorithms (especially hierarchical matrices satisfying the weak-admissibility criterion) for an arbitrary distribution of particles. We also present numerical experiments in one-, two-, and three-dimensions, showing the expected rank growth and variance of the rank for different types of interactions. The numerical results, not surprisingly, align with our theoretical predictions.

math.NA↗

arXiv2024

New Algebraic Fast Algorithms for $N$-body Problems in Two and Three Dimensions

We present two new algebraic multilevel hierarchical matrix algorithms to perform fast matrix-vector product (MVP) for $N$-body problems in $d$ dimensions, namely efficient $\mathcal{H}^2_{*}$ (fully nested algorithm, i.e., $\mathcal{H}^2$ matrix-like algorithm) and $(\mathcal{H}^2 + \mathcal{H})_{*}$ (semi-nested algorithm, i.e., cross of $\mathcal{H}^2$ and $\mathcal{H}$ matrix-like algorithms). The efficient $\mathcal{H}^2_{*}$ and $(\mathcal{H}^2 + \mathcal{H})_{*}$ hierarchical representations are based on our recently introduced weak admissibility condition in higher dimensions, where the admissible clusters are the far-field and the vertex-sharing clusters. Due to the use of nested form of the bases, the proposed hierarchical matrix algorithms are more efficient than the non-nested algorithms ($\mathcal{H}$ matrix algorithms). We rely on purely algebraic low-rank approximation techniques (e.g., ACA and NCA) and develop both algorithms in a black-box fashion. Another noteworthy contribution of this article is that we perform a comparative study of the proposed algorithms with different algebraic (NCA or ACA-based compression) fast MVP algorithms in $2$D and $3$D. The fast algorithms are tested on various kernel matrices and applied to get fast iterative solutions of a dense linear system arising from the discretized integral equations and radial basis function interpolation. Notably, all the algorithms are developed in a similar fashion in $\texttt{C++}$ and tested within the same environment, allowing for meaningful comparisons. The numerical results demonstrate that the proposed algorithms are competitive to the NCA-based standard $\mathcal{H}^2$ matrix algorithm with respect to the memory and time. The C++ implementation of the proposed algorithms is available at https://github.com/riteshkhan/H2weak/.

math.NA↗

arXiv2024

A hybrid interpolation ACA accelerated method for parabolic boundary integral operators

We consider piecewise polynomial discontinuous Galerkin discretizations of boundary integral reformulations of the heat equation. The resulting linear systems are dense and block-lower triangular and hence can be solved by block forward elimination. For the fast evaluation of the history part, the matrix is subdivided into a family of sub-matrices according to the temporal separation. Separated blocks are approximated by Chebyshev interpolation of the heat kernel in time. For the spatial variable, we propose an adaptive cross approximation (ACA) framework to obtain a data-sparse approximation of the entire matrix. We analyse how the ACA tolerance must be adjusted to the temporal separation and present numerical results for a benchmark problem to confirm the theoretical estimates.

math.NA↗

arXiv2023

HODLR$d$D: A new Black-box fast algorithm for $N$-body problems in $d$-dimensions with guaranteed error bounds

In this article, we prove new theorems bounding the rank of different sub-matrices arising from these kernel functions. Bounds like these are often useful for analyzing the complexity of various hierarchical matrix algorithms. We also plot the numerical rank growth of different sub-matrices arising out of various kernel functions in $1$D, $2$D, $3$D and $4$D, which, not surprisingly, agrees with the proposed theorems. Another significant contribution of this article is that, using the obtained rank bounds, we also propose a way to extend the notion of \textbf{\emph{weak-admissibility}} for hierarchical matrices in higher dimensions. Based on this proposed \textbf{\emph{weak-admissibility}} condition, we develop a black-box (kernel-independent) fast algorithm for $N$-body problems, hierarchically off-diagonal low-rank matrix in $d$ dimensions (HODLR$d$D), which can perform matrix-vector products with $\mathcal{O}(pN \log (N))$ complexity in any dimension $d$, where $p$ doesn't grow with any power of $N$. More precisely, our theorems guarantee that $p \in \mathcal{O} (\log (N) \log^d (\log (N)))$, which implies our HODLR$d$D algorithm scales almost linearly. The $\texttt{C++}$ implementation with \texttt{OpenMP} parallelization of the HODLR$d$D is available at \url{https://github.com/SAFRAN-LAB/HODLRdD}. We also discuss the scalability of the HODLR$d$D algorithm and showcase the applicability by solving an integral equation in $4$ dimensions and accelerating the training phase of the support vector machines (SVM) for the data sets with four and five features.

math.NA↗

arXiv2023

A new Nested Cross Approximation

In this article, we present a new Nested Cross Approximation (NNCA) for constructing H2 matrices. It differs from the existing NCAs~\cite{bebendorf2012constructing, zhao2019fast} in the technique of choosing pivots, a key part of the approximation. Our technique of choosing pivots is purely algebraic and involves only a single tree traversal. We demonstrate its applicability by developing a fast H2 matrix-vector product, that uses NNCA for the appropriate low-rank approximations. We illustrate the timing profiles and the accuracy of NNCA based H2 matrix-vector product. We also provide a comparison of NNCA based H2 matrix-vector product with the existing NCA based H2 matrix-vector products. A key observation is that NNCA performs better than the existing NCAs. In addition, using the NNCA based H2 matrix-vector product, we accelerate i) solving an integral equation in 3D and ii) Support Vector Machine (SVM). In the spirit of reproducible computational science, the implementation of the algorithm developed in this article is made available at \url{https://github.com/SAFRAN-LAB/NNCA}.

math.NA↗

arXiv2023

HODLR3D: Hierarchical matrices for $N$-body problems in three dimensions

This article introduces HODLR3D, a class of hierarchical matrices arising out of $N$-body problems in three dimensions. HODLR3D relies on the fact that certain off-diagonal matrix sub-blocks arising out of the $N$-body problems in three dimensions are numerically low-rank. For the Laplace kernel in $3$D, which is widely encountered, we prove that all the off-diagonal matrix sub-blocks are rank deficient in finite precision. We also obtain the growth of the rank as a function of the size of these matrix sub-blocks. For other kernels in three dimensions, we numerically illustrate a similar scaling in rank for the different off-diagonal sub-blocks. We leverage this hierarchical low-rank structure to construct HODLR3D representation, with which we accelerate matrix-vector products. The storage and computational complexity of the HODLR3D matrix-vector product scales almost linearly with system size. We demonstrate the computational performance of HODLR3D representation through various numerical experiments. Further, we explore the performance of the HODLR3D representation on distributed memory systems. HODLR3D, described in this article, is based on a weak admissibility condition. Among the hierarchical matrices with different weak admissibility conditions in $3$D, only in HODLR3D did the rank of the admissible off-diagonal blocks not scale with any power of the system size. Thus, the storage and the computational complexity of the HODLR3D matrix-vector product remain tractable for $N$-body problems with large system sizes.

math.NA↗

arXiv2023

A new Directional Algebraic Fast Multipole Method based iterative solver for the Lippmann-Schwinger equation accelerated with HODLR preconditioner

We present a fast iterative solver for scattering problems in 2D, where a penetrable object with compact support is considered. By representing the scattered field as a volume potential in terms of the Green's function, we arrive at the Lippmann-Schwinger equation in integral form, which is then discretized using an appropriate quadrature technique. The discretized linear system is then solved using an iterative solver accelerated by Directional Algebraic Fast Multipole Method (DAFMM). The DAFMM presented here relies on the directional admissibility condition of the 2D Helmholtz kernel. And the construction of low-rank factorizations of the appropriate low-rank matrix sub-blocks is based on our new Nested Cross Approximation (NCA)~\cite{ arXiv:2203.14832 [math.NA]}. The advantage of our new NCA is that the search space of so-called far-field pivots is smaller than that of the existing NCAs. Another significant contribution of this work is the use of HODLR based direct solver as a preconditioner to further accelerate the iterative solver. In one of our numerical experiments, the iterative solver does not converge without a preconditioner. We show that the HODLR preconditioner is capable of solving problems that the iterative solver can not. Another noteworthy contribution of this article is that we perform a comparative study of the HODLR based fast direct solver, DAFMM based fast iterative solver, and HODLR preconditioned DAFMM based fast iterative solver for the discretized Lippmann-Schwinger problem. To the best of our knowledge, this work is one of the first to provide a systematic study and comparison of these different solvers for various problem sizes and contrast functions. In the spirit of reproducible computational science, the implementation of the algorithms developed in this article is made available at \url{https://github.com/vaishna77/Lippmann_Schwinger_Solver}.

math.NA↗

arXiv2023

Algebraic Inverse Fast Multipole Method: A fast direct solver that is better than HODLR based fast direct solver

This article presents a fast direct solver, termed Algebraic Inverse Fast Multipole Method (from now on abbreviated as AIFMM), for linear systems arising out of $N$-body problems. AIFMM relies on the following three main ideas: (i) Certain sub-blocks in the matrix corresponding to $N$-body problems can be efficiently represented as low-rank matrices; (ii) The low-rank sub-blocks in the above matrix are leveraged to construct an extended sparse linear system; (iii) While solving the extended sparse linear system, certain fill-ins that arise in the elimination phase are represented as low-rank matrices and are "redirected" though other variables maintaining zero fill-in sparsity. The main highlights of this article are the following: (i) Our method is completely algebraic (as opposed to the existing Inverse Fast Multipole Method~\cite{ arXiv:1407.1572,doi:10.1137/15M1034477,TAKAHASHI2017406}, from now on abbreviated as IFMM). We rely on our new Nested Cross Approximation~\cite{arXiv:2203.14832} (from now on abbreviated as NNCA) to represent the matrix arising out of $N$-body problems. (ii) A significant contribution is that the algorithm presented in this article is more efficient than the existing IFMMs. In the existing IFMMs, the fill-ins are compressed and redirected as and when they are created. Whereas in this article, we update the fill-ins first without affecting the computational complexity. We then compress and redirect them only once. (iii) Another noteworthy contribution of this article is that we provide a comparison of AIFMM with Hierarchical Off-Diagonal Low-Rank (from now on abbreviated as HODLR) based fast direct solver and NNCA powered GMRES based fast iterative solver. (iv) Additionally, AIFMM is also demonstrated as a preconditioner.

math.NA↗

arXiv2022

Spectrum of MATLABs magic squares

This article looks at the eigenvalues of magic squares generated by the MATLAB's magic($n$) function. The magic($n$) function constructs doubly even ($n = 4k$) magic squares, singly even ($n = 4k+2$) magic squares and odd ($n = 2k+1$) magic squares using different algorithms. The doubly even magic squares are constructed by a criss-cross method that involves reflecting the entries of a simple square about the center. The odd magic squares are constructed using the Siamese method. The singly even magic squares are constructed using a lower-order odd magic square (Strachey method). We obtain approximations of eigenvalues of odd and singly even magic squares and prove error bounds on the approximation. For the sake of completeness, we also obtain the eigenpairs of doubly even magic squares generated by MATLAB. The approximation of the spectra involves some interesting connections with the spectrum of g-circulant matrices and the use of Bauer-Fike theorem.

math.GM↗

arXiv2022

HODLR2D: A new class of Hierarchical matrices

This article introduces HODLR2D, a new hierarchical low-rank representation for a class of dense matrices arising out of $N$ body problems in two dimensions. Using this new hierarchical framework, we propose a new fast matrix-vector product that scales almost linearly. We apply this fast matrix-vector product to accelerate the iterative solution of large dense linear systems arising out of radial basis function interpolation and discretized integral equation. The space and computational complexity of HODLR2D matrix-vector products scales as $\mathcal{O}(pN \log(N))$, where $p$ is the maximum rank of the compressed matrix subblocks. We also prove that $p \in \mathcal{O}(\log(N)\log(\log(N)))$, which ensures that the storage and computational complexity of HODLR2D matrix-vector products remain tractable for large $N$. Additionally, we also present the parallel scalability of HODLR2D as part of this article.

math.NA↗

arXiv2017

Fast and scalable Gaussian process modeling with applications to astronomical time series

The growing field of large-scale time domain astronomy requires methods for probabilistic data analysis that are computationally tractable, even with large datasets. Gaussian Processes are a popular class of models used for this purpose but, since the computational cost scales, in general, as the cube of the number of data points, their application has been limited to small datasets. In this paper, we present a novel method for Gaussian Process modeling in one-dimension where the computational requirements scale linearly with the size of the dataset. We demonstrate the method by applying it to simulated and real astronomical time series datasets. These demonstrations are examples of probabilistic inference of stellar rotation periods, asteroseismic oscillation spectra, and transiting planet parameters. The method exploits structure in the problem when the covariance function is expressed as a mixture of complex exponentials, without requiring evenly spaced observations or uniform noise. This form of covariance arises naturally when the process is a mixture of stochastically-driven damped harmonic oscillators -- providing a physical motivation for and interpretation of this choice -- but we also demonstrate that it can be a useful effective model in some other cases. We present a mathematical description of the method and compare it to existing scalable Gaussian Process methods. The method is fast and interpretable, with a range of potential applications within astronomical data analysis and beyond. We provide well-tested and documented open-source implementations of this method in C++, Python, and Julia.

astro-ph.IM↗

arXiv2016

Fast symmetric factorization of hierarchical matrices with applications

We present a fast direct algorithm for computing symmetric factorizations, i.e. $A = WW^T$, of symmetric positive-definite hierarchical matrices with weak-admissibility conditions. The computational cost for the symmetric factorization scales as $\mathcal{O}(n \log^2 n)$ for hierarchically off-diagonal low-rank matrices. Once this factorization is obtained, the cost for inversion, application, and determinant computation scales as $\mathcal{O}(n \log n)$. In particular, this allows for the near optimal generation of correlated random variates in the case where $A$ is a covariance matrix. This symmetric factorization algorithm depends on two key ingredients. First, we present a novel symmetric factorization formula for low-rank updates to the identity of the form $I+UKU^T$. This factorization can be computed in $\mathcal{O}(n)$ time if the rank of the perturbation is sufficiently small. Second, combining this formula with a recursive divide-and-conquer strategy, near linear complexity symmetric factorizations for hierarchically structured matrices can be obtained. We present numerical results for matrices relevant to problems in probability \& statistics (Gaussian processes), interpolation (Radial basis functions), and Brownian dynamics calculations in fluid mechanics (the Rotne-Prager-Yamakawa tensor).

math.NA↗

arXiv2016

An accurate, fast, mathematically robust, universal, non-iterative algorithm for computing multi-component diffusion velocities

Using accurate multi-component diffusion treatment in numerical combustion studies remains formidable due to the computational cost associated with solving for diffusion velocities. To obtain the diffusion velocities, for low density gases, one needs to solve the Stefan-Maxwell equations along with the zero diffusion flux criteria, which scales as $\mathcal{O}(N^3)$, when solved exactly. In this article, we propose an accurate, fast, direct and robust algorithm to compute multi-component diffusion velocities. To our knowledge, this is the first provably accurate algorithm (the solution can be obtained up to an arbitrary degree of precision) scaling at a computational complexity of $\mathcal{O}(N)$ in finite precision. The key idea involves leveraging the fact that the matrix of the reciprocal of the binary diffusivities, $V$, is low rank, with its rank being independent of the number of species involved. The low rank representation of matrix $V$ is computed in a fast manner at a computational complexity of $\mathcal{O}(N)$ and the Sherman-Morrison-Woodbury formula is used to solve for the diffusion velocities at a computational complexity of $\mathcal{O}(N)$. Rigorous proofs and numerical benchmarks illustrate the low rank property of the matrix $V$ and scaling of the algorithm.

physics.chem-ph↗

arXiv2015

Fast, adaptive, high order accurate discretization of the Lippmann-Schwinger equation in two dimension

We present a fast direct solver for two dimensional scattering problems, where an incident wave impinges on a penetrable medium with compact support. We represent the scattered field using a volume potential whose kernel is the outgoing Green's function for the exterior domain. Inserting this representation into the governing partial differential equation, we obtain an integral equation of the Lippmann-Schwinger type. The principal contribution here is the development of an automatically adaptive, high-order accurate discretization based on a quad tree data structure which provides rapid access to arbitrary elements of the discretized system matrix. This permits the straightforward application of state-of-the-art algorithms for constructing compressed versions of the solution operator. These solvers typically require $O(N^{3/2})$ work, where $N$ denotes the number of degrees of freedom. We demonstrate the performance of the method for a variety of problems in both the low and high frequency regimes.

math.NA↗

arXiv2015

Generalized Rybicki Press algorithm

This article discusses a more general and numerically stable Rybicki Press algorithm, which enables inverting and computing determinants of covariance matrices, whose elements are sums of exponentials. The algorithm is true in exact arithmetic and relies on introducing new variables and corresponding equations, thereby converting the matrix into a banded matrix of larger size. Linear complexity banded algorithms for solving linear systems and computing determinants on the larger matrix enable linear complexity algorithms for the initial semi-separable matrix as well. Benchmarks provided illustrate the linear scaling of the algorithm.

math.NA↗

arXiv2015

Fast Direct Methods for Gaussian Processes

A number of problems in probability and statistics can be addressed using the multivariate normal (Gaussian) distribution. In the one-dimensional case, computing the probability for a given mean and variance simply requires the evaluation of the corresponding Gaussian density. In the $n$-dimensional setting, however, it requires the inversion of an $n \times n$ covariance matrix, $C$, as well as the evaluation of its determinant, $\det(C)$. In many cases, such as regression using Gaussian processes, the covariance matrix is of the form $C = σ^2 I + K$, where $K$ is computed using a specified covariance kernel which depends on the data and additional parameters (hyperparameters). The matrix $C$ is typically dense, causing standard direct methods for inversion and determinant evaluation to require $\mathcal O(n^3)$ work. This cost is prohibitive for large-scale modeling. Here, we show that for the most commonly used covariance functions, the matrix $C$ can be hierarchically factored into a product of block low-rank updates of the identity matrix, yielding an $\mathcal O (n\log^2 n) $ algorithm for inversion. More importantly, we show that this factorization enables the evaluation of the determinant $\det(C)$, permitting the direct calculation of probabilities in high dimensions under fairly broad assumptions on the kernel defining $K$. Our fast algorithm brings many problems in marginalization and the adaptation of hyperparameters within practical reach using a single CPU core. The combination of nearly optimal scaling in terms of problem size with high-performance computing resources will permit the modeling of previously intractable problems. We illustrate the performance of the scheme on standard covariance kernels.

math.NA↗