arXiv subjects

Kirk M. Soodhalter

Publications and source records attributed to Kirk M. Soodhalter.

At least 19 recordsLinked to original sources

arXiv2026

Constraints on admissible behavior of GMRES applied to tridiagonal Toeplitz systems

The result of Greenbaum and Strakoš that for a given set of eigenvalues, any convergence curve is possible [SIMAX 1996] and the subsequent parameterization of such matrix-right-hand side pairs $(A,\mathbf{b})$ of Arioli, Pták, and Strakoš [BIT 1998] demonstrated that the behavior of the \gmres could not be completely characterized by the eigenvalues of $A$ alone. In this paper, we consider how to use this theory to understand the admissible and attainable \gmres behavior for matrices with constrained structure, focussing on non-Hermitian (non-symmetric) tridiagonal Toeplitz matrices. We show that Toeptliz structure necessarily constrains the how the theory from these papers can manifest but that a continuum of admissible behaviors is still attainable.

math.NA↗

arXiv2026

A block Recycled GMRES method with investigations into aspects of solver performance

We propose a block Krylov subspace version of the GCRO-DR method proposed in [Parks et al.; SISC 2005], which is an iterative method allowing for the efficient minimization of the the residual over an augmented Krylov subspace. We offer a clean derivation of our proposed method and discuss methods of selecting recycling subspaces at restart as well as implementation decisions in the context of high-performance computing. Numerical experiments are split into those demonstrating convergence properties and those demonstrating the data movement and cache efficiencies of the dominant operations of the method, measured using processor monitoring code from Intel.

math.NA↗

arXiv2026

Admissible and attainable convergence behavior with stagnation mirroring in restarted (block) GMRES

In this work, we describe how to construct matrices and block right-hand sides that exhibit a specified restarted block \gmres convergence pattern, such that the eigenvalues and Ritz values at each iteration can be chosen independent of the specified convergence behavior. This work is a generalization of the work in [Meurant and Tebbens, Num.~Alg.~2019] in which the authors do the same for restarted non-block \gmres. We use the same tools as were used in [Kubínová and Soodhalter, SIMAX 2020], namely to analyze block \gmres as an iteration over a right vector space with scalars from the $^\ast$-algebra of matrices. To facilitate our work, we also extend the work of Meurant and Tebbens and offer alternative proofs of some of their results, that can be more easily generalized to the block setting.

math.NA↗

arXiv2026

Hybrid CG-Tikhonov is a filtration of the CG Lanczos vectors

We consider iterative methods for solving linear ill-posed problems with compact operator and right-hand side only available via noise-polluted measurements. Conjugate gradients (CG) applied to the normal equations with an appropriate stopping rule and CG applied to the system solving for a Tikhonov-regularized solution (CGT) $(A^\ast A + c I_{\mathcal{X}}) x^{(δ,c)} = A^\ast y^δ$ are closely related regularization methods that build iterates from the same Krylov subspaces. In this work, we show that the CGT iterate can be expressed as $ x^{(δ,c)}_m = \sum_{i=1}^{m} γ^{(m)}_i(c) z_i^{(m)}v_i, $ where $\left\lbraceγ_i^{(m)}(c)\right\rbrace_{i=1}^m$ are functions of the Tikhonov parameter $c$ and $x^{(δ)}_m = \sum_{i=1}^{m} z_i^{(m)}v_i$ is the $m$-th CG iterate. We call these functions Lanczos filters, and they can be shown to have decay properties as $c\rightarrow\infty$ with the speed of decay increasing with $i$. This has the effect of filtering out the contribution of the later terms of the CG iterate. The filters can be constructed using quantities defined via recursions at each iteration. We demonstrate with numerical experiments that good parameter choices correspond to appropriate damping of the Lanczos vectors. The filtration approach also provides a platform for further development of parameter choice rules, and similar representations may hold for other hybrid iterative schemes.

math.NA↗

arXiv2025

The quantum super-Krylov method

The problem of estimating the ground-state energy of a quantum system is ubiquitous in chemistry and condensed matter physics. Krylov quantum diagonalization (KQD) has emerged as a promising approach for this task. However, many KQD methods rely on subroutines, particularly the Hadamard test, that are challenging to implement on near-term quantum computers. We present a novel KQD method that uses only real-time evolutions and recovery probabilities, making it well adapted for existing quantum hardware. The method entails numerical differentiation in post-processing, and so we present a novel derivative estimation algorithm that is robust to noisy data. Under assumptions on the spectrum of the Hamiltonian, we prove that our algorithm converges exponentially quickly to the ground-state energy and present a numerical demonstration using tensor network simulations.

quant-ph↗

arXiv2025

Krylov Subspace Recycling For Matrix Functions

We derive an augmented Krylov subspace method with subspace recycling for computing a sequence of matrix function applications on a set of vectors. The matrix is either fixed or changes as the sequence progresses. We assume consecutive matrices are closely related, but make no assumptions on the relationship between the vectors. We present three versions of the method with different practical implementations. We demonstrate the effectiveness of the method using a range of numerical experiments with a selection of functions and matrices. We primarily focus our attention on the sign function arising in the overlap formalism of lattice QCD.

math.NA↗

arXiv2025

Chebyshev HOPGD with sparse grid sampling for parameterized linear systems

We consider approximating solutions to parameterized linear systems of the form $A(μ_1,μ_2) x(μ_1,μ_2) = b$. Here the matrix $A(μ_1,μ_2) \in \mathbb{R}^{n \times n}$ is nonsingular, large, and sparse and depends nonlinearly on the parameters. Specifically, the system arises from a discretization of a partial differential equation and $x(μ_1,μ_2) \in \mathbb{R}^n$, $b \in \mathbb{R}^n$. The treatment of linear systems with nonlinear dependence on a single parameter has been well-studied, and robust methods combining companion linearization, Krylov subspace methods, and Chebyshev interpolation have enabled fast solution for multiple parameter values at the cost of a single iteration. Solution of systems depending nonlinearly on multiple parameters is more challenging. This work overcomes those additional challenges by combining companion linearization, the Krylov subspace method preconditioned bi-conjugate gradient (BiCG), and a decomposition of a tensor matrix of precomputed solutions, called snapshots. This produces a reduced order model of $x(μ_1,μ_2)$, and this model can be evaluated inexpensively for many values of the parameters. An interpolation of the model is used to produce approximations on the entire parameter space. In addition this method can be used to solve a parameter estimation problem. This approach allows us to achieve similar computational savings as for the one-parameter case; we can solve for many parameter pairs at the cost of many fewer applications of an efficient iterative method. The technique is presented for dependence on two parameters, but the strategy can be extended to more parameters using the same approach. Numerical examples of a parameterized Helmholtz equation show the competitiveness of our approach.

math.NA↗

arXiv2024

GMRES with randomized sketching and deflated restarting

We present a new Krylov subspace recycling method for solving a linear system of equations, or a sequence of slowly changing linear systems. Our approach is to reduce the computational overhead of recycling techniques while still benefiting from the acceleration afforded by such techniques. As such, this method augments an unprojected Krylov subspace. Furthermore, it combines randomized sketching and deflated restarting in a way that avoids orthogononalizing a full Krylov basis. We call this new method GMRES-SDR (sketched deflated restarting). With this new method, we provide new theory, which initially characterizes unaugmented sketched GMRES as a projection method for which the projectors involve the sketching operator. We demonstrate that sketched GMRES and its sibling method sketched FOM are an MR/OR pairing, just like GMRES and FOM. We furthermore obtain residual convergence estimates. Building on this, we characterize GMRES-SDR also in terms of sketching-based projectors. Compression of the augmented Krylov subspace for recycling is performed using a sketched version of harmonic Ritz vectors. We present results of numerical experiments demonstrating the effectiveness of GMRES-SDR over competitor methods such as GMRES-DR and GCRO-DR.

math.NA↗

arXiv2024

Fast solution of Sylvester-structured systems for spatial source separation of the Cosmic Microwave Background

Implementation of many statistical methods for large, multivariate data sets requires one to solve a linear system that, depending on the method, is of the dimension of the number of observations or each individual data vector. This is often the limiting factor in scaling the method with data size and complexity. In this paper we illustrate the use of Krylov subspace methods to address this issue in a statistical solution to a source separation problem in cosmology where the data size is prohibitively large for direct solution of the required system. Two distinct approaches, adapted from techniques in the literature, are described: one that uses the method of conjugate gradients directly to the Kronecker-structured problem and another that reformulates the system as a Sylvester matrix equation. We show that both approaches produce an accurate solution within an acceptable computation time and with practical memory requirements for the data size that is currently available.

math.NA↗

arXiv2024

The effect of approximate coarsest-level solves on the convergence of multigrid V-cycle methods

The multigrid V-cycle method is a popular method for solving systems of linear equations. It computes an approximate solution by using smoothing on fine levels and solving a system of linear equations on the coarsest level. Solving on the coarsest level depends on the size and difficulty of the problem. If the size permits, it is typical to use a direct method based on LU or Cholesky decomposition. In settings with large coarsest-level problems, approximate solvers such as iterative Krylov subspace methods, or direct methods based on low-rank approximation, are often used. The accuracy of the coarsest-level solver is typically determined based on the experience of the users with the concrete problems and methods. In this paper we present an approach to analyzing the effects of approximate coarsest-level solves on the convergence of the V-cycle method for symmetric positive definite problems. Using these results, we derive coarsest-level stopping criterion through which we may control the difference between the approximation computed by a V-cycle method with approximate coarsest-level solver and the approximation which would be computed if the coarsest-level problems were solved exactly. The coarsest-level stopping criterion may thus be set up such that the V-cycle method converges to a chosen finest-level accuracy in (nearly) the same number of V-cycle iterations as the V-cycle method with exact coarsest-level solver. We also utilize the theoretical results to discuss how the convergence of the V-cycle method may be affected by the choice of a tolerance in a coarsest-level stopping criterion based on the relative residual norm.

math.NA↗

arXiv2023

Augmented unprojected Krylov subspace methods

Augmented Krylov subspace methods aid in accelerating the convergence of a standard Krylov subspace method by including additional vectors in the search space. A residual projection framework based on residual (Petrov-) Galerkin constraints was presented in [Gaul et al. SIAM J. Matrix Anal. Appl 2013], and later generalised in a recent survey on subspace recycling iterative methods [Soodhalter et al. GAMM-Mitt. 2020]. The framework describes augmented Krylov subspace methods in terms of applying a standard Krylov subspace method to an appropriately projected problem. In this work we show that the projected problem has an equivalent unprojected formulation, and that viewing the framework in this way provides a similar description for the class of unprojected augmented Krylov subspace methods. We derive the first unprojected augmented Full Orthogonalization Method (FOM), and demonstrate its effectiveness as a recycling method. We then show how the R$^{3}$GMRES algorithm fits within the framework. We show that unprojected augmented short recurrence methods fit within the framework, but can only be implemented in practice under certain conditions on the augmentation subspace. We demonstrate this using the Augmented Conjugate Gradient (AugCG) algorithm as an example.

math.NA↗

arXiv2022

A projected Nesterov-Kaczmarz approach to stellar population-kinematic distribution reconstruction in Extragalactic Archaeology

In this paper, we consider the problem of reconstructing a galaxy's stellar population-kinematic distribution function from optical integral field unit measurements. These quantities are connected via a high-dimensional integral equation. To solve this problem, we propose a projected Nesterov-Kaczmarz reconstruction (PNKR) method, which efficiently leverages the problem structure and incorporates physical prior information such as smoothness and non-negativity constraints. To test the performance of our reconstruction approach, we apply it to a dataset simulated from a known ground truth density, and validate it by comparing our recoveries to those obtained by the widely used pPXF software.

math.NA↗

arXiv2022

Preconditioned infinite GMRES for parameterized linear systems

We are interested in obtaining approximate solutions to parameterized linear systems of the form $A(μ) x(μ) = b$ for many values of the parameter $μ$. Here $A(μ)$ is large, sparse, and nonsingular, with a nonlinear analytic dependence on $μ$. Our approach is based on a companion linearization for parameterized linear systems. The companion matrix is similar to the operator in the infinite Arnoldi method, and we use this to adapt the flexible GMRES setting. In this way, our method returns a function $\tilde{x}(μ)$ which is cheap to evaluate for different $μ$, and the preconditioner is applied only approximately. This novel approach leads to increased freedom to carry out the action of the operation inexactly, which provides performance improvement over the method infinite GMRES, without a loss of accuracy in general. We show that the error of our method is estimated based on the magnitude of the parameter $μ$, the inexactness of the preconditioning, and the spectrum of the linear companion matrix. Numerical examples from a finite element discretization of a Helmholtz equation with a parameterized material coefficient illustrate the competitiveness of our approach. The simulations are reproducible and publicly available online.

math.NA↗

arXiv2022

A note on augmented unprojected Krylov subspace methods

Subspace recycling iterative methods and other subspace augmentation schemes are a successful extension to Krylov subspace methods in which a Krylov subspace is augmented with a fixed subspace spanned by vectors deemed to be helpful in accelerating convergence or conveying knowledge of the solution. Recently, a survey was published, in which a framework describing the vast majority of such methods was proposed [Soodhalter et al, GAMM-Mitt. 2020]. In many of these methods, the Krylov subspace is one generated by the system matrix composed with a projector that depends on the augmentation space. However, it is not a requirement that a projected Krylov subspace be used. There are augmentation methods built on using Krylov subspaces generated by the original system matrix, and these methods also fit into the general framework. In this note, we observe that one gains implementation benefits by considering such augmentation methods with unprojected Krylov subspaces in the general framework. We demonstrate this by applying the idea to the R$^3$GMRES method proposed in [Dong et al. ETNA 2014] to obtain a simplified implementation and to connect that algorithm to early augmentation schemes based on flexible preconditioning [Saad. SIMAX 1997].

math.NA↗

arXiv2021

Subspace Recycling-based Regularization Methods

Subspace recycling techniques have been used quite successfully for the acceleration of iterative methods for solving large-scale linear systems. These methods often work by augmenting a solution subspace generated iteratively by a known algorithm with a fixed subspace of vectors which are ``useful'' for solving the problem. Often, this has the effect of inducing a projected version of the original linear system to which the known iterative method is then applied, and this projection can act as a deflation preconditioner, accelerating convergence. Most often, these methods have been applied for the solution of well-posed problems. However, they have also begun to be considered for the solution of ill-posed problems. In this paper, we consider subspace augmentation-type iterative schemes applied to linear ill-posed problems in a continuous Hilbert space setting, based on a recently developed framework describing these methods. We show that under suitable assumptions, a recycling method satisfies the formal definition of a regularization, as long as the underlying scheme is itself a regularization. We then develop an augmented subspace version of the gradient descent method and demonstrate its effectiveness, both on an academic Gaussian blur model and on problems arising from the adaptive optics community for the resolution of large sky images by ground-based extremely large telescopes.

math.NA↗

arXiv2020

A survey of subspace recycling iterative methods

This survey concerns subspace recycling methods, a popular class of iterative methods that enable effective reuse of subspace information in order to speed up convergence and find good initial guesses over a sequence of linear systems with slowly changing coefficient matrices, multiple right-hand sides, or both. The subspace information that is recycled is usually generated during the run of an iterative method (usually a Krylov subspace method) on one or more of the systems. Following introduction of definitions and notation, we examine the history of early augmentation schemes along with deflation preconditioning schemes and their influence on the development of recycling methods. We then discuss a general residual constraint framework through which many augmented Krylov and recycling methods can both be viewed. We review several augmented and recycling methods within this framework. We then discuss some known effective strategies for choosing subspaces to recycle before taking the reader through more recent developments that have generalized recycling for (sequences of) shifted linear systems, some of them with multiple right-hand sides in mind. We round out our survey with a brief review of application areas that have seen benefit from subspace recycling methods.

math.NA↗

arXiv2020

Admissible and attainable convergence behavior of block Arnoldi and GMRES

It is well-established that any non-increasing convergence curve is possible for GMRES and a family of pairs $(A,b)$ can be constructed for which GMRES exhibits a given convergence curve with $A$ having arbitrary spectrum. No analog of this result has been established for block GMRES, wherein multiple right-hand sides are considered. By reframing the problem as a single linear system over a ring of square matrices, we develop convergence results for block Arnoldi and block GMRES. In particular, we show what convergence behavior is admissible for block GMRES and how the matrices and right-hand sides producing any admissible behavior can be constructed. Moreover, we show that the convergence of the block Arnoldi method for eigenvalue approximation can be almost fully independent of the convergence of block GMRES for the same coefficient matrix and the same starting vectors.

math.NA↗

arXiv2019

Error Analysis and Improving the Accuracy of Winograd Convolution for Deep Neural Networks

Popular deep neural networks (DNNs) spend the majority of their execution time computing convolutions. The Winograd family of algorithms can greatly reduce the number of arithmetic operations required and is present in many DNN software frameworks. However, the performance gain is at the expense of a reduction in floating point (FP) numerical accuracy. In this paper, we analyse the worst case FP error and prove the estimation of norm and conditioning of the algorithm. We show that the bound grows exponentially with the size of the convolution, but the error bound of the \textit{modified} algorithm is smaller than the original one. We propose several methods for reducing FP error. We propose a canonical evaluation ordering based on Huffman coding that reduces summation error. We study the selection of sampling "points" experimentally and find empirically good points for the most important sizes. We identify the main factors associated with good points. In addition, we explore other methods to reduce FP error, including mixed-precision convolution, and pairwise summation across DNN channels. Using our methods we can significantly reduce FP error for a given block size, which allows larger block sizes and reduced computation.

math.NA↗