arXiv subjects

John W. Pearson

Publications and source records attributed to John W. Pearson.

At least 19 recordsLinked to original sources

arXiv2026

Eigenvalue bounds for preconditioned symmetric multiple saddle-point matrices with block-triangular preconditioners

We develop eigenvalue bounds for symmetric, block-tridiagonal multiple saddle-point linear systems, preconditioned with block-triangular matrices, based on approximate Schur complements. Irrespective on the number of blocks, we prove that all complex eigenvalues, with nontrivial imaginary part, are strictly contained in a circle within the complex plane, with center 1. The real and positive eigenvalues are bounded in terms of the extremal roots of a sequences of parametric polynomials. Numerical results reveal that the bounds describe very well the eigenvalue distribution of the preconditioned matrix.

math.NA↗

arXiv2026

Block Alpha-Circulant Preconditioners for All-at-Once Diffusion-Based Covariance Operators

Covariance matrices are central to data assimilation and inverse methods derived from statistical estimation theory. Previous work has considered the application of an all-at-once diffusion-based representation of a covariance matrix operator in order to exploit inherent parallelism in the underlying problem. In this paper, we provide practical methods to apply block $α$-circulant preconditioners to the all-at-once system for the case where the main diffusion operation matrix cannot be readily diagonalized using a discrete Fourier transform. Our new framework applies the block $α$-circulant preconditioner approximately by solving an inner block diagonal problem via a choice of inner iterative approaches. Our first method applies Chebyshev semi-iteration to a symmetric positive definite matrix, shifted by a complex scaling of the identity. We extend theoretical results for Chebyshev semi-iteration in the symmetric positive definite setting, to obtain computable bounds on the asymptotic convergence factor for each of the complex sub-problems. The second approach transforms the complex sub-problem into a (generalized) saddle point system with real coefficients. Numerical experiments reveal that in the case of unlimited computational resources, both methods can match the iteration counts of the `best-case' block $α$-circulant preconditioner. We also provide a practical adaptation to the nested Chebyshev approach, which improves performance in the case of a limited computational budget. Using an appropriate choice of $α$ our new approaches are robust and efficient in terms of outer iterations and matrix--vector products.

math.NA↗

arXiv2026

Stabilizing randomized GMRES through flexible GMRES

We explore the use of flexible GMRES as an outer wrapper for sketched GMRES. Building on a new bound for the residual of FGMRES in terms of the residual of the preconditioner, we derive a practical randomized solver that requires very little parameter tuning, while still being efficient and robust in the sense of generating non-increasing residual norms.

math.NA↗

arXiv2026

Parallel-in-Time Preconditioning for Time-Dependent Variational Mean Field Games

We study the numerical approximation of a time-dependent variational mean field game system with local couplings and either periodic or Neumann boundary conditions. Following a variational approach, we employ a finite difference discretization and solve the resulting finite-dimensional optimization problem using the Chambolle--Pock primal--dual algorithm. As this involves computing proximal operators and solving ill-conditioned linear systems at each iteration, we embed within our solver a general class of parallel-in-time preconditioners based on suitably-chosen diagonalization techniques, applied using discrete Fourier transforms. These enable efficient, scalable iterative solvers for each linear system, with robustness across a wide range of viscosities. We further develop fast solvers for the resulting ill-conditioned systems arising at each time step, using exact recursive schemes for structured grids while allowing for other geometries. Numerical experiments confirm the improved performance and parallel scalability of our approach.

math.NA↗

arXiv2025

Carleman Linearization of Parabolic PDEs: Well-posedness, convergence, and efficient numerical methods

We explore how the analysis of the Carleman linearization can be extended to dynamical systems on infinite-dimensional Hilbert spaces with quadratic nonlinearities. We demonstrate the well-posedness and convergence of the truncated Carleman linearization under suitable assumptions on the dynamical system, which encompass common parabolic semi-linear partial differential equations such as the Navier-Stokes equations and nonlinear diffusion-advection-reaction equations. Upon discretization, we show that the total approximation error of the linearization decomposes into two independent components: the discretization error and the linearization error. This decomposition yields a convergence radius and convergence rate for the discretized linearization that are independent of the discretization. We thus justify the application of the linearization to parabolic PDE problems. Furthermore, it motivates the use of non-standard structure-exploiting numerical methods, such as sparse grids, taming the curse of dimensionality associated with the Carleman linearization. Finally, we verify the results with numerical experiments.

math.NA↗

arXiv2025

Efficient nonlocal linear image denoising: Bilevel optimization with Nonequispaced Fast Fourier Transform and matrix-free preconditioning

We present a new approach for nonlocal image denoising, based around the application of an unnormalized extended Gaussian ANOVA kernel within a bilevel optimization algorithm. A critical bottleneck when solving such problems for finely-resolved images is the solution of huge-scale, dense linear systems arising from the minimization of an energy term. We tackle this using a Krylov subspace approach, with a Nonequispaced Fast Fourier Transform utilized to approximate matrix-vector products in a matrix-free manner. We accelerate the algorithm using a novel change of basis approach to account for the (known) smallest eigenvalue-eigenvector pair of the matrices involved, coupled with a simple but frequently very effective diagonal preconditioning approach. We present a number of theoretical results concerning the eigenvalues and predicted convergence behavior, and a range of numerical experiments which validate our solvers and use them to tackle parameter learning problems. These demonstrate that very large problems may be effectively and rapidly denoised with very low storage requirements on a computer.

math.NA↗

arXiv2024

A Constrained Optimisation Framework for Parameter Identification of the SIRD Model

We consider a numerical framework tailored to identifying optimal parameters in the context of modelling disease propagation. Our focus is on understanding the behaviour of optimisation algorithms for such problems, where the dynamics are described by a system of ordinary differential equations associated with the epidemiological SIRD model. Applying an optimise-then-discretise approach, we examine properties of the solution operator and determine existence of optimal parameters for the problem considered. Further, first-order optimality conditions are derived, the solution of which provides a certificate of goodness of fit, which is not always guaranteed with parameter tuning techniques. We then propose strategies for the numerical solution of such problems, based on projected gradient descent, Fast Iterative Shrinkage-Thresholding Algorithm (FISTA), nonmonotone Accelerated Proximal Gradient (nmAPG), and limited memory BFGS trust region approaches. We carry out a thorough computational study for a range of problems of interest, determining the relative performance of these numerical methods. Our results provide insights into the effectiveness of these strategies, contributing to ongoing research into optimising parameters for accurate and reliable disease spread modelling. Moreover, our approach paves the way for calibration of more intricate compartmental models.

math.OC↗

arXiv2024

A Framework for the Solution of Tree-Coupled Saddle-Point Systems

We consider the solution of saddle-point systems with a tree-based block structure, introducing a parallelizable direct method for their solution. As our key contribution, we then propose several structure-exploiting preconditioners to be used during applications of the MINRES and GMRES algorithms and analyze their properties. We adapt several concepts originating in the field of multigrid methods, obtaining a variety of problem-adapted multi-level methods. We analyze the complexity of all algorithms, and derive a number of results on eigenvalues of the preconditioned system and convergence of iterative methods. We validate our theoretical findings through a range of numerical experiments.

math.NA↗

arXiv2024

Automatic Differentiation for All-at-once Systems Arising in Certain PDE-Constrained Optimization Problems

An automated framework is presented for the numerical solution of optimal control problems with PDEs as constraints, in both the stationary and instationary settings. The associated code can solve both linear and non-linear problems, and examples for incompressible flow equations are considered. The software, which is based on a Python interface to the Firedrake system, allows for a compact definition of the problem considered by providing a few lines of code in a high-level language. The software is provided with efficient iterative linear solvers for optimal control problems with PDEs as constraints. The use of advanced preconditioning techniques results in a significant speed-up of the solution process for large-scale problems. We present numerical examples of the applicability of the software on classical control problems with PDEs as constraints.

math.NA↗

arXiv2024

Fast numerical solvers for parameter identification problems in mathematical biology

In this paper, we consider effective discretization strategies and iterative solvers for nonlinear PDE-constrained optimization models for pattern evolution within biological processes. Upon a Sequential Quadratic Programming linearization of the optimization problem, we devise appropriate time-stepping schemes and discrete approximations of the cost functionals such that the discretization and optimization operations are commutative, a highly desirable property of a discretization of such problems. We formulate the large-scale, coupled linear systems in such a way that efficient preconditioned iterative methods can be applied within a Krylov subspace solver. Numerical experiments demonstrate the viability and efficiency of our approach.

math.NA↗

arXiv2024

Diagonalization-Based Parallel-in-Time Preconditioners for Instationary Fluid Flow Control Problems

We derive a new parallel-in-time approach for solving large-scale optimization problems constrained by time-dependent partial differential equations arising from fluid dynamics. The solver involves the use of a block circulant approximation of the original matrices, enabling parallelization-in-time via the use of fast Fourier transforms, and we devise bespoke matrix approximations which may be applied within this framework. These make use of permutations, saddle-point approximations, commutator arguments, as well as inner solvers such as the Uzawa method, Chebyshev semi-iteration, and multigrid. Theoretical results underpin our strategy of applying a block circulant strategy, and numerical experiments demonstrate the effectiveness and robustness of our approach on Stokes and Oseen problems. Noteably, satisfying results for the strong and weak scaling of our methods are provided within a fully parallel architecture.

math.NA↗

arXiv2023

Saddle point preconditioners for weak-constraint 4D-Var

Data assimilation algorithms combine information from observations and prior model information to obtain the most likely state of a dynamical system. The linearised weak-constraint four-dimensional variational assimilation problem can be reformulated as a saddle point problem, which admits more scope for preconditioners than the primal form. In this paper we design new terms which can be used within existing preconditioners, such as block diagonal and constraint-type preconditioners. Our novel preconditioning approaches: (i) incorporate model information, and (ii) are designed to target correlated observation error covariance matrices. To our knowledge (i) has not previously been considered for data assimilation problems. We develop new theory demonstrating the effectiveness of the new preconditioners within Krylov subspace methods. Linear and non-linear numerical experiments reveal that our new approach leads to faster convergence than existing state-of-the-art preconditioners for a broader range of problems than indicated by the theory alone. We present a range of numerical experiments performed in serial.

math.NA↗

arXiv2023

A Preconditioned Interior Point Method for Support Vector Machines Using an ANOVA-Decomposition and NFFT-Based Matrix-Vector Products

In this paper we consider the numerical solution to the soft-margin support vector machine optimization problem. This problem is typically solved using the SMO algorithm, given the high computational complexity of traditional optimization algorithms when dealing with large-scale kernel matrices. In this work, we propose employing an NFFT-accelerated matrix-vector product using an ANOVA decomposition for the feature space that is used within an interior point method for the overall optimization problem. As this method requires the solution of a linear system of saddle point form we suggest a preconditioning approach that is based on low-rank approximations of the kernel matrix together with a Krylov subspace solver. We compare the accuracy of the ANOVA-based kernel with the default LIBSVM implementation. We investigate the performance of the different preconditioners as well as the accuracy of the ANOVA kernel on several large-scale datasets.

math.NA↗

arXiv2023

Double Saddle-Point Preconditioning for Krylov Methods in the Inexact Sequential Homotopy Method

We derive an extension of the sequential homotopy method that allows for the application of inexact solvers for the linear (double) saddle-point systems arising in the local semismooth Newton method for the homotopy subproblems. For the class of problems that exhibit (after suitable partitioning of the variables) a zero in the off-diagonal blocks of the Hessian of the Lagrangian, we propose and analyze an efficient, parallelizable, symmetric positive definite preconditioner based on a double Schur complement approach. For discretized optimal control problems with PDE constraints, this structure is often present with the canonical partitioning of the variables in states and controls. We conclude with numerical results for a badly conditioned and highly nonlinear benchmark optimization problem with elliptic partial differential equations and control bounds. The resulting method allows for the parallel solution of large 3D problems.

math.OC↗

arXiv2023

Parallel-in-Time Solver for the All-at-Once Runge--Kutta Discretization

In this article, we derive fast and robust parallel-in-time preconditioned iterative methods for the all-at-once linear systems arising upon discretization of time-dependent PDEs. The discretization we employ is based on a Runge--Kutta method in time, for which the development of parallel solvers is an emerging research area in the literature of numerical methods for time-dependent PDEs. By making use of classical theory of block matrices, one is able to derive a preconditioner for the systems considered. The block structure of the preconditioner allows for parallelism in the time variable, as long as one is able to provide an optimal solver for the system of the stages of the method. We thus propose a preconditioner for the latter system based on a singular value decomposition (SVD) of the (real) Runge--Kutta matrix $A_{\mathrm{RK}} = U ΣV^\top$. Supposing $A_{\mathrm{RK}}$ is invertible, we prove that the spectrum of the system for the stages preconditioned by our SVD-based preconditioner is contained within the right-half of the unit circle, under suitable assumptions on the matrix $U^\top V$ (the assumptions are well posed due to the polar decomposition of $A_{\mathrm{RK}}$). We show the numerical efficiency of our SVD-based preconditioner by solving the system of the stages arising from the discretization of the heat equation and the Stokes equations, with sequential time-stepping. Finally, we provide numerical results of the all-at-once approach for both problems, showing the speed-up achieved on a parallel architecture.

math.NA↗

arXiv2023

On Symmetric Positive Definite Preconditioners for Multiple Saddle-Point Systems

We consider symmetric positive definite preconditioners for multiple saddle-point systems of block tridiagonal form, which can be applied within the MINRES algorithm. We describe such a preconditioner for which the preconditioned matrix has only two distinct eigenvalues, 1 and -1, when the preconditioner is applied exactly. We discuss the relative merits of such an approach compared to a more widely studied block diagonal preconditioner, specify the computational work associated with applying the new preconditioner inexactly, and survey a number of theoretical results for the block diagonal case. Numerical results validate our theoretical findings.

math.NA↗

arXiv2022

MultiShape: A Spectral Element Method, with Applications to Dynamic Density Functional Theory and PDE-Constrained Optimization

A numerical framework is developed to solve various types of PDEs on complicated domains, including steady and time-dependent, non-linear and non-local PDEs, with different boundary conditions that can also include non-linear and non-local terms. This numerical framework, called MultiShape, is a class in Matlab, and the software is open source. We demonstrate that MultiShape is compatible with other numerical methods, such as differential--algebraic equation solvers and optimization algorithms. The numerical implementation is designed to be user-friendly, with most of the set-up and computations done automatically by MultiShape and with intuitive operator definition, notation, and user-interface. Validation tests are presented, before we introduce three examples motivated by applications in Dynamic Density Functional Theory and PDE-constrained optimization, illustrating the versatility of the method.

math.NA↗

arXiv2022

General-purpose preconditioning for regularized interior point methods

In this paper we present general-purpose preconditioners for regularized augmented systems arising from optimization problems, and their corresponding normal equations. We discuss positive definite preconditioners, suitable for CG and MINRES. We consider "sparsifications" which avoid situations in which eigenvalues of the preconditioned matrix may become complex. Special attention is given to systems arising from the application of regularized interior point methods to linear or nonlinear convex programming problems.

math.OC↗