arXiv subjects

John W. Pearson

Publications and source records attributed to John W. Pearson.

29 records · Page 2Linked to original sources

arXiv2021

PDE-Constrained Optimization Models and Pseudospectral Methods for Multiscale Particle Dynamics

We derive novel algorithms for optimization problems constrained by partial differential equations describing multiscale particle dynamics, including non-local integral terms representing interactions between particles. In particular, we investigate problems where the control acts as an advection 'flow' vector or a source term of the partial differential equation, and the constraint is equipped with boundary conditions of Dirichlet or no-flux type. After deriving continuous first-order optimality conditions for such problems, we solve the resulting systems by developing a link with computational methods for statistical mechanics, deriving pseudospectral methods in both space and time variables, and utilizing variants of existing fixed point methods. Numerical experiments indicate the effectiveness of our approach for a range of problem set-ups, boundary conditions, as well as regularization and model parameters.

math.NA↗

arXiv2021

Parameter-Robust Preconditioning for Oseen Iteration Applied to Stationary and Instationary Navier--Stokes Control

We derive novel, fast, and parameter-robust preconditioned iterative methods for steady and time-dependent Navier--Stokes control problems. Our approach may be applied to time-dependent problems which are discretized using backward Euler or Crank--Nicolson, and is also a valuable candidate for Stokes control problems discretized using Crank--Nicolson. The key ingredients of the solver are a saddle-point type approximation for the linear systems, an inner iteration for the $(1,1)$-block accelerated by a preconditioner for convection--diffusion control, and an approximation to the Schur complement based on a potent commutator argument applied to an appropriate block matrix. A range of numerical experiments validate the effectiveness of our new approach.

math.NA↗

arXiv2020

A New Preconditioning Approach for an Interior Point-Proximal Method of Multipliers for Linear and Convex Quadratic Programming

In this paper, we address the efficient numerical solution of linear and quadratic programming problems, often of large scale. With this aim, we devise an infeasible interior point method, blended with the proximal method of multipliers, which in turn results in a primal-dual regularized interior point method. Application of this method gives rise to a sequence of increasingly ill-conditioned linear systems which cannot always be solved by factorization methods, due to memory and CPU time restrictions. We propose a novel preconditioning strategy which is based on a suitable sparsification of the normal equations matrix in the linear case, and also constitutes the foundation of a block-diagonal preconditioner to accelerate MINRES for linear systems arising from the solution of general quadratic programming problems. Numerical results for a range of test problems demonstrate the robustness of the proposed preconditioning strategy, together with its ability to solve linear systems of very large dimension.

math.NA↗

arXiv2020

Fast Solution Methods for Convex Quadratic Optimization of Fractional Differential Equations

In this paper, we present numerical methods suitable for solving convex quadratic Fractional Differential Equation (FDE) constrained optimization problems, with box constraints on the state and/or control variables. We develop an Alternating Direction Method of Multipliers (ADMM) framework, which uses preconditioned Krylov subspace solvers for the resulting sub-problems. The latter allows us to tackle a range of Partial Differential Equation (PDE) optimization problems with box constraints, posed on space-time domains, that were previously out of the reach of state-of-the-art preconditioners. In particular, by making use of the powerful Generalized Locally Toeplitz (GLT) sequences theory, we show that any existing GLT structure present in the problem matrices is preserved by ADMM, and we propose some preconditioning methodologies that could be used within the solver, to demonstrate the generality of the approach. Focusing on convex quadratic programs with time-dependent 2-dimensional FDE constraints, we derive multilevel circulant preconditioners, which may be embedded within Krylov subspace methods, for solving the ADMM sub-problems. Discretized versions of FDEs involve large dense linear systems. In order to overcome this difficulty, we design a recursive linear algebra, which is based on the Fast Fourier Transform (FFT). We manage to keep the storage requirements linear, with respect to the grid size $N$, while ensuring an order $N \log N$ computational complexity per iteration of the Krylov solver. We implement the proposed method, and demonstrate its scalability, generality, and efficiency, through a series of experiments over different setups of the FDE optimization problem.

math.OC↗

arXiv2020

Fast Iterative Solver for the Optimal Control of Time-Dependent PDEs with Crank-Nicolson Discretization in Time

In this article, we derive a new, fast, and robust preconditioned iterative solution strategy for the all-at-once solution of optimal control problems with time-dependent PDEs as constraints, including the heat equation and the non-steady convection--diffusion equation. After applying an optimize-then-discretize approach, one is faced with continuous first-order optimality conditions consisting of a coupled system of PDEs. As opposed to most work in preconditioning the resulting discretized systems, where a (first-order accurate) backward Euler method is used for the discretization of the time derivative, we employ a (second-order accurate) Crank--Nicolson method in time. We apply a carefully tailored invertible transformation for symmetrizing the matrix, and then derive an optimal preconditioner for the saddle-point system obtained. The key components of this preconditioner are an accurate mass matrix approximation, a good approximation of the Schur complement, and an appropriate multigrid process to apply this latter approximation---these are constructed using our work in transforming the matrix system. We prove the optimality of the approximation of the Schur complement through bounds on the eigenvalues, and test our solver against a widely-used preconditioner for the linear system arising from a backward Euler discretization. These demonstrate the effectiveness and robustness of our solver with respect to mesh-sizes, regularization parameter, and diffusion coefficient.

math.NA↗

arXiv2020

Constructing Gradient Controllable Recurrent Neural Networks Using Hamiltonian Dynamics

Recurrent neural networks (RNNs) have gained a great deal of attention in solving sequential learning problems. The learning of long-term dependencies, however, remains challenging due to the problem of a vanishing or exploding hidden states gradient. By exploring further the recently established connections between RNNs and dynamical systems we propose a novel RNN architecture, which we call a Hamiltonian recurrent neural network (Hamiltonian RNN), based on a symplectic discretization of an appropriately chosen Hamiltonian system. The key benefit of this approach is that the corresponding RNN inherits the favorable long time properties of the Hamiltonian system, which in turn allows us to control the hidden states gradient with a hyperparameter of the Hamiltonian RNN architecture. This enables us to handle sequential learning problems with arbitrary sequence lengths, since for a range of values of this hyperparameter the gradient neither vanishes nor explodes. Additionally, we provide a heuristic for the optimal choice of the hyperparameter, which we use in our numerical simulations to illustrate that the Hamiltonian RNN is able to outperform other state-of-the-art RNNs without the need of computationally intensive hyperparameter optimization.

cs.LG↗

arXiv2019

Interior Point Methods and Preconditioning for PDE-Constrained Optimization Problems Involving Sparsity Terms

PDE-constrained optimization problems with control or state constraints are challenging from an analytical as well as numerical perspective. The combination of these constraints with a sparsity-promoting $\rm L^1$ term within the objective function requires sophisticated optimization methods. We propose the use of an Interior Point scheme applied to a smoothed reformulation of the discretized problem, and illustrate that such a scheme exhibits robust performance with respect to parameter changes. To increase the potency of this method we introduce fast and efficient preconditioners which enable us to solve problems from a number of PDE applications in low iteration numbers and CPU times, even when the parameters involved are altered dramatically.

math.OC↗

arXiv2018

Preconditioners and Tensor Product Solvers for Optimal Control Problems from Chemotaxis

In this paper, we consider the fast numerical solution of an optimal control formulation of the Keller--Segel model for bacterial chemotaxis. Upon discretization, this problem requires the solution of huge-scale saddle point systems to guarantee accurate solutions. We consider the derivation of effective preconditioners for these matrix systems, which may be embedded within suitable iterative methods to accelerate their convergence. We also construct low-rank tensor-train techniques which enable us to present efficient and feasible algorithms for problems that are finely discretized in the space and time variables. Numerical results demonstrate that the number of preconditioned GMRES iterations depends mildly on the model parameters. Moreover, the low-rank solver makes the computing time and memory costs sublinear in the original problem size.

math.NA↗

arXiv2018

Fast iterative solvers for an optimal transport problem

Optimal transport problems pose many challenges when considering their numerical treatment. We investigate the solution of a PDE-constrained optimisation problem subject to a particular transport equation arising from the modelling of image metamorphosis. We present the nonlinear optimisation problem, and discuss the discretisation and treatment of the nonlinearity via a Gauss--Newton scheme. We then derive preconditioners that can be used to solve the linear systems at the heart of the (Gauss--)Newton method. With the optical flow in mind, we further propose the reduction of dimensionality by choosing a radial basis function discretisation that uses the centres of superpixels as the collocation points. Again, we derive suitable preconditioners that can be used for this formulation.

math.NA↗

arXiv2016

A preconditioner for the Ohta--Kawasaki equation

We propose a new preconditioner for the Ohta--Kawasaki equation, a nonlocal Cahn--Hilliard equation that describes the evolution of diblock copolymer melts. We devise a computable approximation to the inverse of the Schur complement of the coupled second-order formulation via a matching strategy. The preconditioner achieves mesh independence: as the mesh is refined, the number of Krylov iterations required for its solution remains approximately constant. In addition, the preconditioner is robust with respect to the interfacial thickness parameter if a timestep criterion is satisfied. This enables the highly resolved finite element simulation of three-dimensional diblock copolymer melts with over one billion degrees of freedom.

math.NA↗

arXiv2015

Numerical Methods for the Computation of the Confluent and Gauss Hypergeometric Functions

The two most commonly used hypergeometric functions are the confluent hypergeometric function and the Gauss hypergeometric function. We review the available techniques for accurate, fast, and reliable computation of these two hypergeometric functions in different parameter and variable regimes. The methods that we investigate include Taylor and asymptotic series computations, Gauss-Jacobi quadrature, numerical solution of differential equations, recurrence relations, and others. We discuss the results of numerical experiments used to determine the best methods, in practice, for each parameter and variable regime considered. We provide 'roadmaps' with our recommendation for which methods should be used in each situation.

math.NA↗