arXiv subjects

Matthew J. Colbrook

Publications and source records attributed to Matthew J. Colbrook.

At least 55 records · Page 3Linked to original sources

arXiv2023

Beyond expectations: Residual Dynamic Mode Decomposition and Variance for Stochastic Dynamical Systems

Koopman operators linearize nonlinear dynamical systems, making their spectral information of crucial interest. Numerous algorithms have been developed to approximate these spectral properties, and Dynamic Mode Decomposition (DMD) stands out as the poster child of projection-based methods. Although the Koopman operator itself is linear, the fact that it acts in an infinite-dimensional space of observables poses challenges. These include spurious modes, essential spectra, and the verification of Koopman mode decompositions. While recent work has addressed these challenges for deterministic systems, there remains a notable gap in verified DMD methods for stochastic systems, where the Koopman operator measures the expectation of observables. We show that it is necessary to go beyond expectations to address these issues. By incorporating variance into the Koopman framework, we address these challenges. Through an additional DMD-type matrix, we approximate the sum of a squared residual and a variance term, each of which can be approximated individually using batched snapshot data. This allows verified computation of the spectral properties of stochastic Koopman operators, controlling the projection error. We also introduce the concept of variance-pseudospectra to gauge statistical coherency. Finally, we present a suite of convergence results for the spectral information of stochastic Koopman operators. Our study concludes with practical applications using both simulated and experimental data. In neural recordings from awake mice, we demonstrate how variance-pseudospectra can reveal physiologically significant information unavailable to standard expectation-based dynamical models.

math.DS↗

arXiv2023

Rigorous data-driven computation of spectral properties of Koopman operators for dynamical systems

Koopman operators are infinite-dimensional operators that globally linearize nonlinear dynamical systems, making their spectral information valuable for understanding dynamics. However, Koopman operators can have continuous spectra and infinite-dimensional invariant subspaces, making computing their spectral information a considerable challenge. This paper describes data-driven algorithms with rigorous convergence guarantees for computing spectral information of Koopman operators from trajectory data. We introduce residual dynamic mode decomposition (ResDMD), which provides the first scheme for computing the spectra and pseudospectra of general Koopman operators from snapshot data without spectral pollution. Using the resolvent operator and ResDMD, we compute smoothed approximations of spectral measures associated with general measure-preserving dynamical systems. We prove explicit convergence theorems for our algorithms, which can achieve high-order convergence even for chaotic systems when computing the density of the continuous spectrum and the discrete spectrum. Since our algorithms come with error control, ResDMD allows aposteri verification of spectral quantities, Koopman mode decompositions, and learned dictionaries. We demonstrate our algorithms on the tent map, circle rotations, Gauss iterated map, nonlinear pendulum, double pendulum, and Lorenz system. Finally, we provide kernelized variants of our algorithms for dynamical systems with a high-dimensional state space. This allows us to compute the spectral measure associated with the dynamics of a protein molecule with a 20,046-dimensional state space and compute nonlinear Koopman modes with error bounds for turbulent flow past aerofoils with Reynolds number $>10^5$ that has a 295,122-dimensional state space.

math.NA↗

arXiv2023

Avoiding discretization issues for nonlinear eigenvalue problems

The first step when solving an infinite-dimensional eigenvalue problem is often to discretize it. We show that one must be extremely careful when discretizing nonlinear eigenvalue problems. Using examples, we show that discretization can: (1) introduce spurious eigenvalues, (2) entirely miss spectra, and (3) bring in severe ill-conditioning. While there are many eigensolvers for solving matrix nonlinear eigenvalue problems, we propose a solver for general holomorphic infinite-dimensional nonlinear eigenvalue problems that avoids discretization issues, which we prove is stable and converges. Moreover, we provide an algorithm that computes the problem's pseudospectra with explicit error control, allowing verification of computed spectra. The algorithm and numerical examples are publicly available in $\texttt{infNEP}$, which is a software package written in MATLAB.

math.NA↗

arXiv2022

The foundations of spectral computations via the Solvability Complexity Index hierarchy

The problem of computing spectra of operators is arguably one of the most investigated areas of computational mathematics. However, the problem of computing spectra of general bounded infinite matrices has only recently been solved. We establish some of the foundations of computational spectral theory through the Solvability Complexity Index (SCI) hierarchy, an approach closely related to Smale's program on the foundations of computational mathematics and McMullen's results on polynomial root finding with rational maps. Infinite-dimensional problems yield an intricate infinite classification theory, determining which spectral problems can be solved and with what types of algorithms. We provide answers to many longstanding open questions on the existence of algorithms. For example, we show that spectra can be computed, with error control, from point sampling operator coefficients for large classes of partial differential operators on unbounded domains. Further results include: computing spectra of (possibly unbounded) operators on graphs and separable Hilbert spaces with error control; determining if the spectrum intersects a compact set; the computational spectral gap problem and computing spectral classifications at the bottom of the spectrum; and computing discrete spectra, multiplicities, eigenspaces and determining if the discrete spectrum is non-empty. Moreover, the positive results with error control can be used in computer-assisted proofs. In contrast, the negative results preclude computer-assisted proofs for classes of operators as a whole. Our proofs are constructive, yielding a library of new algorithms and techniques that handle problems that before were out of reach. We demonstrate these algorithms on challenging problems, giving concrete examples of the failure of traditional approaches (e.g., "spectral pollution") compared to the introduced techniques.

math.SP↗

arXiv2022

On the computation of geometric features of spectra of linear operators on Hilbert spaces

Computing spectra is a central problem in computational mathematics with an abundance of applications throughout the sciences. However, in many applications gaining an approximation of the spectrum is not enough. Often it is vital to determine geometric features of spectra such as Lebesgue measure, capacity or fractal dimensions, different types of spectral radii and numerical ranges, or to detect essential spectral gaps and the corresponding failure of the finite section method. Despite new results on computing spectra and the substantial interest in these geometric problems, there remain no general methods able to compute such geometric features of spectra of infinite-dimensional operators. We provide the first algorithms for the computation of many of these longstanding problems (including the above). As demonstrated with computational examples, the new algorithms yield a library of new methods. Recent progress in computational spectral problems in infinite dimensions has led to the Solvability Complexity Index (SCI) hierarchy, which classifies the difficulty of computational problems. These results reveal that infinite-dimensional spectral problems yield an intricate infinite classification theory determining which spectral problems can be solved and with which type of algorithm. This is very much related to S. Smale's comprehensive program on the foundations of computational mathematics initiated in the 1980s. We classify the computation of geometric features of spectra in the SCI hierarchy, allowing us to precisely determine the boundaries of what computers can achieve (in any model of computation) and prove that our algorithms are optimal. We also provide a new universal technique for establishing lower bounds in the SCI hierarchy, which both greatly simplifies previous SCI arguments and allows new, formerly unattainable, classifications.

math.SP↗

arXiv2022

The mpEDMD Algorithm for Data-Driven Computations of Measure-Preserving Dynamical Systems

Koopman operators globally linearize nonlinear dynamical systems and their spectral information is a powerful tool for the analysis and decomposition of nonlinear dynamical systems. However, Koopman operators are infinite-dimensional, and computing their spectral information is a considerable challenge. We introduce measure-preserving extended dynamic mode decomposition ($\texttt{mpEDMD}$), the first truncation method whose eigendecomposition converges to the spectral quantities of Koopman operators for general measure-preserving dynamical systems. $\texttt{mpEDMD}$ is a data-driven algorithm based on an orthogonal Procrustes problem that enforces measure-preserving truncations of Koopman operators using a general dictionary of observables. It is flexible and easy to use with any pre-existing DMD-type method, and with different types of data. We prove convergence of $\texttt{mpEDMD}$ for projection-valued and scalar-valued spectral measures, spectra, and Koopman mode decompositions. For the case of delay embedding (Krylov subspaces), our results include the first convergence rates of the approximation of spectral measures as the size of the dictionary increases. We demonstrate $\texttt{mpEDMD}$ on a range of challenging examples, its increased robustness to noise compared with other DMD-type methods, and its ability to capture the energy conservation and cascade of experimental measurements of a turbulent boundary layer flow with Reynolds number $> 6\times 10^4$ and state-space dimension $>10^5$.

math.NA↗

arXiv2022

Residual Dynamic Mode Decomposition: Robust and verified Koopmanism

Dynamic Mode Decomposition (DMD) describes complex dynamic processes through a hierarchy of simpler coherent features. DMD is regularly used to understand the fundamental characteristics of turbulence and is closely related to Koopman operators. However, verifying the decomposition, equivalently the computed spectral features of Koopman operators, remains a major challenge due to the infinite-dimensional nature of Koopman operators. Challenges include spurious (unphysical) modes, and dealing with continuous spectra, both of which occur regularly in turbulent flows. Residual Dynamic Mode Decomposition (ResDMD), introduced by (Colbrook & Townsend 2021), overcomes some of these challenges through the data-driven computation of residuals associated with the full infinite-dimensional Koopman operator. ResDMD computes spectra and pseudospectra of general Koopman operators with error control, and computes smoothed approximations of spectral measures (including continuous spectra) with explicit high-order convergence theorems. ResDMD thus provides robust and verified Koopmanism. We implement ResDMD and demonstrate its application in a variety of fluid dynamic situations, at varying Reynolds numbers, arising from both numerical and experimental data. Examples include: vortex shedding behind a cylinder; hot-wire data acquired in a turbulent boundary layer; particle image velocimetry data focusing on a wall-jet flow; and acoustic pressure signals of laser-induced plasma. We present some advantages of ResDMD, namely, the ability to verifiably resolve non-linear, transient modes, and spectral calculation with reduced broadening effects. We also discuss how a new modal ordering based on residuals enables greater accuracy with a smaller dictionary than the traditional modulus ordering. This paves the way for greater dynamic compression of large datasets without sacrificing accuracy.

physics.flu-dyn↗

arXiv2022

SpecSolve: Spectral methods for spectral measures

Self-adjoint operators on infinite-dimensional spaces with continuous spectra are abundant but do not possess a basis of eigenfunctions. Rather, diagonalization is achieved through spectral measures. The SpecSolve package [SIAM Rev., 63(3) (2021), pp. 489--524] computes spectral measures of general (self-adjoint) differential and integral operators by combining state-of-the-art adaptive spectral methods with an efficient resolvent-based strategy. The algorithm achieves arbitrarily high orders of convergence in terms of a smoothing parameter, allowing computation of both discrete and continuous spectral components. This article extends SpecSolve to two important classes of operators: singular integro-differential operators and general operator pencils. Essential computational steps are performed with off-the-shelf spectral methods, including spectral methods on the real line, the ultraspherical spectral method, Chebyshev and Fourier spectral methods, and the ($hp$-adaptive and sparse) ultraspherical spectral element method. This collection illustrates the power and flexibility of SpecSolve's "discretization-oblivious" paradigm.

math.NA↗

arXiv2021

Computing spectral properties of topological insulators without artificial truncation or supercell approximation

Topological insulators (TIs) are renowned for their remarkable electronic properties: quantised bulk Hall and edge conductivities, and robust edge wave-packet propagation, even in the presence of material defects and disorder. Computations of these physical properties generally rely on artificial periodicity (the supercell approximation), or unphysical boundary conditions (artificial truncation). In this work, we build on recently developed methods for computing spectral properties of infinite-dimensional operators. We apply these techniques to develop efficient and accurate computational tools for computing the physical properties of TIs. These tools completely avoid such artificial restrictions and allow one to probe the spectral properties of the infinite-dimensional operator directly, even in the presence of material defects and disorder. Our methods permit computation of spectra, approximate eigenstates, spectral measures, spectral projections, transport properties, and conductances. Numerical examples are given for the Haldane model, and the techniques can be extended similarly to other TIs in two and three dimensions.

math.NA↗

arXiv2021

A contour method for time-fractional PDEs and an application to fractional viscoelastic beam equations

We develop a rapid and accurate contour method for the solution of time-fractional PDEs. The method inverts the Laplace transform via an optimised stable quadrature rule, suitable for infinite-dimensional operators, whose error decreases like $\exp(-cN/\log(N))$ for $N$ quadrature points. The method is parallisable, avoids having to resolve singularities of the solution as $t\downarrow 0$, and avoids the large memory consumption that can be a challenge for time-stepping methods applied to time-fractional PDEs. The ODEs resulting from quadrature are solved using adaptive sparse spectral methods that converge exponentially with optimal linear complexity. These solutions of ODEs are reused for different times. We provide a complete analysis of our approach for fractional beam equations used to model small-amplitude vibration of viscoelastic materials with a fractional Kelvin-Voigt stress-strain relationship. We calculate the system's energy evolution over time and the surface deformation in cases of both constant and non-constant viscoelastic parameters. An infinite-dimensional ``solve-then-discretise'' approach considerably simplifies the analysis, which studies the generalisation of the numerical range of a quasi-linearisation of a suitable operator pencil. This allows us to build an efficient algorithm with explicit error control. The approach can be readily adapted to other time-fractional PDEs and is not constrained to fractional parameters in the range $0<ν<1$.

math.NA↗

arXiv2021

WARPd: A linearly convergent first-order method for inverse problems with approximate sharpness conditions

Reconstruction of signals from undersampled and noisy measurements is a topic of considerable interest. Sharpness conditions directly control the recovery performance of restart schemes for first-order methods without the need for restrictive assumptions such as strong convexity. However, they are challenging to apply in the presence of noise or approximate model classes (e.g., approximate sparsity). We provide a first-order method: Weighted, Accelerated and Restarted Primal-dual (WARPd), based on primal-dual iterations and a novel restart-reweight scheme. Under a generic approximate sharpness condition, WARPd achieves stable linear convergence to the desired vector. Many problems of interest fit into this framework. For example, we analyze sparse recovery in compressed sensing, low-rank matrix recovery, matrix completion, TV regularization, minimization of $\|Bx\|_{l^1}$ under constraints ($l^1$-analysis problems for general $B$), and mixed regularization problems. We show how several quantities controlling recovery performance also provide explicit approximate sharpness constants. Numerical experiments show that WARPd compares favorably with specialized state-of-the-art methods and is ideally suited for solving large-scale problems. We also present a noise-blind variant based on the Square-Root LASSO decoder. Finally, we show how to unroll WARPd as neural networks. This approximation theory result provides lower bounds for stable and accurate neural networks for inverse problems and sheds light on architecture choices. Code and a gallery of examples are made available online as a MATLAB package.

math.NA↗

arXiv2021

Computing semigroups with error control

We develop an algorithm that computes strongly continuous semigroups on infinite-dimensional Hilbert spaces with explicit error control. Given a generator $A$, a time $t>0$, an arbitrary initial vector $u_0$ and an error tolerance $ε>0$, the algorithm computes $\exp(tA)u_0$ with error bounded by $ε$. The algorithm is based on a combination of a regularized functional calculus, suitable contour quadrature rules, and the adaptive computation of resolvents in infinite dimensions. As a particular case, we show that it is possible, even when only allowing pointwise evaluation of coefficients, to compute, with error control, semigroups on the unbounded domain $L^2(\mathbb{R}^d)$ that are generated by partial differential operators with polynomially bounded coefficients of locally bounded total variation. For analytic semigroups (and more general Laplace transform inversion), we provide a quadrature rule whose error decreases like $\exp(-cN/\log(N))$ for $N$ quadrature points, that remains stable as $N\rightarrow\infty$, and which is also suitable for infinite-dimensional operators. Numerical examples are given, including: Schrödinger and wave equations on the aperiodic Ammann--Beenker tiling, complex perturbed fractional diffusion equations on $L^2(\mathbb{R})$, and damped Euler--Bernoulli beam equations.

math.NA↗

arXiv2021

Bulk Localised Transport States in Infinite and Finite Quasicrystals via Magnetic Aperiodicity

Robust edge transport can occur when particles in crystalline lattices interact with an external magnetic field. This system is well described by Bloch's theorem, with the spectrum being composed of bands of bulk states and in-gap edge states. When the confining lattice geometry is altered to be quasicrystaline, then Bloch's theorem breaks down. However, we still expect to observe the basic characteristics of bulk states and current carrying edge states. Here, we show that for quasicrystals in magnetic fields, there is also a third option; the bulk localised transport states. These states share the in-gap nature of the well-known edge states and can support transport along them, but they are fully contained within the bulk of the system, with no support along the edge. We consider both finite and infinite systems, using rigorous error controlled computational techniques that are not prone to finite-size effects. The bulk localised transport states are preserved for infinite systems, in stark contrast to the normal edge states. This allows for transport to be observed in infinite systems, without any perturbations, defects, or boundaries being introduced. We confirm the in-gap topological nature of the bulk localised transport states for finite and infinite systems by computing common topological measures; namely the Bott index and local Chern marker. The bulk localised transport states form due to a magnetic aperiodicity arising from the interplay of length scales between the magnetic field and quasiperiodic lattice. Bulk localised transport could have interesting applications similar to those of the edge states on the boundary, but that could now take advantage of the larger bulk of the lattice. The infinite size techniques introduced here, especially the calculation of topological measures, could also be widely applied to other crystalline, quasicrystalline, and disordered models.

cond-mat.str-el↗

arXiv2021

Can stable and accurate neural networks be computed? -- On the barriers of deep learning and Smale's 18th problem

Deep learning (DL) has had unprecedented success and is now entering scientific computing with full force. However, current DL methods typically suffer from instability, even when universal approximation properties guarantee the existence of stable neural networks (NNs). We address this paradox by demonstrating basic well-conditioned problems in scientific computing where one can prove the existence of NNs with great approximation qualities, however, there does not exist any algorithm, even randomised, that can train (or compute) such a NN. For any positive integers $K > 2$ and $L$, there are cases where simultaneously: (a) no randomised training algorithm can compute a NN correct to $K$ digits with probability greater than $1/2$, (b) there exists a deterministic training algorithm that computes a NN with $K-1$ correct digits, but any such (even randomised) algorithm needs arbitrarily many training data, (c) there exists a deterministic training algorithm that computes a NN with $K-2$ correct digits using no more than $L$ training samples. These results imply a classification theory describing conditions under which (stable) NNs with a given accuracy can be computed by an algorithm. We begin this theory by establishing sufficient conditions for the existence of algorithms that compute stable NNs in inverse problems. We introduce Fast Iterative REstarted NETworks (FIRENETs), which we both prove and numerically verify are stable. Moreover, we prove that only $\mathcal{O}(|\log(ε)|)$ layers are needed for an $ε$-accurate solution to the inverse problem.

cs.LG↗

arXiv2020

Computing spectral measures of self-adjoint operators

Using the resolvent operator, we develop an algorithm for computing smoothed approximations of spectral measures associated with self-adjoint operators. The algorithm can achieve arbitrarily high-orders of convergence in terms of a smoothing parameter for computing spectral measures of general differential, integral, and lattice operators. Explicit pointwise and $L^p$-error bounds are derived in terms of the local regularity of the measure. We provide numerical examples, including a partial differential operator, a magnetic tight-binding model of graphene, and compute one thousand eigenvalues of a Dirac operator to near machine precision without spectral pollution. The algorithm is publicly available in $\texttt{SpecSolve}$, which is a software package written in MATLAB.

math.NA↗

arXiv2020

On the infinite-dimensional QR algorithm

Spectral computations of infinite-dimensional operators are notoriously difficult, yet ubiquitous in the sciences. Indeed, despite more than half a century of research, it is still unknown which classes of operators allow for computation of spectra and eigenvectors with convergence rates and error control. Recent progress in classifying the difficulty of spectral problems into complexity hierarchies has revealed that the most difficult spectral problems are so hard that one needs three limits in the computation, and no convergence rates nor error control is possible. This begs the question: which classes of operators allow for computations with convergence rates and error control? In this paper we address this basic question, and the algorithm used is an infinite-dimensional version of the QR algorithm. Indeed, we generalise the QR algorithm to infinite-dimensional operators. We prove that not only is the algorithm executable on a finite machine, but one can also recover the extremal parts of the spectrum and corresponding eigenvectors, with convergence rates and error control. This allows for new classification results in the hierarchy of computational problems that existing algorithms have not been able to capture. The algorithm and convergence theorems are demonstrated on a wealth of examples with comparisons to standard approaches (that are notorious for providing false solutions).We also find that in some cases the IQR algorithm performs better than predicted by theory and make conjectures for future study.

math.NA↗

arXiv2020

Computing Spectra -- On the Solvability Complexity Index Hierarchy and Towers of Algorithms

This paper establishes some of the fundamental barriers in the theory of computations and finally settles the long-standing computational spectral problem. That is to determine the existence of algorithms that can compute spectra $\mathrm{sp}(A)$ of classes of bounded operators $A = \{a_{ij}\}_{i,j \in \mathbb{N}} \in \mathcal{B}(l^2(\mathbb{N}))$, given the matrix elements $\{a_{ij}\}_{i,j \in \mathbb{N}}$, that are sharp in the sense that they achieve the boundary of what a digital computer can achieve. Similarly, for a Schrödinger operator $H = -Δ+V$, determine the existence of algorithms that can compute the spectrum $\mathrm{sp}(H)$ given point samples of the potential function $V$. In order to solve these problems, we establish the Solvability Complexity Index (SCI) hierarchy and provide a collection of new algorithms that allow for problems that were previously out of reach. The SCI is the smallest number of limits needed in the computation, yielding a classification hierarchy for all types of problems in computational mathematics that determines the boundaries of what computers can achieve in scientific computing. In addition, the SCI hierarchy provides classifications of computational problems that can be used in computer-assisted proofs. The SCI hierarchy captures many key computational issues in the history of mathematics including the insolvability of the quintic, Smale's problem on the existence of iterative generally convergent algorithm for polynomial root finding, the computational spectral problem, inverse problems, optimisation etc.

cs.CC↗

arXiv2020

Kernel Density Estimation with Linked Boundary Conditions

Kernel density estimation on a finite interval poses an outstanding challenge because of the well-recognized bias at the boundaries of the interval. Motivated by an application in cancer research, we consider a boundary constraint linking the values of the unknown target density function at the boundaries. We provide a kernel density estimator (KDE) that successfully incorporates this linked boundary condition, leading to a non-self-adjoint diffusion process and expansions in non-separable generalized eigenfunctions. The solution is rigorously analyzed through an integral representation given by the unified transform (or Fokas method). The new KDE possesses many desirable properties, such as consistency, asymptotically negligible bias at the boundaries, and an increased rate of approximation, as measured by the AMISE. We apply our method to the motivating example in biology and provide numerical experiments with synthetic data, including comparisons with state-of-the-art KDEs (which currently cannot handle linked boundary constraints). Results suggest that the new method is fast and accurate. Furthermore, we demonstrate how to build statistical estimators of the boundary conditions satisfied by the target function without apriori knowledge. Our analysis can also be extended to more general boundary conditions that may be encountered in applications.

math.ST↗