arXiv subjects

Andrew Stuart

Publications and source records attributed to Andrew Stuart.

At least 37 records · Page 2Linked to original sources

arXiv2022

Learning Dissipative Dynamics in Chaotic Systems

Chaotic systems are notoriously challenging to predict because of their sensitivity to perturbations and errors due to time stepping. Despite this unpredictable behavior, for many dissipative systems the statistics of the long term trajectories are governed by an invariant measure supported on a set, known as the global attractor; for many problems this set is finite dimensional, even if the state space is infinite dimensional. For Markovian systems, the statistical properties of long-term trajectories are uniquely determined by the solution operator that maps the evolution of the system over arbitrary positive time increments. In this work, we propose a machine learning framework to learn the underlying solution operator for dissipative chaotic systems, showing that the resulting learned operator accurately captures short-time trajectories and long-time statistical behavior. Using this framework, we are able to predict various statistics of the invariant measure for the turbulent Kolmogorov Flow dynamics with Reynolds numbers up to 5000.

cs.LG↗

arXiv2021

Fourier Neural Operator for Parametric Partial Differential Equations

The classical development of neural networks has primarily focused on learning mappings between finite-dimensional Euclidean spaces. Recently, this has been generalized to neural operators that learn mappings between function spaces. For partial differential equations (PDEs), neural operators directly learn the mapping from any functional parametric dependence to the solution. Thus, they learn an entire family of PDEs, in contrast to classical methods which solve one instance of the equation. In this work, we formulate a new neural operator by parameterizing the integral kernel directly in Fourier space, allowing for an expressive and efficient architecture. We perform experiments on Burgers' equation, Darcy flow, and Navier-Stokes equation. The Fourier neural operator is the first ML-based method to successfully model turbulent flows with zero-shot super-resolution. It is up to three orders of magnitude faster compared to traditional PDE solvers. Additionally, it achieves superior accuracy compared to previous learning-based solvers under fixed resolution.

cs.LG↗

arXiv2021

Multiscale modeling of materials: Computing, data science,uncertainty and goal-oriented optimization

The recent decades have seen various attempts at accelerating the process of developing materials targeted towards specific applications. The performance required for a particular application leads to the choice of a particular material system whose properties are optimized by manipulating its underlying microstructure through processing. The specific configuration of the structure is then designed by characterizing the material in detail, and using this characterization along with physical principles in system level simulations and optimization. These have been advanced by multiscale modeling of materials, high-throughput experimentations, materials data-bases, topology optimization and other ideas. Still, developing materials for extreme applications involving large deformation, high strain rates and high temperatures remains a challenge. This article reviews a number of recent methods that advance the goal of designing materials targeted by specific applications.

cond-mat.mtrl-sci↗

arXiv2021

A learning-based multiscale method and its application to inelastic impact problems

The macroscopic properties of materials that we observe and exploit in engineering application result from complex interactions between physics at multiple length and time scales: electronic, atomistic, defects, domains etc. Multiscale modeling seeks to understand these interactions by exploiting the inherent hierarchy where the behavior at a coarser scale regulates and averages the behavior at a finer scale. This requires the repeated solution of computationally expensive finer-scale models, and often a priori knowledge of those aspects of the finer-scale behavior that affect the coarser scale (order parameters, state variables, descriptors, etc.). We address this challenge in a two-scale setting where we learn the fine-scale behavior from off-line calculations and then use the learnt behavior directly in coarse scale calculations. The approach draws from recent successes of deep neural networks, in combination with ideas from model reduction. The approach builds on the recent success of deep neural networks by combining their approximation power in high dimensions with ideas from model reduction. It results in a neural network approximation that has high fidelity, is computationally inexpensive, is independent of the need for a priori knowledge, and can be used directly in the coarse scale calculations. We demonstrate the approach on problems involving the impact of magnesium, a promising light-weight structural and protective material.

cond-mat.mtrl-sci↗

arXiv2020

Kernel Analog Forecasting: Multiscale Test Problems

Data-driven prediction is becoming increasingly widespread as the volume of data available grows and as algorithmic development matches this growth. The nature of the predictions made, and the manner in which they should be interpreted, depends crucially on the extent to which the variables chosen for prediction are Markovian, or approximately Markovian. Multiscale systems provide a framework in which this issue can be analyzed. In this work kernel analog forecasting methods are studied from the perspective of data generated by multiscale dynamical systems. The problems chosen exhibit a variety of different Markovian closures, using both averaging and homogenization; furthermore, settings where scale-separation is not present and the predicted variables are non-Markovian, are also considered. The studies provide guidance for the interpretation of data-driven prediction methods when used in practice.

math.ST↗

arXiv2020

Multipole Graph Neural Operator for Parametric Partial Differential Equations

One of the main challenges in using deep learning-based methods for simulating physical systems and solving partial differential equations (PDEs) is formulating physics-based data in the desired structure for neural networks. Graph neural networks (GNNs) have gained popularity in this area since graphs offer a natural way of modeling particle interactions and provide a clear way of discretizing the continuum models. However, the graphs constructed for approximating such tasks usually ignore long-range interactions due to unfavorable scaling of the computational complexity with respect to the number of nodes. The errors due to these approximations scale with the discretization of the system, thereby not allowing for generalization under mesh-refinement. Inspired by the classical multipole methods, we propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Our multi-level formulation is equivalent to recursively adding inducing points to the kernel matrix, unifying GNNs with multi-resolution matrix factorization of the kernel. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.

cs.LG↗

arXiv2020

Site characterization at downhole arrays by joint inversion of dispersion data and acceleration time series

We present a sequential data assimilation algorithm based on the ensemble Kalman inversion to estimate the near-surface shear wave velocity profile and damping when heterogeneous data and a priori information that can be represented in forms of (physical) equality and inequality constraints in the inverse problem are available. Although non-invasive methods, such as surface wave testing, are efficient and cost effective methods for inferring Vs profile, one should acknowledge that site characterization using inverse analyses can yield erroneous results associated with the inverse problem non-uniqueness. One viable solution to alleviate the inverse problem ill-posedness is to enrich the prior knowledge and/or the data space with complementary observations. In the case of non-invasive methods, the pertinent data are the dispersion curve of surface waves, typically resolved by means of active source methods at high frequencies and passive methods at low frequencies. To improve the inverse problem well-posedness, horizontal to vertical spectral ratio (HVSR) data are commonly used jointly with the dispersion data in the inversion. In this paper, we show that the joint inversion of dispersion and strong motion downhole array data can also reduce the margins of uncertainty in the Vs profile estimation. This is because acceleration time-series recorded at downhole arrays include both body and surface waves and therefore can enrich the observational data space in the inverse problem setting. We also show how the proposed algorithm can be modified to systematically incorporate physical constraints that further enhance its well-posedness. We use both synthetic and real data to examine the performance of the proposed framework in estimation of Vs profile and damping at the Garner Valley downhole array, and compare them against the Vs estimations in previous studies.

physics.geo-ph↗

arXiv2020

Neural Operator: Graph Kernel Network for Partial Differential Equations

The classical development of neural networks has been primarily for mappings between a finite-dimensional Euclidean space and a set of classes, or between two finite-dimensional Euclidean spaces. The purpose of this work is to generalize neural networks so that they can learn mappings between infinite-dimensional spaces (operators). The key innovation in our work is that a single set of network parameters, within a carefully designed network architecture, may be used to describe mappings between infinite-dimensional spaces and between different finite-dimensional approximations of those spaces. We formulate approximation of the infinite-dimensional mapping by composing nonlinear activation functions and a class of integral operators. The kernel integration is computed by message passing on graph networks. This approach has substantial practical consequences which we will illustrate in the context of mappings between input data to partial differential equations (PDEs) and their solutions. In this context, such learned networks can generalize among different approximation methods for the PDE (such as finite difference or finite element methods) and among approximations corresponding to different underlying levels of resolution and discretization. Experiments confirm that the proposed graph kernel network does have the desired properties and show competitive performance compared to the state of the art solvers.

cs.LG↗

arXiv2019

Ensemble Kalman Methods With Constraints

Ensemble Kalman methods constitute an increasingly important tool in both state and parameter estimation problems. Their popularity stems from the derivative-free nature of the methodology which may be readily applied when computer code is available for the underlying state-space dynamics (for state estimation) or for the parameter-to-observable map (for parameter estimation). There are many applications in which it is desirable to enforce prior information in the form of equality or inequality constraints on the state or parameter. This paper establishes a general framework for doing so, describing a widely applicable methodology, a theory which justifies the methodology, and a set of numerical experiments exemplifying it.

math.OC↗

arXiv2019

Diffusive optical tomography in the Bayesian framework

Many naturally-occuring models in the sciences are well-approximated by simplified models, using multiscale techniques. In such settings it is natural to ask about the relationship between inverse problems defined by the original problem and by the multiscale approximation. We develop an approach to this problem and exemplify it in the context of optical tomographic imaging. Optical tomographic imaging is a technique for infering the properties of biological tissue via measurements of the incoming and outgoing light intensity; it may be used as a medical imaging methodology. Mathematically, light propagation is modeled by the radiative transfer equation (RTE), and optical tomography amounts to reconstructing the scattering and the absorption coefficients in the RTE from boundary measurements. We study this problem in the Bayesian framework, focussing on the strong scattering regime. In this regime the forward RTE is close to the diffusion equation (DE). We study the RTE in the asymptotic regime where the forward problem approaches the DE, and prove convergence of the inverse RTE to the inverse DE in both nonlinear and linear settings. Convergence is proved by studying the distance between the two posterior distributions using the Hellinger metric, and using Kullback-Leibler divergence.

math.NA↗

arXiv2017

Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High-Resolution Simulations

Climate projections continue to be marred by large uncertainties, which originate in processes that need to be parameterized, such as clouds, convection, and ecosystems. But rapid progress is now within reach. New computational tools and methods from data assimilation and machine learning make it possible to integrate global observations and local high-resolution simulations in an Earth system model (ESM) that systematically learns from both. Here we propose a blueprint for such an ESM. We outline how parameterization schemes can learn from global observations and targeted high-resolution simulations, for example, of clouds and convection, through matching low-order statistics between ESMs, observations, and high-resolution simulations. We illustrate learning algorithms for ESMs with a simple dynamical system that shares characteristics of the climate system; and we discuss the opportunities the proposed framework presents and the challenges that remain to realize it.

stat.ML↗

arXiv2017

Offline and online data assimilation for real-time blood glucose forecasting in type 2 diabetes

We evaluate the benefits of combining different offline and online data assimilation methodologies to improve personalized blood glucose prediction with type 2 diabetes self-monitoring data. We collect self-monitoring data (nutritional reports and pre- and post-prandial glucose measurements) from 4 individuals with diabetes and 2 individuals without diabetes. We write online to refer to methods that update state and parameters sequentially as nutrition and glucose data are received, and offline to refer to methods that estimate parameters over a fixed data set, distributed over a time window containing multiple nutrition and glucose measurements. We fit a model of ultradian glucose dynamics to the first half of each data set using offline (MCMC and nonlinear optimization) and online (unscented Kalman filter and an unfiltered model---a dynamical model driven by nutrition data that does not update states) data assimilation methods. Model parameters estimated over the first half of the data are used within online forecasting methods to issue forecasts over the second half of each data set. Offline data assimilation methods provided consistent advantages in predictive performance and practical usability in 4 of 6 patient data sets compared to online data assimilation methods alone; yet 2 of 6 patients were best predicted with a strictly online approach. Interestingly, parameter estimates generated offline led to worse predictions when fed to a stochastic filter than when used in a simple, unfiltered model that incorporates new nutritional information, but does not update model states based on glucose measurements. The relative improvements seen from the unfiltered model, when carefully trained offline, exposes challenges in model sensitivity and filtering applications, but also opens possibilities for improved glucose forecasting and relaxed patient self-monitoring requirements.

q-bio.QM↗

arXiv2017

Convergence Analysis of Ensemble Kalman Inversion: The Linear, Noisy Case

We present an analysis of ensemble Kalman inversion, based on the continuous time limit of the algorithm. The analysis of the dynamical behaviour of the ensemble allows us to establish well-posedness and convergence results for a fixed ensemble size. We will build on the results presented in [26] and generalise them to the case of noisy observational data, in particular the influence of the noise on the convergence will be investigated, both theoretically and numerically. We focus on linear inverse problems where a very complete theoretical analysis is possible.

math.NA↗

arXiv2015

Derivation and Analysis of Simplified Filters for Complex Dynamical Systems

Filtering is concerned with the sequential estimation of the state, and uncertainties, of a Markovian system, given noisy observations. It is particularly difficult to achieve accurate filtering in complex dynamical systems, such as those arising in turbulence, in which effective low-dimensional representation of the desired probability distribution is challenging. Nonetheless recent advances have shown considerable success in filtering based on certain carefully chosen simplifications of the underlying system, which allow closed form filters. This leads to filtering algorithms with significant, but judiciously chosen, model error. The purpose of this article is to analyze the effectiveness of these simplified filters, and to suggest modifications of them which lead to improved filtering in certain time-scale regimes. We employ a Markov switching process for the true signal underlying the data, rather than working with a fully resolved DNS PDE model. Such Markov switching models haven been demonstrated to provide an excellent surrogate test-bed for the turbulent bursting phenomena which make filtering of complex physical models, such as those arising in atmospheric sciences, so challenging.

math.PR↗

arXiv2015

Probability Measures for Numerical Solutions of Differential Equations

In this paper, we present a formal quantification of epistemic uncertainty induced by numerical solutions of ordinary and partial differential equation models. Numerical solutions of differential equations contain inherent uncertainties due to the finite dimensional approximation of an unknown and implicitly defined function. When statistically analysing models based on differential equations describing physical, or other naturally occurring, phenomena, it is therefore important to explicitly account for the uncertainty introduced by the numerical method. This enables objective determination of its importance relative to other uncertainties, such as those caused by data contaminated with noise or model error induced by missing physical or inadequate descriptors. To this end we show that a wide variety of existing solvers can be randomised, inducing a probability measure over the solutions of such differential equations. These measures exhibit contraction to a Dirac measure around the true unknown solution, where the rates of convergence are consistent with the underlying deterministic numerical method. Ordinary differential equations and elliptic partial differential equations are used to illustrate the approach to quantifying uncertainty in both the statistical analysis of the forward and inverse problems.

stat.ME↗

arXiv2014

Sequential Monte Carlo Methods for Bayesian Elliptic Inverse Problems

In this article we consider a Bayesian inverse problem associated to elliptic partial differential equations (PDEs) in two and three dimensions. This class of inverse problems is important in applications such as hydrology, but the complexity of the link function between unknown field and measurements can make it difficult to draw inference from the associated posterior. We prove that for this inverse problem a basic SMC method has a Monte Carlo rate of convergence with constants which are independent of the dimension of the discretization of the problem; indeed convergence of the SMC method is established in a function space setting. We also develop an enhancement of the sequential Monte Carlo (SMC) methods for inverse problems which were introduced in \cite{kantas}; the enhancement is designed to deal with the additional complexity of this elliptic inverse problem. The efficacy of the methodology, and its desirable theoretical properties, are demonstrated on numerical examples in both two and three dimensions.

stat.CO↗

arXiv2014

Kullback-Leibler Approximation for Probability Measures on Infinite Dimensional Spaces

In a variety of applications it is important to extract information from a probability measure $μ$ on an infinite dimensional space. Examples include the Bayesian approach to inverse problems and possibly conditioned) continuous time Markov processes. It may then be of interest to find a measure $ν$, from within a simple class of measures, which approximates $μ$. This problem is studied in the case where the Kullback-Leibler divergence is employed to measure the quality of the approximation. A calculus of variations viewpoint is adopted and the particular case where $ν$ is chosen from the set of Gaussian measures is studied in detail. Basic existence and uniqueness theorems are established, together with properties of minimising sequences. Furthermore, parameterisation of the class of Gaussians through the mean and inverse covariance is introduced, the need for regularisation is explained, and a regularised minimisation is studied in detail. The calculus of variations framework resulting from this work provides the appropriate underpinning for computational algorithms.

math.PR↗

arXiv2011

Besov priors for Bayesian inverse problems

We consider the inverse problem of estimating a function $u$ from noisy, possibly nonlinear, observations. We adopt a Bayesian approach to the problem. This approach has a long history for inversion, dating back to 1970, and has, over the last decade, gained importance as a practical tool. However most of the existing theory has been developed for Gaussian prior measures. Recently Lassas, Saksman and Siltanen (Inv. Prob. Imag. 2009) showed how to construct Besov prior measures, based on wavelet expansions with random coefficients, and used these prior measures to study linear inverse problems. In this paper we build on this development of Besov priors to include the case of nonlinear measurements. In doing so a key technical tool, established here, is a Fernique-like theorem for Besov measures. This theorem enables us to identify appropriate conditions on the forward solution operator which, when matched to properties of the prior Besov measure, imply the well-definedness and well-posedness of the posterior measure. We then consider the application of these results to the inverse problem of finding the diffusion coefficient of an elliptic partial differential equation, given noisy measurements of its solution.

math.ST↗