arXiv subjects

A. M. Stuart

Publications and source records attributed to A. M. Stuart.

At least 37 records · Page 2Linked to original sources

arXiv2013

Accuracy and Stability of Filters for Dissipative PDEs

Data assimilation methodologies are designed to incorporate noisy observations of a physical system into an underlying model in order to infer the properties of the state of the system. Filters refer to a class of data assimilation algorithms designed to update the estimation of the state as data is acquired sequentially. For linear problems subject to Gaussian noise filtering can be performed exactly using the Kalman filter. For nonlinear systems it can be approximated in a systematic way by particle filters. However in high dimensions these particle filtering methods can break down. Hence, for the large nonlinear systems arising in applications such as oceanography and weather forecasting, various ad hoc filters are used, based on Gaussian approximations. In this work, we study the accuracy and stability of these ad hoc filters in the context of the 2D incompressible Navier-Stokes equation. The ideas readily generalize to a range of dissipative partial differential equations (PDEs). By working in this infinite dimensional setting we provide an analysis which is useful for the understanding of high dimensional filtering, and is robust to mesh-refinement. We describe theoretical results showing that, in the small observational noise limit, the filters can be tuned to perform accurately in tracking the signal itself (filter accuracy), provided the system is observed in a sufficiently large low dimensional space; roughly speaking this space should be large enough to contain the unstable modes of the linearized dynamics. The tuning corresponds to what is known as variance inflation in the applied literature. Numerical results are given which illustrate the theory. The positive results herein concerning filter stability complement recent numerical studies which demonstrate that the ad hoc filters can perform poorly in reproducing statistical variation about the true signal.

math.OC↗

arXiv2013

Analysis of the 3DVAR Filter for the Partially Observed Lorenz '63 Model

The problem of effectively combining data with a mathematical model constitutes a major challenge in applied mathematics. It is particular challenging for high-dimensional dynamical systems where data is received sequentially in time and the objective is to estimate the system state in an on-line fashion; this situation arises, for example, in weather forecasting. The sequential particle filter is then impractical and ad hoc filters, which employ some form of Gaussian approximation, are widely used. Prototypical of these ad hoc filters is the 3DVAR method. The goal of this paper is to analyze the 3DVAR method, using the Lorenz '63 model to exemplify the key ideas. The situation where the data is partial and noisy is studied, and both discrete time and continuous time data streams are considered. The theory demonstrates how the widely used technique of variance inflation acts to stabilize the filter, and hence leads to asymptotic accuracy.

math.DS↗

arXiv2012

Accuracy and Stability of The Continuous-Time 3DVAR Filter for The Navier-Stokes Equation

The 3DVAR filter is prototypical of methods used to combine observed data with a dynamical system, online, in order to improve estimation of the state of the system. Such methods are used for high dimensional data assimilation problems, such as those arising in weather forecasting. To gain understanding of filters in applications such as these, it is hence of interest to study their behaviour when applied to infinite dimensional dynamical systems. This motivates study of the problem of accuracy and stability of 3DVAR filters for the Navier-Stokes equation. We work in the limit of high frequency observations and derive continuous time filters. This leads to a stochastic partial differential equation (SPDE) for state estimation, in the form of a damped-driven Navier-Stokes equation, with mean-reversion to the signal, and spatially-correlated time-white noise. Both forward and pullback accuracy and stability results are proved for this SPDE, showing in particular that when enough low Fourier modes are observed, and when the model uncertainty is larger than the data uncertainty in these modes (variance inflation), then the filter can lock on to a small neighbourhood of the true signal, recovering from order one initial error, if the error in the observations modes is small. Numerical examples are given to illustrate the theory.

math.PR↗

arXiv2012

Posterior Consistency via Precision Operators for Bayesian Nonparametric Drift Estimation in SDEs

We study a Bayesian approach to nonparametric estimation of the periodic drift function of a one-dimensional diffusion from continuous-time data. Rewriting the likelihood in terms of local time of the process, and specifying a Gaussian prior with precision operator of differential form, we show that the posterior is also Gaussian with precision operator also of differential form. The resulting expressions are explicit and lead to algorithms which are readily implementable. Using new functional limit theorems for the local time of diffusions on the circle, we bound the rate at which the posterior contracts around the true drift function.

stat.ME↗

arXiv2012

Evaluating Data Assimilation Algorithms

Data assimilation leads naturally to a Bayesian formulation in which the posterior probability distribution of the system state, given the observations, plays a central conceptual role. The aim of this paper is to use this Bayesian posterior probability distribution as a gold standard against which to evaluate various commonly used data assimilation algorithms. A key aspect of geophysical data assimilation is the high dimensionality and low predictability of the computational model. With this in mind, yet with the goal of allowing an explicit and accurate computation of the posterior distribution, we study the 2D Navier-Stokes equations in a periodic geometry. We compute the posterior probability distribution by state-of-the-art statistical sampling techniques. The commonly used algorithms that we evaluate against this accurate gold standard, as quantified by comparing the relative error in reproducing its moments, are 4DVAR and a variety of sequential filtering approximations based on 3DVAR and on extended and ensemble Kalman filters. The primary conclusions are that: (i) with appropriate parameter choices, approximate filters can perform well in reproducing the mean of the desired probability distribution; (ii) however they typically perform poorly when attempting to reproduce the covariance; (iii) this poor performance is compounded by the need to modify the covariance, in order to induce stability. Thus, whilst filters can be a useful tool in predicting mean behavior, they should be viewed with caution as predictors of uncertainty. These conclusions are intrinsic to the algorithms and will not change if the model complexity is increased, for example by employing a smaller viscosity, or by using a detailed NWP model.

physics.data-an↗

arXiv2012

Sparse Deterministic Approximation of Bayesian Inverse Problems

We present a parametric deterministic formulation of Bayesian inverse problems with input parameter from infinite dimensional, separable Banach spaces. In this formulation, the forward problems are parametric, deterministic elliptic partial differential equations, and the inverse problem is to determine the unknown, parametric deterministic coefficients from noisy observations comprising linear functionals of the solution. We prove a generalized polynomial chaos representation of the posterior density with respect to the prior measure, given noisy observational data. We analyze the sparsity of the posterior density in terms of the summability of the input data's coefficient sequence. To this end, we estimate the fluctuations in the prior. We exhibit sufficient conditions on the prior model in order for approximations of the posterior density to converge at a given algebraic rate, in terms of the number $N$ of unknowns appearing in the parameteric representation of the prior measure. Similar sparsity and approximation results are also exhibited for the solution and covariance of the elliptic partial differential equation under the posterior. These results then form the basis for efficient uncertainty quantification, in the presence of data with noise.

math.AP↗

arXiv2011

Stability of Filters for the Navier-Stokes Equation

Data assimilation methodologies are designed to incorporate noisy observations of a physical system into an underlying model in order to infer the properties of the state of the system. Filters refer to a class of data assimilation algorithms designed to update the estimation of the state in a on-line fashion, as data is acquired sequentially. For linear problems subject to Gaussian noise filtering can be performed exactly using the Kalman filter. For nonlinear systems it can be approximated in a systematic way by particle filters. However in high dimensions these particle filtering methods can break down. Hence, for the large nonlinear systems arising in applications such as weather forecasting, various ad hoc filters are used, mostly based on making Gaussian approximations. The purpose of this work is to study the properties of these ad hoc filters, working in the context of the 2D incompressible Navier-Stokes equation. By working in this infinite dimensional setting we provide an analysis which is useful for understanding high dimensional filtering, and is robust to mesh-refinement. We describe theoretical results showing that, in the small observational noise limit, the filters can be tuned to accurately track the signal itself (filter stability), provided the system is observed in a sufficiently large low dimensional space; roughly speaking this space should be large enough to contain the unstable modes of the linearized dynamics. Numerical results are given which illustrate the theory. In a simplified scenario we also derive, and study numerically, a stochastic PDE which determines filter stability in the limit of frequent observations, subject to large observational noise. The positive results herein concerning filter stability complement recent numerical studies which demonstrate that the ad hoc filters perform poorly in reproducing statistical variation about the true signal.

math.OC↗

arXiv2011

Gamma Limit for Transition Paths of Maximal Probability

Chemical reactions can be modelled via diffusion processes conditioned to make a transition between specified molecular configurations representing the state of the system before and after the chemical reaction. In particular the model of Brownian dynamics - gradient flow subject to additive noise - is frequently used. If the chemical reaction is specified to take place on a given time interval, then the most likely path taken by the system is a minimizer of the Onsager-Machlup functional. The Gamma limit of this functional is determined in the case where the temperature is small and the transition time scales as the inverse temperature

math.PR↗

arXiv2010

Multiscale Modelling and Inverse Problems

The need to blend observational data and mathematical models arises in many applications and leads naturally to inverse problems. Parameters appearing in the model, such as constitutive tensors, initial conditions, boundary conditions, and forcing can be estimated on the basis of observed data. The resulting inverse problems are often ill-posed and some form of regularization is required. These notes discuss parameter estimation in situations where the unknown parameters vary across multiple scales. We illustrate the main ideas using a simple model for groundwater flow. We will highlight various approaches to regularization for inverse problems, including Tikhonov and Bayesian methods. We illustrate three ideas that arise when considering inverse problems in the multiscale context. The first idea is that the choice of space or set in which to seek the solution to the inverse problem is intimately related to whether a homogenized or full multiscale solution is required. This is a choice of regularization. The second idea is that, if a homogenized solution to the inverse problem is what is desired, then this can be recovered from carefully designed observations of the full multiscale system. The third idea is that the theory of homogenization can be used to improve the estimation of homogenized coefficients from multiscale data.

math.ST↗

arXiv2009

Approximation of Bayesian Inverse Problems for PDEs

Inverse problems are often ill-posed, with solutions that depend sensitively on data. In any numerical approach to the solution of such problems, regularization of some form is needed to counteract the resulting instability. This paper is based on an approach to regularization, employing a Bayesian formulation of the problem, which leads to a notion of well-posedness for inverse problems, at the level of probability measures. The stability which results from this well-posedness may be used as the basis for quantifying the approximation, in finite dimensional spaces, of inverse problems for functions. This paper contains a theory which utilizes the stability to estimate the distance between the true and approximate posterior distributions, in the Hellinger metric, in terms of error estimates for approximation of the underlying forward problem. This is potentially useful as it allows for the transfer of estimates from the numerical analysis of forward problems into estimates for the solution of the related inverse problem. In particular controlling differences in the Hellinger metric leads to control on the differences between expected values of polynomially bounded functions and operators, including the mean and covariance operator. The ideas are illustrated with the classical inverse problem for the heat equation, and then applied to some more complicated non-Gaussian inverse problems arising in data assimilation, involving determination of the initial condition for the Stokes or Navier-Stokes equation from Lagrangian and Eulerian observations respectively.

math.NA↗

arXiv2008

Calculating Effective Diffusivities in the Limit of Vanishing Molecular Diffusion

In this paper we study the problem of the numerical calculation (by Monte Carlo Methods) of the effective diffusivity for a particle moving in a periodic divergent-free velocity filed, in the limit of vanishing molecular diffusion. In this limit traditional numerical methods typically fail, since they do not represent accurately the geometry of the underlying deterministic dynamics. We propose a stochastic splitting method that takes into account the volume preserving property of the equations motion in the absence of noise, and when inertial effects can be neglected. An extension of the method is then proposed for the cases where the noise has a non trivial time-correlation structure and when inertial effects cannot be neglected. Modified equations are used to perform backward error analysis. The new stochastic geometric integrators are shown to outperform standard Euler-based integrators. Various asymptotic limits of physical interest are investigated by means of numerical experiments, using the new integrators.

math.NA↗

arXiv2008

Maximum Likelihood Drift Estimation for Multiscale Diffusions

We study the problem of parameter estimation using maximum likelihood for fast/slow systems of stochastic differential equations. Our aim is to shed light on the problem of model/data mismatch at small scales. We consider two classes of fast/slow problems for which a closed coarse-grained equation for the slow variables can be rigorously derived, which we refer to as averaging and homogenization problems. We ask whether, given data from the slow variable in the fast/slow system, we can correctly estimate parameters in the drift of the coarse-grained equation for the slow variable, using maximum likelihood. We show that, whereas the maximum likelihood estimator is asymptotically unbiased for the averaging problem, for the homogenization problem maximum likelihood fails unless we subsample the data at an appropriate rate. An explicit formula for the asymptotic error in the log likelihood function is presented. Our theory is applied to two simple examples from molecular dynamics.

math.ST↗

arXiv2007

Parameter Estimation for Partially Observed Hypoelliptic Diffusions

Hypoelliptic diffusion processes can be used to model a variety of phenomena in applications ranging from molecular dynamics to audio signal analysis. We study parameter estimation for such processes in situations where we observe some components of the solution at discrete times. Since exact likelihoods for the transition densities are typically not known, approximations are used that are expected to work well in the limit of small inter-sample times $Δt$ and large total observation times $NΔt$. Hypoellipticity together with partial observation leads to ill-conditioning requiring a judicious combination of approximate likelihoods for the various parameters to be estimated. We combine these in a deterministic scan Gibbs sampler alternating between missing data in the unobserved solution components, and parameters. Numerical experiments illustrate asymptotic consistency of the method when applied to simulated data. The paper concludes with application of the Gibbs sampler to molecular dynamics data.

stat.ME↗

arXiv2007

Analysis of SPDEs arising in path sampling part II: The nonlinear case

In many applications, it is important to be able to sample paths of SDEs conditional on observations of various kinds. This paper studies SPDEs which solve such sampling problems. The SPDE may be viewed as an infinite-dimensional analogue of the Langevin equation used in finite-dimensional sampling. In this paper, conditioned nonlinear SDEs, leading to nonlinear SPDEs for the sampling, are studied. In addition, a class of preconditioned SPDEs is studied, found by applying a Green's operator to the SPDE in such a way that the invariant measure remains unchanged; such infinite dimensional evolution equations are important for the development of practical algorithms for sampling infinite dimensional problems. The resulting SPDEs provide several significant challenges in the theory of SPDEs. The two primary ones are the presence of nonlinear boundary conditions, involving first order derivatives, and a loss of the smoothing property in the case of the pre-conditioned SPDEs. These challenges are overcome and a theory of existence, uniqueness and ergodicity is developed in sufficient generality to subsume the sampling problems of interest to us. The Gaussian theory developed in Part I of this paper considers Gaussian SDEs, leading to linear Gaussian SPDEs for sampling. This Gaussian theory is used as the basis for deriving nonlinear SPDEs which affect the desired sampling in the nonlinear case, via a change of measure.

math.PR↗

arXiv2007

Homogenization for Inertial Particles in a Random Flow

We study the problem of homogenization for inertial particles moving in a time dependent random velocity field and subject to molecular diffusion. We show that, under appropriate assumptions on the velocity field, the large--scale, long--time behavior of the inertial particles is governed by an effective diffusion equation for the position variable alone. This is achieved by the use of a formal multiple scales expansion in the scale parameter. The expansion relies on the hypoellipticity of the underlying diffusion. An expression for the diffusivity tensor is found and various of its properties are studied. The results of the formal multiscale analysis are justified rigorously by the use of the martingale central limit theorem. Our theoretical findings are supported by numerical investigations where we study the parametric dependence of the effective diffusivity on the various non--dimensional parameters of the problem.

math-ph↗

arXiv2006

An Adaptive Euler-Maruyama Scheme For SDEs: Convergence and Stability

The understanding of adaptive algorithms for SDEs is an open area where many issues related to both convergence and stability (long time behaviour) of algorithms are unresolved. This paper considers a very simple adaptive algorithm, based on controlling only the drift component of a time-step. Both convergence and stability are studied. The primary issue in the convergence analysis is that the adaptive method does not necessarily drive the time-steps to zero with the user-input tolerance. This possibility must be quantified and shown to have low probability. The primary issue in the stability analysis is ergodicity. It is assumed that the noise is non-degenerate, so that the diffusion process is elliptic, and the drift is assumed to satisfy a coercivity condition. The SDE is then geometrically ergodic (converges to statistical equilibrium exponentially quickly). If the drift is not linearly bounded then explicit fixed time-step approximations, such as the Euler-Maruyama scheme, may fail to be ergodic. In this work, it is shown that the simple adaptive time-stepping strategy cures this problem. In addition to proving ergodicity, an exponential moment bound is also proved, generalizing a result known to hold for the SDE itself.

math.NA↗

arXiv2006

Parameter Estimation for Multiscale Diffusions

We study the problem of parameter estimation for time-series possessing two, widely separated, characteristic time scales. The aim is to understand situations where it is desirable to fit a homogenized singlescale model to such multiscale data. We demonstrate, numerically and analytically, that if the data is sampled too finely then the parameter fit will fail, in that the correct parameters in the homogenized model are not identified. We also show, numerically and analytically, that if the data is subsampled at an appropriate rate then it is possible to estimate the coefficients of the homogenized model correctly.

math.ST↗

arXiv2006

Analysis of SPDEs Arising in Path Sampling Part I: The Gaussian Case

In many applications it is important to be able to sample paths of SDEs conditional on observations of various kinds. This paper studies SPDEs which solve such sampling problems. The SPDE may be viewed as an infinite dimensional analogue of the Langevin SDE used in finite dimensional sampling. Here the theory is developed for conditioned Gaussian processes for which the resulting SPDE is linear. Applications include the Kalman-Bucy filter/smoother. A companion paper studies the nonlinear case, building on the linear analysis provided here.

math.PR↗