arXiv subjects

Alexandre J. Chorin

Publications and source records attributed to Alexandre J. Chorin.

At least 19 recordsLinked to original sources

arXiv2016

Comparison of continuous and discrete-time data-based modeling for hypoelliptic systems

We compare two approaches to the predictive modeling of dynamical systems from partial observations at discrete times. The first is continuous in time, where one uses data to infer a model in the form of stochastic differential equations, which are then discretized for numerical solution. The second is discrete in time, where one directly infers a discrete-time model in the form of a nonlinear autoregression moving average model. The comparison is performed in a special case where the observations are known to have been obtained from a hypoelliptic stochastic differential equation. We show that the discrete-time approach has better predictive skills, especially when the data are relatively sparse in time. We discuss open questions as well as the broader significance of the results.

math.NA↗

arXiv2015

Data-based stochastic model reduction for the Kuramoto--Sivashinsky equation

The problem of constructing data-based, predictive, reduced models for the Kuramoto-Sivashinsky equation is considered, under circumstances where one has observation data only for a small subset of the dynamical variables. Accurate prediction is achieved by developing a discrete-time stochastic reduced system, based on a NARMAX (Nonlinear Autoregressive Moving Average with eXogenous input) representation. The practical issue, with the NARMAX representation as with any other, is to identify an efficient structure, i.e., one with a small number of terms and coefficients. This is accomplished here by estimating coefficients for an approximate inertial form. The broader significance of the results is discussed.

math.NA↗

arXiv2015

Sampling, feasibility, and priors in Bayesian estimation

Importance sampling algorithms are discussed in detail, with an emphasis on implicit sampling, and applied to data assimilation via particle filters. Implicit sampling makes it possible to use the data to find high-probability samples at relatively low cost, making the assimilation more efficient. A new analysis of the feasibility of data assimilation is presented, showing in detail why feasibility depends on the Frobenius norm of the covariance matrix of the noise and not on the number of variables. A discussion of the convergence of particular particle filters follows. A major open problem in numerical data assimilation is the determination of appropriate priors, a progress report on recent work on this problem is given. The analysis highlights the need for a careful attention both to the data and to the physics in data assimilation problems.

stat.CO↗

arXiv2015

A discrete approach to stochastic parametrization and dimensional reduction in nonlinear dynamics

Many physical systems are described by nonlinear differential equations that are too complicated to solve in full. A natural way to proceed is to divide the variables into those that are of direct interest and those that are not, formulate solvable approximate equations for the variables of greater interest, and use data and statistical methods to account for the impact of the other variables. In the present paper the problem is considered in a fully discrete-time setting, which simplifies both the analysis of the data and the numerical algorithms. The resulting time series are identified by a NARMAX (nonlinear autoregression moving average with exogenous input) representation familiar from engineering practice. The connections with the Mori-Zwanzig formalism of statistical physics are discussed, as well as an application to the Lorenz 96 system.

math.NA↗

arXiv2014

Conditions for successful data assimilation

We show, using idealized models, that numerical data assimilation can be successful only if an effective dimension of the problem is not excessive. This effective dimension depends on the noise in the model and the data, and in physically reasonable problems it can be moderate even when the number of variables is huge. We then analyze several data assimilation algorithms, including particle filters and variational methods. We show that well-designed particle filters can solve most of those data assimilation problems that can be solved in principle, and compare the conditions under which variational methods can succeed to the conditions required of particle filters. We also discuss the limitations of our analysis.

math-ph↗

arXiv2014

Path integral formulation of stochastic optimal control with generalized costs

Path integral control solves a class of stochastic optimal control problems with a Monte Carlo (MC) method for an associated Hamilton-Jacobi-Bellman (HJB) equation. The MC approach avoids the need for a global grid of the domain of the HJB equation and, therefore, path integral control is in principle applicable to control problems of moderate to large dimension. The class of problems path integral control can solve, however, is defined by requirements on the cost function, the noise covariance matrix and the control input matrix. We relax the requirements on the cost function by introducing a new state that represents an augmented running cost. In our new formulation the cost function can contain stochastic integral terms and linear control costs, which are important in applications in engineering, economics and finance. We find an efficient numerical implementation of our grid-free MC approach and demonstrate its performance and usefulness in examples from hierarchical electric load management. The dimension of one of our examples is large enough to make classical grid-based HJB solvers impractical.

math.OC↗

arXiv2014

Limitations of polynomial chaos expansions in the Bayesian solution of inverse problems

Polynomial chaos expansions are used to reduce the computational cost in the Bayesian solutions of inverse problems by creating a surrogate posterior that can be evaluated inexpensively. We show, by analysis and example, that when the data contain significant information beyond what is assumed in the prior, the surrogate posterior can be very different from the posterior, and the resulting estimates become inaccurate. One can improve the accuracy by adaptively increasing the order of the polynomial chaos, but the cost may increase too fast for this to be cost effective compared to Monte Carlo sampling without a surrogate posterior.

math.NA↗

arXiv2013

Parameter estimation by implicit sampling

Implicit sampling is a weighted sampling method that is used in data assimilation, where one sequentially updates estimates of the state of a stochastic model based on a stream of noisy or incomplete data. Here we describe how to use implicit sampling in parameter estimation problems, where the goal is to find parameters of a numerical model, e.g.~a partial differential equation (PDE), such that the output of the numerical model is compatible with (noisy) data. We use the Bayesian approach to parameter estimation, in which a posterior probability density describes the probability of the parameter conditioned on data and compute an empirical estimate of this posterior with implicit sampling. Our approach generates independent samples, so that some of the practical difficulties one encounters with Markov Chain Monte Carlo methods, e.g.~burn-in time or correlations among dependent samples, are avoided. We describe a new implementation of implicit sampling for parameter estimation problems that makes use of multiple grids (coarse to fine) and BFGS optimization coupled to adjoint equations for the required gradient calculations. The implementation is "dimension independent", in the sense that a well-defined finite dimensional subspace is sampled as the mesh used for discretization of the PDE is refined. We illustrate the algorithm with an example where we estimate a diffusion coefficient in an elliptic equation from sparse and noisy pressure measurements. In the example, dimension\slash mesh-independence is achieved via Karhunen-Lo\`{e}ve expansions.

math.NA↗

arXiv2013

Estimating the uncertainty in underresolved nonlinear dynamics

The Mori-Zwanzig formalism of statistical mechanics is used to estimate the uncertainty caused by underresolution in the solution of a nonlinear dynamical system. A general approach is outlined and applied to a simple example. The noise term that describes the uncertainty turns out to be neither Markovian nor Gaussian. It is argued that this is the general situation.

math.NA↗

arXiv2012

Implicit particle methods and their connection with variational data assimilation

The implicit particle filter is a sequential Monte Carlo method for data assimilation that guides the particles to the high-probability regions via a sequence of steps that includes minimizations. We present a new and more general derivation of this approach and extend the method to particle smoothing as well as to data assimilation for perfect models. We show that the minimizations required by implicit particle methods are similar to the ones one encounters in variational data assimilation and explore the connection of implicit particle methods with variational data assimilation. In particular, we argue that existing variational codes can be converted into implicit particle methods at a low cost, often yielding better estimates, that are also equipped with quantitative measures of the uncertainty. A detailed example is presented.

physics.data-an↗

arXiv2011

Implicit particle filtering for models with partial noise, and an application to geomagnetic data assimilation

Implicit particle filtering is a sequential Monte Carlo method for data assim- ilation, designed to keep the number of particles manageable by focussing attention on regions of large probability. These regions are found by min- imizing, for each particle, a scalar function F of the state variables. Some previous implementations of the implicit filter rely on finding the Hessians of these functions. The calculation of the Hessians can be cumbersome if the state dimension is large or if the underlying physics are such that derivatives of F are difficult to calculate. This is the case in many geophysical applica- tions, in particular for models with partial noise, i.e. with a singular state covariance matrix. Examples of models with partial noise include stochastic partial differential equations driven by spatially smooth noise processes and models for which uncertain dynamic equations are supplemented by con- servation laws with zero uncertainty. We make the implicit particle filter applicable to such situations by combining gradient descent minimization with random maps and show that the filter is efficient, accurate and reliable because it operates in a subspace whose dimension is smaller than the state dimension. As an example, we assimilate data for a system of nonlinear partial differential equations that appears in models of geomagnetism.

math.NA↗

arXiv2011

A random map implementation of implicit filters

Implicit particle filters for data assimilation generate high-probability samples by representing each particle location as a separate function of a common reference variable. This representation requires that a certain underdetermined equation be solved for each particle and at each time an observation becomes available. We present a new implementation of implicit filters in which we find the solution of the equation via a random map. As examples, we assimilate data for a stochastically driven Lorenz system with sparse observations and for a stochastic Kuramoto-Sivashinski equation with observations that are sparse in both space and time.

math.NA↗

arXiv2010

Implicit particle filters for data assimilation

Implicit particle filters for data assimilation update the particles by first choosing probabilities and then looking for particle locations that assume them, guiding the particles one by one to the high probability domain. We provide a detailed description of these filters, with illustrative examples, together with new, more general, methods for solving the algebraic equations and with a new algorithm for parameter identification.

math.NA↗

arXiv2009

Interpolation and Iteration for Nonlinear Filters

We present a general form of the iteration and interpolation process used in implicit particle filters. Implicit filters are based on a pseudo-Gaussian representation of posterior densities, and are designed to focus the particle paths so as to reduce the number of particles needed in nonlinear data assimilation. Examples are given.

math.NA↗

arXiv2009

Non-Bayesian particle filters

Particle filters for data assimilation in nonlinear problems use "particles" (replicas of the underlying system) to generate a sequence of probability density functions (pdfs) through a Bayesian process. This can be expensive because a significant number of particles has to be used to maintain accuracy. We offer here an alternative, in which the relevant pdfs are sampled directly by an iteration. An example is discussed in detail.

math.NA↗

arXiv2005

Numerical scaling analysis of the small-scale structure in turbulence

We show how to use numerical methods within the framework of successive scaling to analyse the microstructure of turbulence, in particular to find inertial range exponents and structure functions. The methods are first calibrated on the Burgers problem and are then applied to the 3D Euler equations. Known properties of low order structure functions appear with a relatively small computational outlay; however, more sensitive properties cannot yet be resolved with this approach well enough to settle ongoing controversies.

math.NA↗

arXiv2005

Problem reduction, renormalization, and memory

Methods for the reduction of the complexity of computational problems are presented, as well as their connections to renormalization, scaling, and irreversible statistical mechanics. Several statistically stationary cases are analyzed; for time dependent problems averaging usually fails, and averaged equations must be augmented by appropriate memory and random forcing terms. Approximations are described and examples are given.

math.NA↗

arXiv2002

Conditional Expectations and Renormalization

In optimal prediction methods one estimates the future behavior of underresolved systems by solving reduced systems of equations for expectations conditioned by partial data; renormalization group methods reduce the number of variables in complex systems through integration of unwanted scales. We establish the relation between these methods for systems in thermal equilibrium, and use this relation to find renormalization parameter flows and the coefficients in reduced systems by expanding conditional expectations in series and evaluating the coefficients by Monte-Carlo. We illustrate the construction by finding parameter flows for simple spin systems and then using the renormalized (=reduced) systems to calculate the critical temperature and the magnetization.

math-ph↗