arXiv subjects

Adrian Sandu

Publications and source records attributed to Adrian Sandu.

At least 55 records · Page 3Linked to original sources

arXiv2020

An Explicit Probabilistic Derivation of Inflation in a Scalar Ensemble Kalman Filter for Finite Step, Finite Ensemble Convergence

This paper uses a probabilistic approach to analyze the converge of an ensemble Kalman filter solution to an exact Kalman filter solution in the simplest possible setting, the scalar case, as it allows us to build upon a rich literature of scalar probability distributions and non-elementary functions. To this end we introduce the bare-bones Scalar Pedagogical Ensemble Kalman Filter (SPEnKF). We show that in the asymptotic case of ensemble size, the expected value of both the analysis mean and variance estimate of the SPEnKF converges to that of the true Kalman filter, and that the variances of both tend towards zero, at each time moment. We also show that the ensemble converges in probability in the complementary case, when the ensemble is finite, and time is taken to infinity. Moreover, we show that in the finite-ensemble, finite-time case, variance inflation and mean correction can be leveraged to coerce the SPEnKF converge to its scalar Kalman filter counterpart. We then apply this framework to analyze perturbed observations and explain why perturbed observations ensemble Kalman filters underperform their deterministic counterparts.

math.OC↗

arXiv2020

Coupled Multirate Infinitesimal GARK Schemes for Stiff Systems with Multiple Time Scales

Traditional time discretization methods use a single timestep for the entire system of interest and can perform poorly when the dynamics of the system exhibits a wide range of time scales. Multirate infinitesimal step (MIS) methods (Knoth and Wolke, 1998) offer an elegant and flexible approach to efficiently integrate such systems. The slow components are discretized by a Runge-Kutta method, and the fast components are resolved by solving modified fast differential equations. Sandu (2018) developed the Multirate Infinitesimal General-structure Additive Runge-Kutta (MRI-GARK) family of methods that includes traditional MIS schemes as a subset. The MRI-GARK framework allowed the construction of the first fourth order MIS schemes. This framework also enabled the introduction of implicit methods, which are decoupled in the sense that any implicitness lies entirely within the fast or slow integrations. It was shown by Sandu that the stability of decoupled implicit MRI-GARK methods has limitations when both the fast and slow components are stiff and interact strongly. This work extends the MRI-GARK framework by introducing coupled implicit methods to solve stiff multiscale systems. The coupled approach has the potential to considerably improve the overall stability of the scheme, at the price of requiring implicit stage calculations over the entire system. Two coupling strategies are considered. The first computes coupled Runge-Kutta stages before solving a single differential equation to refine the fast solution. The second alternates between computing coupled Runge-Kutta stages and solving fast differential equations. We derive order conditions and perform the stability analysis for both strategies. The new coupled methods offer improved stability compared to the decoupled MRI-GARK schemes. The theoretical properties of the new methods are validated with numerical experiments.

math.NA↗

arXiv2020

Goal-oriented a posteriori estimation of numerical errors in the solution of multiphysics systems

This paper develops a general methodology for a posteriori error estimation in time-dependent multiphysics numerical simulations. The methodology builds upon the generalized-structure additive Runge--Kutta (GARK) approach to time integration. GARK provides a unified formulation of multimethods that simulate complex systems by applying different discretization formulas and/or different time steps to individual components of the system. We derive discrete GARK adjoints and analyze their time accuracy. Based on the adjoint method, we establish computable a posteriori identities for the impacts of both temporal and spatial discretization errors on a given goal function. Numerical examples with reaction-diffusion systems illustrate the accuracy of the derived error measures. Local error decompositions are used to illustrate the power of this framework in adaptive refinements of both temporal and spatial meshes.

math.NA↗

arXiv2019

Alternating Directions Implicit Integration in a General Linear Method Framework

Alternating Directions Implicit (ADI) integration is an operator splitting approach to solve parabolic and elliptic partial differential equations in multiple dimensions based on solving sequentially a set of related one-dimensional equations. Classical ADI methods have order at most two, due to the splitting errors. Moreover, when the time discretization of stiff one-dimensional problems is based on Runge-Kutta schemes, additional order reduction may occur. This work proposes a new ADI approach based on the partitioned General Linear Methods framework. This approach allows the construction of high order ADI methods. Due to their high stage order, the proposed methods can alleviate the order reduction phenomenon seen with other schemes. Numerical experiments are shown to provide further insight into the accuracy, stability, and applicability of these new methods.

math.NA↗

arXiv2019

Efficient implementation of partitioned stiff exponential Runge-Kutta methods

Multiphysics systems are driven by multiple processes acting simultaneously, and their simulation leads to partitioned systems of differential equations. This paper studies the solution of partitioned systems of differential equations using exponential Runge-Kutta methods. We propose specific multiphysics implementations of exponential Runge-Kutta methods satisfying stiff order conditions that were developed in [Hochbruck et al., SISC, 1998] and [Luan and Osterman, JCAM, 2014]. We reformulate stiffly--accurate exponential Runge--Kutta methods in a way that naturally allows of the structure of multiphysics systems, and discuss their application to both component and additively partitioned systems. The resulting partitioned exponential methods only compute matrix functions of the Jacobians of individual components, rather than the Jacobian of the full, coupled system. We derive modified formulations of particular methods of order two, three and four, and apply them to solve a partitioned reaction-diffusion problem. The proposed methods retain full order for several partitionings of the discretized problem, including by components and by physical processes.

math.NA↗

arXiv2019

Adaptive Krylov-Type Time Integration Methods

The Rosenbrock-Krylov family of time integration schemes is an extension of Rosenbrock-W methods that employs a specific Krylov based approximation of the linear system solutions arising within each stage of the integrator. This work proposes an extension of Rosenbrock-Krylov methods to address stability questions which arise for methods making use of inexact linear system solution strategies. Two approaches for improving the stability and efficiency of Rosenbrock-Krylov methods are proposed, one through direct control of linear system residuals and the second through a novel extension of the underlying Krylov space to include stage right hand side vectors. Rosenbrock-Krylov methods employing the new approaches show a substantial improvement in computational efficiency relative to prior implementations.

math.NA↗

arXiv2019

Partitioned Exponential Methods for Coupled Multiphysics Systems

Multiphysics problems involving two or more coupled physical phenomena are ubiquitous in science and engineering. This work develops a new partitioned exponential approach for the time integration of multiphysics problems. After a possible semi-discretization in space, the class of problems under consideration is modeled by a system of ordinary differential equations where the right-hand side is a summation of two component functions, each corresponding to a given set of physical processes. The partitioned-exponential methods proposed herein evolve each component of the system via an exponential integrator, and information between partitions is exchanged via coupling terms. The traditional approach to constructing exponential methods, based on the variation-of-constants formula, is not directly applicable to partitioned systems. Rather, our approach to developing new partitioned-exponential families is based on a general-structure additive formulation of the schemes. Two method formulations are considered, one based on a linear-nonlinear splitting of the right hand component functions, and another based on approximate Jacobians. The paper develops classical (non-stiff) order conditions theory for partitioned exponential schemes based on particular families of T-trees and B-series theory. Several practical methods of third order are constructed that extend the Rosenbrock-type and EPIRK families of exponential integrators. Several implementation optimizations specific to the application of these methods to reaction-diffusion systems are also discussed. Numerical experiments reveal that the new partitioned-exponential methods can perform better than traditional unpartitioned exponential methods on some problems.

math.NA↗

arXiv2019

Biorthogonal Rosenbrock-Krylov time discretization methods

Many scientific applications require the solution of large initial-value problems, such as those produced by the method of lines after semi-discretization in space of partial differential equations. The computational cost of implicit time discretizations is dominated by the solution of nonlinear systems of equations at each time step. In order to decrease this cost, the recently developed Rosenbrock-Krylov (ROK) time integration methods extend the classical linearly-implicit Rosenbrock(-W) methods, and make use of a Krylov subspace approximation to the Jacobian computed via an Arnoldi process. Since the ROK order conditions rely on the construction of a single Krylov space, no restarting of the Arnoldi process is allowed, and the iterations quickly become expensive with increasing subspace dimensions. This work extends the ROK framework to make use of the Lanczos biorthogonalization procedure for constructing Jacobian approximations. The resulting new family of methods is named biorthogonal ROK (BOROK). The Lanczos procedure's short two-term recurrence allows BOROK methods to utilize larger subspaces for the Jacobian approximation, resulting in increased numerical stability of the time integration at a reduced computational cost. Adaptive subspace size selection and basis extension procedures are also developed for the new schemes. Numerical experiments show that for stiff problems, where a large subspace used to approximate the Jacobian is required for stability, the BOROK methods outperform the original ROK methods.

math.NA↗

arXiv2019

ODE Test Problems: a MATLAB suite of initial value problems

ODE Test Problems (OTP) is an object-oriented MATLAB package offering a broad range of initial value problems which can be used to test numerical methods such as time integration methods and data assimilation (DA) methods. It includes problems that are linear and nonlinear, homogeneous and nonhomogeneous, autonomous and nonautonomous, scalar and high-dimensional, stiff and nonstiff, and chaotic and nonchaotic. Many are real-world problems from fields such as chemistry, astrophysics, meteorology, and electrical engineering. OTP also supports partitioned ODEs for testing IMEX methods, multirate methods, and other multimethods. Functions for plotting solutions and creating movies are available for all problems, and exact solutions are provided when available. OTP is desgined for ease of use-meaning that working with and modifying problems is simple and intuitive.

math.NA↗

arXiv2018

A Bayesian Approach to Multivariate Adaptive Localization in Ensemble-Based Data Assimilation with Time-Dependent Extensions

Ever since its inception, the Ensemble Kalman Filter has elicited many heuristic methods that sought to correct it. One such method is localization---the thought that `nearby' variables should be highly correlated with `far away' variable not. Recognizing that correlation is a time-dependent property, adaptive localization is a natural extension to these heuristics. We propose a Bayesian approach to adaptive Schur-product localization for the DEnKF, and extend it to support multiple radii of influence. We test both the empirical validity of (multivariate) adaptive localization, and of our approach. We test a simple toy problem (Lorenz'96), extending it to a multivariate model, and a more realistic geophysical problem (1.5 Layer Quasi-Geostrophic). We show that the multivariate approach has great promise on the toy problem, and that the univariate approach leads to improved filter performance for the realistic geophysical problem.

math.NA↗

arXiv2018

DATeS: A Highly-Extensible Data Assimilation Testing Suite v1.0

A flexible and highly-extensible data assimilation testing suite, named DATeS, is described in this paper. DATeS aims to offer a unified testing environment that allows researchers to compare different data assimilation methodologies and understand their performance in various settings. The core of DATeS is implemented in Python and takes advantage of its object-oriented capabilities. The main components of the package (the numerical models, the data assimilation algorithms, the linear algebra solvers, and the time discretization routines) are independent of each other, which offers great flexibility to configure data assimilation applications. DATeS can interface easily with large third-party numerical models written in Fortran or in C, and with a plethora of external solvers.

cs.MS↗

arXiv2018

A Learning Based Approach for Uncertainty Analysis in Numerical Weather Prediction Models

Complex numerical weather prediction models incorporate a variety of physical processes, each described by multiple alternative physical schemes with specific parameters. The selection of the physical schemes and the choice of the corresponding physical parameters during model configuration can significantly impact the accuracy of model forecasts. There is no combination of physical schemes that works best for all times, at all locations, and under all conditions. It is therefore of considerable interest to understand the interplay between the choice of physics and the accuracy of the resulting forecasts under different conditions. This paper demonstrates the use of machine learning techniques to study the uncertainty in numerical weather prediction models due to the interaction of multiple physical processes. The first problem addressed herein is the estimation of systematic model errors in output quantities of interest at future times, and the use of this information to improve the model forecasts. The second problem considered is the identification of those specific physical processes that contribute most to the forecast uncertainty in the quantity of interest under specified meteorological conditions. The discrepancies between model results and observations at past times are used to learn the relationships between the choice of physical processes and the resulting forecast errors. Numerical experiments are carried out with the Weather Research and Forecasting (WRF) model. The output quantity of interest is the model precipitation, a variable that is both extremely important and very challenging to forecast. The physical processes under consideration include various micro-physics schemes, cumulus parameterizations, short wave, and long wave radiation schemes. The experiments demonstrate the strong potential of machine learning approaches to aid the study of model errors.

math.NA↗

arXiv2018

Adjoint Sensitivity Analysis of Hybrid Multibody Dynamical Systems

Sensitivity analysis of multibody systems computes the derivatives of general cost functions that depend on the system solution with respect to parameters or initial conditions. This work develops adjoint sensitivity analysis for hybrid multibody dynamic systems. Hybrid systems are characterized by trajectories that are piecewise continuous in time, with finitely-many discontinuities being caused by events such as elastic/inelastic impacts or sudden changes in constraints. The corresponding direct and adjoint sensitivity variables are also discontinuous at the time of events. The framework discussed herein uses a jump sensitivity matrix to relate the jump conditions for the direct and adjoint sensitivities before and after the time event, and provides analytical jump equations for the adjoint variables. The theoretical framework for sensitivities for hybrid systems is validated on a five-bar mechanism with non-smooth contacts.

math.OC↗

arXiv2018

A Machine Learning Approach to Adaptive Covariance Localization

Data assimilation plays a key role in large-scale atmospheric weather forecasting, where the state of the physical system is estimated from model outputs and observations, and is then used as initial condition to produce accurate future forecasts. The Ensemble Kalman Filter (EnKF) provides a practical implementation of the statistical solution of the data assimilation problem and has gained wide popularity as. This success can be attributed to its simple formulation and ease of implementation. EnKF is a Monte-Carlo algorithm that solves the data assimilation problem by sampling the probability distributions involved in Bayes theorem. Because of this, all flavors of EnKF are fundamentally prone to sampling errors when the ensemble size is small. In typical weather forecasting applications, the model state space has dimension $10^{9}-10^{12}$, while the ensemble size typically ranges between $30-100$ members. Sampling errors manifest themselves as long-range spurious correlations and have been shown to cause filter divergence. To alleviate this effect covariance localization dampens spurious correlations between state variables located at a large distance in the physical space, via an empirical distance-dependent function. The quality of the resulting analysis and forecast is greatly influenced by the choice of the localization function parameters, e.g., the radius of influence. The localization radius is generally tuned empirically to yield desirable results.This work, proposes two adaptive algorithms for covariance localization in the EnKF framework, both based on a machine learning approach. The first algorithm adapts the localization radius in time, while the second algorithm tunes the localization radius in both time and space. Numerical experiments carried out with the Lorenz-96 model, and a quasi-geostrophic model, reveal the potential of the proposed machine learning approaches.

stat.ME↗

arXiv2017

Modeling and sensitivity analysis methodology for hybrid dynamical systems

This paper provides an analytical methodology to compute the sensitivities with respect to system parameters for any second order hybrid Ordinary Differential Equation (ODE) system. The hybrid ODE system is characterized by discontinuities in the velocity state variables due to an impulsive jump caused by an instantaneous impact in the motion of the system. The analytical methodology that solves this problem is structured based on jumping conditions for both the state variables and the sensitivities matrix. The proposed analytical approach is of the benchmarked against a numerical method.

math.OC↗

arXiv2017

A Numerical Investigation of Matrix-Free Implicit Time-Stepping Methods for Large CFD Simulations

This paper is concerned with the development and testing of advanced time-stepping methods suited for the integration of time-accurate, real-world applications of computational fluid dynamics (CFD). The performance of several time discretization methods is studied numerically with regards to computational efficiency, order of accuracy, and stability, as well as the ability to treat effectively stiff problems. We consider matrix-free implementations, a popular approach for time-stepping methods applied to large CFD applications due to its adherence to scalable matrix-vector operations and a small memory footprint. We compare explicit methods with matrix-free implementations of implicit, linearly-implicit, as well as Rosenbrock-Krylov methods. We show that Rosenbrock-Krylov methods are competitive with existing techniques excelling for a number of problem types and settings.

cs.CE↗

arXiv2017

Solving Parameter Estimation Problems with Discrete Adjoint Exponential Integrators

The solution of inverse problems in a variational setting finds best estimates of the model parameters by minimizing a cost function that penalizes the mismatch between model outputs and observations. The gradients required by the numerical optimization process are computed using adjoint models. Exponential integrators are a promising family of time discretizations for evolutionary partial differential equations. In order to allow the use of these discretizations in the context of inverse problems adjoints of exponential integrators are required. This work derives the discrete adjoint formulae for a W-type exponential propagation iterative methods of Runge-Kutta type (EPIRK-W). These methods allow arbitrary approximations of the Jacobian while maintaining the overall accuracy of the forward integration. The use of Jacobian approximation matrices that do not depend on the model state avoids the complex calculation of Hessians in the discrete adjoint formulae, and allows efficient adjoint code generation via algorithmic differentiation. We use the discrete EPIRK-W adjoints to solve inverse problems with the Lorenz-96 model and a computational magnetics benchmark test. Numerical results validate our theoretical derivations.

math.OC↗

arXiv2017

EPIRK-W and EPIRK-K time discretization methods

Exponential integrators are special time discretization methods where the traditional linear system solves used by implicit schemes are replaced with computing the action of matrix exponential-like functions on a vector. A very general formulation of exponential integrators is offered by the Exponential Propagation Iterative methods of Runge-Kutta type (EPIRK) family of schemes. The use of Jacobian approximations is an important strategy to drastically reduce the overall computational costs of implicit schemes while maintaining the quality of their solutions. This paper extends the EPIRK class to allow the use of inexact Jacobians as arguments of the matrix exponential-like functions. Specifically, we develop two new families of methods: EPIRK-W integrators that can accommodate any approximation of the Jacobian, and EPIRK-K integrators that rely on a specific Krylov-subspace projection of the exact Jacobian. Classical order conditions theories are constructed for these families. A practical EPIRK-W method of order three and an EPIRK-K method of order four are developed. Numerical experiments indicate that the methods proposed herein are computationally favorable when compared to existing exponential integrators.

math.NA↗