arXiv subjects

Melina A. Freitag

Publications and source records attributed to Melina A. Freitag.

11 recordsLinked to original sources

arXiv2026

Dimension and model reduction approaches for linear Bayesian inverse problems with rank-deficient prior covariances

Bayesian inverse problems use observed data to update a prior probability distribution for an unknown state or parameter of a scientific system to a posterior distribution conditioned on the data. In many applications, the unknown parameter is high-dimensional, making computation of the posterior expensive due to the need to sample in a high-dimensional space and the need to evaluate an expensive high-dimensional forward model relating the unknown parameter to the data. However, inverse problems often exhibit low-dimensional structure due to the fact that the available data are only informative in a low-dimensional subspace of the parameter space. Dimension reduction approaches exploit this structure by restricting inference to the low-dimensional subspace informed by the data, which can be sampled more efficiently. Further computational cost reductions can be achieved by replacing expensive high-dimensional forward models with cheaper lower-dimensional reduced models. In this work, we propose new dimension and model reduction approaches for linear Bayesian inverse problems with rank-deficient prior covariances, which arise in many practical inference settings. The dimension reduction approach is applicable to general linear Bayesian inverse problems whereas the model reduction approaches are specific to the problem of inferring the initial condition of a linear dynamical system. We provide theoretical approximation guarantees as well as numerical experiments demonstrating the accuracy and efficiency of the proposed approaches.

math.NA↗

arXiv2025

Chebyshev HOPGD with sparse grid sampling for parameterized linear systems

We consider approximating solutions to parameterized linear systems of the form $A(μ_1,μ_2) x(μ_1,μ_2) = b$. Here the matrix $A(μ_1,μ_2) \in \mathbb{R}^{n \times n}$ is nonsingular, large, and sparse and depends nonlinearly on the parameters. Specifically, the system arises from a discretization of a partial differential equation and $x(μ_1,μ_2) \in \mathbb{R}^n$, $b \in \mathbb{R}^n$. The treatment of linear systems with nonlinear dependence on a single parameter has been well-studied, and robust methods combining companion linearization, Krylov subspace methods, and Chebyshev interpolation have enabled fast solution for multiple parameter values at the cost of a single iteration. Solution of systems depending nonlinearly on multiple parameters is more challenging. This work overcomes those additional challenges by combining companion linearization, the Krylov subspace method preconditioned bi-conjugate gradient (BiCG), and a decomposition of a tensor matrix of precomputed solutions, called snapshots. This produces a reduced order model of $x(μ_1,μ_2)$, and this model can be evaluated inexpensively for many values of the parameters. An interpolation of the model is used to produce approximations on the entire parameter space. In addition this method can be used to solve a parameter estimation problem. This approach allows us to achieve similar computational savings as for the one-parameter case; we can solve for many parameter pairs at the cost of many fewer applications of an efficient iterative method. The technique is presented for dependence on two parameters, but the strategy can be extended to more parameters using the same approach. Numerical examples of a parameterized Helmholtz equation show the competitiveness of our approach.

math.NA↗

arXiv2023

Time-limited Balanced Truncation for Data Assimilation Problems

Balanced truncation is a well-established model order reduction method which has been applied to a variety of problems. Recently, a connection between linear Gaussian Bayesian inference problems and the system-theoretic concept of balanced truncation has been drawn. Although this connection is new, the application of balanced truncation to data assimilation is not a novel idea: it has already been used in four-dimensional variational data assimilation (4D-Var). This paper discusses the application of balanced truncation to linear Gaussian Bayesian inference, and, in particular, the 4D-Var method, thereby strengthening the link between systems theory and data assimilation further. Similarities between both types of data assimilation problems enable a generalisation of the state-of-the-art approach to the use of arbitrary prior covariances as reachability Gramians. Furthermore, we propose an enhanced approach using time-limited balanced truncation that allows to balance Bayesian inference for unstable systems and in addition improves the numerical results for short observation periods.

math.NA↗

arXiv2023

Low-rank solutions to the stochastic Helmholtz equation

In this paper, we consider low-rank approximations for the solutions to the stochastic Helmholtz equation with random coefficients. A Stochastic Galerkin finite element method is used for the discretization of the Helmholtz problem. Existence theory for the low-rank approximation is established when the system matrix is indefinite. The low-rank algorithm does not require the construction of a large system matrix which results in an advantage in terms of CPU time and storage. Numerical results show that, when the operations in a low-rank method are performed efficiently, it is possible to obtain an advantage in terms of storage and CPU time compared to computations in full rank. We also propose a general approach to implement a preconditioner using the low-rank format efficiently.

math.NA↗

arXiv2023

Solving the Parametric Eigenvalue Problem by Taylor Series and Chebyshev Expansion

We discuss two approaches to solving the parametric (or stochastic) eigenvalue problem. One of them uses a Taylor expansion and the other a Chebyshev expansion. The parametric eigenvalue problem assumes that the matrix $A$ depends on a parameter $μ$, where $μ$ might be a random variable. Consequently, the eigenvalues and eigenvectors are also functions of $μ$. We compute a Taylor approximation of these functions about $μ_{0}$ by iteratively computing the Taylor coefficients. The complexity of this approach is $O(n^{3})$ for all eigenpairs, if the derivatives of $A(μ)$ at $μ_{0}$ are given. The Chebyshev expansion works similarly. We first find an initial approximation iteratively which we then refine with Newton's method. This second method is more expensive but provides a good approximation over the whole interval of the expansion instead around a single point. We present numerical experiments confirming the complexity and demonstrating that the approaches are capable of tracking eigenvalues at intersection points. Further experiments shed light on the limitations of the Taylor expansion approach with respect to the distance from the expansion point $μ_{0}$.

math.NA↗

arXiv2020

Optimization based model order reduction for stochastic systems

In this paper, we bring together the worlds of model order reduction for stochastic linear systems and $\mathcal H_2$-optimal model order reduction for deterministic systems. In particular, we supplement and complete the theory of error bounds for model order reduction of stochastic differential equations. With these error bounds, we establish a link between the output error for stochastic systems (with additive and multiplicative noise) and modified versions of the $\mathcal H_2$-norm for both linear and bilinear deterministic systems. When deriving the respective optimality conditions for minimizing the error bounds, we see that model order reduction techniques related to iterative rational Krylov algorithms (IRKA) are very natural and effective methods for reducing the dimension of large-scale stochastic systems with additive and/or multiplicative noise. We apply modified versions of (linear and bilinear) IRKA to stochastic linear systems and show their efficiency in numerical experiments.

math.NA↗

arXiv2020

Numerical Linear Algebra in Data Assimilation

Data assimilation is a method that combines observations (that is, real world data) of a state of a system with model output for that system in order to improve the estimate of the state of the system and thereby the model output. The model is usually represented by a discretised partial differential equation. The data assimilation problem can be formulated as a large scale Bayesian inverse problem. Based on this interpretation we will derive the most important variational and sequential data assimilation approaches, in particular three-dimensional and four-dimensional variational data assimilation (3D-Var and 4D-Var) and the Kalman filter. We will then consider more advanced methods which are extensions of the Kalman filter and variational data assimilation and pay particular attention to their advantages and disadvantages. The data assimilation problem usually results in a very large optimisation problem and/or a very large linear system to solve (due to inclusion of time and space dimensions). Therefore, the second part of this article aims to review advances and challenges, in particular from the numerical linear algebra perspective, within the various data assimilation approaches.

math.NA↗

arXiv2019

Inexact methods for the low rank solution to large scale Lyapunov equations

The rational Krylov subspace method (RKSM) and the low-rank alternating directions implicit (LR-ADI) iteration are established numerical tools for computing low-rank solution factors of large-scale Lyapunov equations. In order to generate the basis vectors for the RKSM, or extend the low-rank factors within the LR-ADI method the repeated solution to a shifted linear system is necessary. For very large systems this solve is usually implemented using iterative methods, leading to inexact solves within this inner iteration. We derive theory for a relaxation strategy within these inexact solves, both for the RKSM and the LR-ADI method. Practical choices for relaxing the solution tolerance within the inner linear system are then provided. The theory is supported by several numerical examples.

math.NA↗

arXiv2017

A low-rank approach to the solution of weak constraint variational data assimilation problems

Weak constraint four-dimensional variational data assimilation is an important method for incorporating data (typically observations) into a model. The linearised system arising within the minimisation process can be formulated as a saddle point problem. A disadvantage of this formulation is the large storage requirements involved in the linear system. In this paper, we present a low-rank approach which exploits the structure of the saddle point system using techniques and theory from solving large scale matrix equations. Numerical experiments with the linear advection-diffusion equation, and the non-linear Lorenz-95 model demonstrate the effectiveness of a low-rank Krylov subspace solver when compared to a traditional solver.

math.NA↗

arXiv2016

Balanced truncation and singular perturbation approximation model order reduction for stochastically controlled linear systems

When solving linear stochastic differential equations numerically, usually a high order spatial discretisation is used. Balanced truncation (BT) and singular perturbation approximation (SPA) are well-known projection techniques in the deterministic framework which reduce the order of a control system and hence reduce computational complexity. This work considers both methods when the control is replaced by a noise term. We provide theoretical tools such as stochastic concepts for reachability and observability, which are necessary for balancing related model order reduction of linear stochastic differential equations with additive Lévy noise. Moreover, we derive error bounds for both BT and SPA and provide numerical results for a specific example which support the theory.

math.NA↗

arXiv2012

The calculation of the distance to a nearby defective matrix

In this paper a new fast algorithm for the computation of the distance of a matrix to a nearby defective matrix is presented. The problem is formulated following Alam & Bora (Linear Algebra Appl., 396 (2005), pp.~273--301) and reduces to finding when a parameter-dependent matrix is singular subject to a constraint. The solution is achieved by an extension of the Implicit Determinant Method introduced by Spence & Poulton (J. Comput. Phys., 204 (2005), pp.~65--81). Numerical results for several examples illustrate the performance of the algorithm.

math.NA↗