arXiv subjects

Han Cheng Lie

Publications and source records attributed to Han Cheng Lie.

At least 19 recordsLinked to original sources

arXiv2026

Error bounds for approximate posteriors from likelihood-informed reduced-order models

In the design of computational methods for Bayesian inverse problems, costly forward model evaluations make it difficult to sample from or compute the posterior. This motivates the need for approximate forward models that are cheaper to evaluate. We consider reduced-order forward models which exploit the lower-dimensional structure in the Bayesian inverse problem by projecting to the "likelihood-informed subspace" of the parameter space where the prior-to-posterior update is significant. However, the theoretical properties of these reduced-order forward models and their impact on the solution of the Baysian inverse problem are not always well-understood. In this work we consider linear Gaussian inverse problems with a possibly singular prior covariance matrix. We analyse a recently proposed reduced-order model which uses a Petrov-Galerkin projection to likelihood-informed subspaces that arise in optimal low-rank approximations of the posterior covariance matrix. We bound the error in the resulting approximation of the root prior-preconditioned Hessian of the data misfit. Based on this we also bound the errors of the approximate posterior covariance and mean. Our analysis shows that this reduced-order model recovers the exact posterior when the rank of the reduced-order model is equal to the "intrinsic dimension" of the inverse problem, i.e. the rank of the prior-preconditioned Hessian. Two numerical experiments from structural engineering illustrate the performance of our bounds.

math.NA↗

arXiv2026

Goal-oriented learning of stochastic differential equations using error bounds on path-space observables

Stochastic differential equations (SDEs), which serve as the governing equations for dynamical systems in a broad range of applications, can become cost-prohibitive for numerical simulation at scales necessary for quantifying key properties. Surrogate models of the drift function of an SDE, learned from data of the high-fidelity system, are routinely used to increase the efficiency of simulation and prediction of properties. However, standard choices of loss function for learning the surrogate model fail to provide error guarantees in certain path-dependent observables, such as transition times. This paper introduces an error bound for path-space observables and employs it as a novel variational loss for the goal-oriented learning of the drift function of a SDE. We show the error bound holds for a broad class of observables, including mean first hitting times on unbounded time domains. We derive an analytical gradient of the goal-oriented loss by leveraging the formula for Fr\'echet derivatives of expected path functionals, which remains tractable for implementation in stochastic gradient descent schemes. We demonstrate that surrogate models of overdamped Langevin systems developed via goal-oriented learning achieve improved accuracy in predicting the statistics of a first hitting time observable and robustness to distributional shift in the data.

stat.ME↗

arXiv2026

Goodness-of-fit testing for nonlinear inverse problems with random observations

This work is concerned with nonparametric goodness-of-fit testing in the context of nonlinear inverse problems with random observations. Bayesian posterior distributions based upon a Gaussian process prior distribution are proven to contract at a certain rate uniformly over a set of true parameters. The corresponding posterior mean is shown to converge uniformly at the posterior contraction rate in the sense of satisfying a concentration inequality. Distinguishability for bounded alternatives separated from a composite null hypothesis at the posterior contraction rate is established using infimum plug-in tests based on the posterior mean and also on maximum a posteriori estimators. The results are applied to a class of inverse problems governed by ordinary differential equation initial value problems that is widely used in pharmacokinetics. For this class, uniform posterior contraction rates are proven and then used to establish distinguishability.

math.ST↗

arXiv2026

Posterior error bounds for prior-driven balancing in linear Gaussian inverse problems

In large-scale Bayesian inverse problems, it is often necessary to apply approximate forward models to reduce the cost of forward model evaluations, while controlling approximation quality. In the context of Bayesian inverse problems with linear forward models, Gaussian priors, and Gaussian noise, we use perturbation theory for inverses to bound the error in the approximate posterior mean and posterior covariance resulting from a linear approximate forward model. We then focus on the smoothing problem of inferring the initial condition of linear time-invariant dynamical systems, using finitely many partial state observations. For such problems, and for a specific model order reduction method based on balanced truncation, we show that the impulse response of a certain prior-driven system is closely related to the prior-preconditioned Hessian of the inverse problem. This reveals a novel connection between systems theory and inverse problems. We exploit this connection to prove the first a priori error bounds for system-theoretic model order reduction methods applied to smoothing problems. The bounds control the approximation error of the posterior mean and covariance in terms of the truncated Hankel singular values of the underlying system.

math.NA↗

arXiv2025

Upper and lower bounds for local Lipschitz stability of Bayesian posteriors

The work of Sprungk (Inverse Problems, 2020) established the local Lipschitz continuity of the misfit-to-posterior and prior-to-posterior maps with respect to the Kullback--Leibler divergence and the total variation, Hellinger, and 1-Wasserstein metrics, by proving certain upper bounds. The upper bounds were also used to show that if a posterior measure is more concentrated, then it can be more sensitive to perturbations in the misfit or prior. We prove upper bounds and lower bounds that emphasise the importance of the evidence. The lower bounds show that the sensitivity of posteriors to perturbations in the misfit or the prior not only can increase, but in general will increase as the posterior measure becomes more concentrated, i.e. as the evidence decreases to zero. Using the explicit dependence of our bounds on the evidence, we identify sufficient conditions for the misfit-to-posterior and prior-to-posterior maps to be locally bi-Lipschitz continuous.

math.ST↗

arXiv2025

Optimal low-rank approximations for linear Gaussian inverse problems on Hilbert spaces, Part I: posterior covariance approximation

For linear inverse problems with Gaussian priors and Gaussian observation noise, the posterior is Gaussian, with mean and covariance determined by the conditioning formula. Using the Feldman-Hajek theorem, we analyse the prior-to-posterior update and its low-rank approximation for infinite-dimensional Hilbert parameter spaces and finite-dimensional observations. We show that the posterior distribution differs from the prior on a finite-dimensional subspace, and construct low-rank approximations to the posterior covariance, while keeping the mean fixed. Since in infinite dimensions, not all low-rank covariance approximations yield approximate posterior distributions which are equivalent to the posterior and prior distribution, we characterise the low-rank covariance approximations which do yield this equivalence, and their respective inverses, or `precisions'. For such approximations, a family of measure approximation problems is solved by identifying the low-rank approximations which are optimal for various losses simultaneously. These loss functions include the family of R\'enyi divergences, the Amari $\alpha$-divergences for $\alpha\in(0,1)$, the Hellinger metric and the Kullback-Leibler divergence. Our results extend those of Spantini et al. (SIAM J. Sci. Comput. 2015) to Hilbertian parameter spaces, and provide theoretical underpinning for the construction of low-rank approximations of discretised versions of the infinite-dimensional inverse problem, by formulating discretization independent results.

math.ST↗

arXiv2025

Optimal low-rank posterior mean and distribution approximation in linear Gaussian inverse problems on Hilbert spaces

We construct optimal low-rank approximations for the Gaussian posterior distribution in linear Gaussian inverse problems with possibly infinite-dimensional separable Hilbert parameter spaces and finite-dimensional data spaces. We first consider approximate posteriors in which the means vary and the posterior covariance is kept fixed, for all possible realisations of the data simultaneously. We give necessary and sufficient conditions for these approximating posteriors to be equivalent to the exact posterior. For such approximations, we measure the data-averaged approximation error with the Kullback-Leibler, R\'enyi and Amari $\alpha$-divergences for $\alpha\in(0,1)$, and the Hellinger distance. With the loss in Kullback-Leibler and R\'enyi divergences, we find the optimal approximations and formulate an equivalent condition for their uniqueness, extending the work in finite dimensions of Spantini et al. (SIAM J. Sci. Comput. 2015). We then consider joint low-rank approximation of the mean and covariance. For the reverse Kullback-Leibler divergence, the optimal approximations of the mean and of the covariance yield an optimal joint approximation of the mean and covariance. We interpret one such joint approximation in terms of an optimal projector in parameter space, and show that this approximation amounts to solving a Bayesian inverse problem with projected forward model. Extensive numerical examples demonstrate some of our theoretical findings.

math.ST↗

arXiv2024

Optimal low-rank posterior covariance approximation in linear Gaussian inverse problems on Hilbert spaces

For linear inverse problems with Gaussian priors and Gaussian observation noise, the posterior is Gaussian, with mean and covariance determined by the conditioning formula. The covariance is the central object for uncertainty quantification, as it encodes the variability of the posterior distribution and thus the uncertainty in the posterior mean estimate. Using the Feldman-Hajek theorem, we analyse the prior-to-posterior update and its low-rank approximation for infinite-dimensional Hilbert parameter spaces and finite-dimensional observations. We show that the posterior distribution differs from the prior on a finite-dimensional subspace, and construct low-rank approximations to the posterior covariance, while keeping the mean fixed. Since in infinite dimensions, not all low-rank covariance approximations yield approximate posterior distributions which are equivalent to the posterior and prior distribution, we characterise the low-rank covariance approximations which do yield this equivalence, and their respective inverses, or `precisions'. For such approximations, a family of measure approximation problems is solved by identifying the low-rank approximations which are optimal for various losses simultaneously. These loss functions include the family of R\'enyi divergences, the Amari $\alpha$-divergences for $\alpha\in(0,1)$, the Hellinger metric and the Kullback-Leibler divergence. Our results extend those of Spantini et al. (SIAM J. Sci. Comput. 2015) to Hilbertian parameter spaces, and provide theoretical underpinning for the construction of low-rank approximations of discretised versions of the infinite-dimensional inverse problem, by formulating discretisation independent results.

math.ST↗

arXiv2024

Data subsampling for Poisson regression with pth-root-link

We develop and analyze data subsampling techniques for Poisson regression, the standard model for count data $y\in\mathbb{N}$. In particular, we consider the Poisson generalized linear model with ID- and square root-link functions. We consider the method of coresets, which are small weighted subsets that approximate the loss function of Poisson regression up to a factor of $1\pm\varepsilon$. We show $\Omega(n)$ lower bounds against coresets for Poisson regression that continue to hold against arbitrary data reduction techniques up to logarithmic factors. By introducing a novel complexity parameter and a domain shifting approach, we show that sublinear coresets with $1\pm\varepsilon$ approximation guarantee exist when the complexity parameter is small. In particular, the dependence on the number of input points can be reduced to polylogarithmic. We show that the dependence on other input parameters can also be bounded sublinearly, though not always logarithmically. In particular, we show that the square root-link admits an $O(\log(y_{\max}))$ dependence, where $y_{\max}$ denotes the largest count presented in the data, while the ID-link requires a $\Theta(\sqrt{y_{\max}/\log(y_{\max})})$ dependence. As an auxiliary result for proving the tightness of the bound with respect to $y_{\max}$ in the case of the ID-link, we show an improved bound on the principal branch of the Lambert $W_0$ function, which may be of independent interest. We further show the limitations of our analysis when $p$th degree root-link functions for $p\geq 3$ are considered, which indicate that other analytical or computational methods would be required if such a generalization is even possible.

cs.LG↗

arXiv2024

Generalised Rank-Constrained Approximations of Hilbert-Schmidt Operators on Separable Hilbert Spaces and Applications

In this work we solve, for given bounded operators $B,C$ and Hilbert-Schmidt operator $M$ acting on potentially infinite-dimensional separable Hilbert spaces, the reduced rank approximation problem, $\min\{\lVert M-BXC\rVert_{L_2}:\ \text{dim ran}\ X\leq r\}.$ This extends the result of Sondermann (Statistische Hefte, 1986) and Friedland and Torokhti (SIAM J. Matrix Analysis and Applications, 2007), which studies this problem in the case of matrices $M$, $B$, $C$, $X$, and the analysis involves the Moore-Penrose inverse. In classical approximation problems that can be solved by the singular value decomposition or Moore-Penrose inverse, the solution satisfies a minimal norm property. Friedland and Torokhti state such a minimal norm property of the solution. We show that this minimal norm property does not hold in general and give a modified minimality property that does hold. We show that the solution may be discontinuous in infinite-dimensional settings. We give conditions for continuity of the solutions and construct continuous approximations when such conditions are not met. Finally, we study problems from signal processing, reduced rank regression and linear operator learning under a rank constraint. Our theoretical results enable us to explicitly find solutions to these problems and to characterise their existence, uniqueness and minimality property.

math.FA↗

arXiv2024

Bayesian inference of covariate-parameter relationships for population modelling

We consider population modelling using parametrised ordinary differential equation initial value problems (ODE-IVPs). For each individual drawn randomly from the unknown population distribution, the corresponding parameters for the ODE-IVP cannot be measured directly, but a vector of covariates is given, and one component of the solution to the corresponding ODE-IVP is observed at a fixed finite time grid. The task is to identify a covariate-parameter relationship that maps covariate vectors to parameter vectors. Such settings and problems arise in pharmacokinetics, where the observations are blood drug concentrations, the covariates are clinically observable quantities, and the covariate-parameter relationship is used for personalised drug dosing. For linear homogeneous ODE-IVPs with vector fields defined by matrices that are diagonalisable over $\mathbb{R}$, and for fixed time and random covariate design, we use recent results of Nickl et al. for Bayesian nonlinear statistical inverse problems, to prove posterior contraction and Bernstein--von Mises results for the unknown covariate-parameter relationship. We analytically demonstrate our results on an example from the pharmacokinetics literature.

math.ST↗

arXiv2023

Choosing observation operators to mitigate model error in Bayesian inverse problems

In statistical inference, a discrepancy between the parameter-to-observable map that generates the data and the parameter-to-observable map that is used for inference can lead to misspecified likelihoods and thus to incorrect estimates. In many inverse problems, the parameter-to-observable map is the composition of a linear state-to-observable map called an `observation operator' and a possibly nonlinear parameter-to-state map called the `model'. We consider such Bayesian inverse problems where the discrepancy in the parameter-to-observable map is due to the use of an approximate model that differs from the best model, i.e. to nonzero `model error'. Multiple approaches have been proposed to address such discrepancies, each leading to a specific posterior. We show how to use local Lipschitz stability estimates of posteriors with respect to likelihood perturbations to bound the Kullback--Leibler divergence of the posterior of each approach with respect to the posterior associated to the best model. Our bounds lead to criteria for choosing observation operators that mitigate the effect of model error for Bayesian inverse problems of this type. We illustrate one such criterion on an advection-diffusion-reaction PDE inverse problem from the literature, and use this example to discuss the importance and challenges of model error-aware inference.

math.ST↗

arXiv2021

\Gamma-convergence of Onsager-Machlup functionals. Part I: With applications to maximum a posteriori estimation in Bayesian inverse problems

The Bayesian solution to a statistical inverse problem can be summarised by a mode of the posterior distribution, i.e. a MAP estimator. The MAP estimator essentially coincides with the (regularised) variational solution to the inverse problem, seen as minimisation of the Onsager-Machlup functional of the posterior measure. An open problem in the stability analysis of inverse problems is to establish a relationship between the convergence properties of solutions obtained by the variational approach and by the Bayesian approach. To address this problem, we propose a general convergence theory for modes that is based on the $\Gamma$-convergence of Onsager-Machlup functionals, and apply this theory to Bayesian inverse problems with Gaussian and edge-preserving Besov priors. Part II of this paper considers more general prior distributions.

math.ST↗

arXiv2021

\Gamma-convergence of Onsager-Machlup functionals. Part II: Infinite product measures on Banach spaces

We derive Onsager-Machlup functionals for countable product measures on weighted $\ell^p$ subspaces of the sequence space $\mathbb{R}^{\mathbb{N}}$. Each measure in the product is a shifted and scaled copy of a reference probability measure on $\mathbb{R}$ that admits a sufficiently regular Lebesgue density. We study the equicoercivity and $\Gamma$-convergence of sequences of Onsager-Machlup functionals associated to convergent sequences of measures within this class. We use these results to establish analogous results for probability measures on separable Banach or Hilbert spaces, including Gaussian, Cauchy, and Besov measures with summability parameter $1 \leq p \leq 2$. Together with Part I of this paper, this provides a basis for analysis of the convergence of maximum a posteriori estimators in Bayesian inverse problems and most likely paths in transition path theory.

math.ST↗

arXiv2021

Fréchet derivatives of expected functionals of solutions to stochastic differential equations

In the analysis of stochastic dynamical systems described by stochastic differential equations (SDEs), it is often of interest to analyse the sensitivity of the expected value of a functional of the solution of the SDE with respect to perturbations in the SDE parameters. In this paper, we consider path functionals that depend on the solution of the SDE up to a stopping time. We derive formulas for Fréchet derivatives of the expected values of these functionals with respect to bounded perturbations of the drift, using the Cameron-Martin-Girsanov theorem for the change of measure. Using these derivatives, we construct an example to show that the map that sends the change of drift to the corresponding relative entropy is not in general convex. We then analyse the existence and uniqueness of solutions to stochastic optimal control problems defined on possibly random time intervals, as well as gradient-based numerical methods for solving such problems.

math.PR↗

arXiv2021

Randomised one-step time integration methods for deterministic operator differential equations

Uncertainty quantification plays an important role in problems that involve inferring a parameter of an initial value problem from observations of the solution. Conrad et al.\ (\textit{Stat.\ Comput.}, 2017) proposed randomisation of deterministic time integration methods as a strategy for quantifying uncertainty due to the unknown time discretisation error. We consider this strategy for systems that are described by deterministic, possibly time-dependent operator differential equations defined on a Banach space or a Gelfand triple. Our main results are strong error bounds on the random trajectories measured in Orlicz norms, proven under a weaker assumption on the local truncation error of the underlying deterministic time integration method. Our analysis establishes the theoretical validity of randomised time integration for differential equations in infinite-dimensional settings.

math.NA↗

arXiv2020

A convergent discretisation method for transition path theory for diffusion processes

Transition path theory (TPT) for diffusion processes is a framework for analysing the transitions of multiscale ergodic diffusion processes between disjoint metastable subsets of state space. Most methods for applying TPT involve the construction of a Markov state model on a discretisation of state space that approximates the underlying diffusion process. However, the assumption of Markovianity is difficult to verify in practice, and there are to date no known error bounds or convergence results for these methods. We propose a Monte Carlo method for approximating the forward committor, probability current, and streamlines from TPT for diffusion processes. Our method uses only sample trajectory data and partitions of state space based on Voronoi tessellations. It does not require the construction of a Markovian approximating process. We rigorously prove error bounds for the approximate TPT objects and use these bounds to show convergence to their exact counterparts in the limit of arbitrarily fine discretisation. We illustrate some features of our method by application to a process that solves the Smoluchowski equation on a triple-well potential.

math.NA↗

arXiv2019

Error bounds for some approximate posterior measures in Bayesian inference

In certain applications involving the solution of a Bayesian inverse problem, it may not be possible or desirable to evaluate the full posterior, e.g. due to the high computational cost of doing so. This problem motivates the use of approximate posteriors that arise from approximating the data misfit or forward model. We review some error bounds for random and deterministic approximate posteriors that arise when the approximate data misfits and approximate forward models are random.

math.ST↗