arXiv subjects

Oliver J. Maclaren

Publications and source records attributed to Oliver J. Maclaren.

10 recordsLinked to original sources

arXiv2026

Invariant Image Reparameterisation: Bridging Symbolic and Numerical Methods for Identifiability Analysis, Model Reduction, and Prediction

Structural and practical parameter non-identifiability issues are common when mathematical models are used to interpret data. Such issues motivate model reparameterisation and reduction methods. Here, we consider Invariant Image Reparameterisation (IIR), which asks when symbolic reparameterisation conditions can be replaced by numerical derivative calculations at a single reference point. The central object is the invariant image: a reduced, basis-independent representation of the parameter combinations controlling observable model behaviour. We show that when a one-to-one componentwise transformation makes observable behaviour depend only on fixed linear combinations of the transformed parameters, a single numerical Jacobian determines the associated lower-dimensional reparameterisation space. This includes models depending on monomial combinations of the original parameters. We also give a first-order invariance condition that distinguishes minimal from non-minimal but exact reductions via the invariant part of the local null space. In structurally identifiable but practically weakly informed settings, the same calculations separate strongly and weakly informed parameter combinations. The invariant image admits multiple coordinate representations: the SVD gives a default orthonormal basis ordered by local identifiability, while sparse monomial bases are often more interpretable. Treating these coordinates as interest parameters in Profile-Wise Analysis gives likelihood-based uncertainty quantification and prediction. We demonstrate the method on parameterised normal models with Poisson-limit, extended Poisson-limit, and non-limit cases, and on the repressilator, a nonlinear differential equation model of gene regulation. A Julia implementation of IIR, with these and further examples, is available at https://github.com/omaclaren/reparam.

stat.AP↗

arXiv2026

Beyond Independence: on Jointly Normal Priors in Bayesian Inversion

We consider joint inversion for two or more unknown parameters from observational data in the Bayesian framework. Standard approaches often either treat the parameters as independent or impose structural similarity through regularisation terms that can be difficult to interpret statistically. We instead construct jointly Gaussian prior models with prescribed Gaussian marginals, so that correlation between the parameters can be incorporated without altering the marginal prior distributions. We propose a joint covariance construction that preserves the marginals, allows spatially varying cross-correlation, and supports uncertainty and inference in the correlation itself. The construction is valid for any strict contraction encoding the desired cross-correlation and is optimal in a canonical correlation sense under the principal square root factorisation. We demonstrate the method using prior sampling and several inference examples: a low-dimensional illustrative example and two higher-dimensional examples, including a PDE-constrained problem. The examples highlight both the potential pitfalls of ignoring or neglecting uncertainty in the correlation as well as reinforcing a key principle of the Bayesian paradigm: unknown quantities included in a model should be treated as random variables.

stat.ME↗

arXiv2025

Data Space Inversion for Efficient Predictions and Uncertainty Quantification for Geothermal Models

The ability to make accurate predictions with quantified uncertainty provides a crucial foundation for the successful management of a geothermal reservoir. Conventional approaches for making predictions using geothermal reservoir models involve estimating unknown model parameters using field data, then propagating the uncertainty in these estimates through to the predictive quantities of interest. However, the unknown parameters are not always of direct interest; instead, the predictions are of primary importance. Data space inversion (DSI) is an alternative methodology that allows for the efficient estimation of predictive quantities of interest, with quantified uncertainty, that avoids the need to estimate model parameters entirely. In this paper, we illustrate the applicability of DSI to geothermal reservoir modelling. We first review the processes of model calibration, prediction and uncertainty quantification from a Bayesian perspective, and introduce data space inversion as a simple, efficient technique for approximating the posterior predictive distribution. We then introduce a modification of the typical DSI algorithm that allows us to sample directly and efficiently from the DSI approximation to the posterior predictive distribution, and apply the algorithm to two model problems in geothermal reservoir modelling. We evaluate the accuracy and efficiency of our DSI algorithm relative to other common methods for uncertainty quantification and study how the number of reservoir model simulations affects the resulting approximation to the posterior predictive distribution. Our results demonstrate that data space inversion is a robust and efficient technique for making predictions with quantified uncertainty using geothermal reservoir models, providing a useful alternative to more conventional approaches.

stat.AP↗

arXiv2023

Implementing measurement error models with mechanistic mathematical models in a likelihood-based framework for estimation, identifiability analysis, and prediction in the life sciences

Throughout the life sciences we routinely seek to interpret measurements and observations using parameterised mechanistic mathematical models. A fundamental and often overlooked choice in this approach involves relating the solution of a mathematical model with noisy and incomplete measurement data. This is often achieved by assuming that the data are noisy measurements of the solution of a deterministic mathematical model, and that measurement errors are additive and normally distributed. While this assumption of additive Gaussian noise is extremely common and simple to implement and interpret, it is often unjustified and can lead to poor parameter estimates and non-physical predictions. One way to overcome this challenge is to implement a different measurement error model. In this review, we demonstrate how to implement a range of measurement error models in a likelihood-based framework for estimation, identifiability analysis, and prediction, called Profile-Wise Analysis. This frequentist approach to uncertainty quantification for mechanistic models leverages the profile likelihood for targeting parameters and understanding their influence on predictions. Case studies, motivated by simple caricature models routinely used in systems biology and mathematical biology literature, illustrate how the same ideas apply to different types of mathematical models. Open-source Julia code to reproduce results is available on GitHub.

q-bio.QM↗

arXiv2023

Generalised likelihood profiles for models with intractable likelihoods

Likelihood profiling is an efficient and powerful frequentist approach for parameter estimation, uncertainty quantification and practical identifiablity analysis. Unfortunately, these methods cannot be easily applied for stochastic models without a tractable likelihood function. Such models are typical in many fields of science, rendering these classical approaches impractical in these settings. To address this limitation, we develop a new approach to generalising the methods of likelihood profiling for situations when the likelihood cannot be evaluated but stochastic simulations of the assumed data generating process are possible. Our approach is based upon recasting developments from generalised Bayesian inference into a frequentist setting. We derive a method for constructing generalised likelihood profiles and calibrating these profiles to achieve desired frequentist coverage for a given coverage level. We demonstrate the performance of our method on realistic examples from the literature and highlight the capability of our approach for the purpose of practical identifability analysis for models with intractable likelihoods.

stat.ME↗

arXiv2022

Likelihood-based estimation and prediction for a measles outbreak in Samoa

Prediction of the progression of an infectious disease outbreak is important for planning and coordinating a response. Differential equations are often used to model an epidemic outbreak's behaviour but are challenging to parameterise. Furthermore, these models can suffer from misspecification, which biases predictions and parameter estimates. Stochastic models can help with misspecification but are even more expensive to simulate and perform inference with. Here, we develop an explicitly likelihood-based variation of the generalised profiling method as a tool for prediction and inference under model misspecification. Our approach allows us to carry out identifiability analysis and uncertainty quantification using profile likelihood-based methods without the need for marginalisation. We provide justification for this approach by introducing a new interpretation of the model approximation component as a stochastic constraint. This preserves the rationale for using profiling rather than integration to remove nuisance parameters while also providing a link back to stochastic models. We applied an initial version of this method during an outbreak of measles in Samoa in 2019-2020 and found that it achieved relatively fast, accurate predictions. Here we present the most recent version of our method and its application to this measles outbreak, along with additional validation.

q-bio.PE↗

arXiv2020

What can be estimated? Identifiability, estimability, causal inference and ill-posed inverse problems

We consider basic conceptual questions concerning the relationship between statistical estimation and causal inference. Firstly, we show how to translate causal inference problems into an abstract statistical formalism without requiring any structure beyond an arbitrarily-indexed family of probability models. The formalism is simple but can incorporate a variety of causal modelling frameworks, including 'structural causal models', but also models expressed in terms of, e.g., differential equations. We focus primarily on the structural/graphical causal modelling literature, however. Secondly, we consider the extent to which causal and statistical concerns can be cleanly separated, examining the fundamental question: 'What can be estimated from data?'. We call this the problem of estimability. We approach this by analysing a standard formal definition of 'can be estimated' commonly adopted in the causal inference literature -- identifiability -- in our abstract statistical formalism. We use elementary category theory to show that identifiability implies the existence of a Fisher-consistent estimator, but also show that this estimator may be discontinuous, and thus unstable, in general. This difficulty arises because the causal inference problem is, in general, an ill-posed inverse problem. Inverse problems have three conditions which must be satisfied to be considered well-posed: existence, uniqueness, and stability of solutions. Here identifiability corresponds to the question of uniqueness; in contrast, we take estimability to mean satisfaction of all three conditions, i.e. well-posedness. Lack of stability implies that naive translation of a causally identifiable quantity into an achievable statistical estimation target may prove impossible. Our article is primarily expository and aimed at unifying ideas from multiple fields, though we provide new constructions and proofs.

math.ST↗

arXiv2019

Incorporating Posterior-Informed Approximation Errors into a Hierarchical Framework to Facilitate Out-of-the-Box MCMC Sampling for Geothermal Inverse Problems and Uncertainty Quantification

We consider geothermal inverse problems and uncertainty quantification from a Bayesian perspective. Our main goal is to make standard, `out-of-the-box' Markov chain Monte Carlo (MCMC) sampling more feasible for complex simulation models by using suitable approximations. To do this, we first show how to pose both the inverse and prediction problems in a hierarchical Bayesian framework. We then show how to incorporate so-called posterior-informed model approximation error into this hierarchical framework, using a modified form of the Bayesian approximation error (BAE) approach. This enables the use of a `coarse', approximate model in place of a finer, more expensive model, while accounting for the additional uncertainty and potential bias that this can introduce. Our method requires only simple probability modelling, a relatively small number of fine model simulations, and only modifies the target posterior -- any standard MCMC sampling algorithm can be used to sample the new posterior. These corrections can also be used in methods that are not based on MCMC sampling. We show that our approach can achieve significant computational speed-ups on two geothermal test problems. We also demonstrate the dangers of naively using coarse, approximate models in place of finer models, without accounting for the induced approximation errors. The naive approach tends to give overly confident and biased posteriors while incorporating BAE into our hierarchical framework corrects for this while maintaining computational efficiency and ease-of-use.

stat.CO↗

arXiv2018

Is profile likelihood a true likelihood? An argument in favor

Profile likelihood is the key tool for dealing with nuisance parameters in likelihood theory. It is often asserted, however, that profile likelihood is not a 'true' likelihood. One implication is that likelihood theory lacks the generality of e.g. Bayesian inference, wherein marginalization is the universal tool for dealing with nuisance parameters. Here we argue that profile likelihood has as much claim to being a true likelihood as a marginal probability has to being a true probability distribution. The crucial point we argue is that a likelihood function is naturally interpreted as a maxitive possibility measure: given this, the associated theory of integration with respect to maxitive measures delivers profile likelihood as the direct analogue of marginal probability in additive measure theory. Thus, given a background likelihood function, we argue that profiling over the likelihood function is as natural (or as unnatural, as the case may be) as marginalizing over a background probability measure. The connections to Bayesian inference can also be further clarified with the introduction of a suitable logarithmic distance function, in which case the present theory can be naturally described as 'Tropical Bayes' in the sense of tropical algebra.

math.ST↗

arXiv2017

Randomized Truncated SVD Levenberg-Marquardt Approach to Geothermal Natural State and History Matching

The Levenberg-Marquardt (LM) method is commonly used for inverting models used to describe geothermal, groundwater, or oil and gas reservoirs. In previous studies LM parameter updates have been made tractable for highly parameterized inverse problems with large data sets by applying matrix factorization methods or iterative linear solvers to approximately solve the update equations. Some studies have shown that basing model updates on the truncated singular value decomposition (TSVD) of a dimensionless sensitivity matrix achieved using Lanczos iteration can speed up the inversion of reservoir models. Lanczos iterations only require the sensitivity matrix times a vector and its transpose times a vector, which are found efficiently using adjoint and direct simulations without the expense of forming a large sensitivity matrix. Nevertheless, Lanczos iteration has the drawback of being a serial process, requiring a separate adjoint solve and direct solve every Lanczos iteration. Randomized methods, developed for low-rank matrix approximation of large matrices, are more efficient alternatives to the standard Lanczos method. Here we develop LM variants which use randomized methods to find a TSVD of a dimensionless sensitivity matrix when updating parameters. The randomized approach offers improved efficiency by enabling simultaneous solution of all adjoint and direct problems for a parameter update.

math.OC↗