arXiv subjects

Simon Cotter

Publications and source records attributed to Simon Cotter.

8 recordsLinked to original sources

arXiv2025

Sampling as Bandits: Evaluation-Efficient Design for Black-Box Densities

We propose bandit importance sampling (BIS), a powerful importance sampling framework tailored for settings in which evaluating the target density is computationally expensive. BIS facilitates accurate sampling while minimizing the required number of target-density evaluations. In contrast to adaptive importance sampling, which optimizes a proposal distribution, BIS directly optimizes the set of samples through a sequential selection process driven by multi-armed bandits. BIS serves as a general framework that accommodates user-defined bandit strategies. Theoretically, the weak convergence of the weighted samples, and thus the consistency of the Monte Carlo estimator, is established regardless of the specific strategy employed. In this paper, we present a practical strategy that leverages Gaussian process surrogates to guide sample selection, adapting the principles of Bayesian optimization for sampling. Comprehensive numerical studies demonstrate the superior performance of BIS across multimodal, heavy-tailed distributions, and real-world Bayesian inference tasks involving Markov random fields.

stat.ME↗

arXiv2024

Exploring natural variation in tendon constitutive parameters via Bayesian data selection and mixed effects models

Combining microstructural mechanical models with experimental data enhances our understanding of the mechanics of soft tissue, such as tendons. In previous work, a Bayesian framework was used to infer constitutive parameters from uniaxial stress-strain experiments on horse tendons, specifically the superficial digital flexor tendon (SDFT) and common digital extensor tendon (CDET), on a per-experiment basis. Here, we extend this analysis to investigate the natural variation of these parameters across a population of horses. Using a Bayesian mixed effects model, we infer population distributions of these parameters. Given that the chosen hyperelastic model does not account for tendon damage, careful data selection is necessary. Avoiding ad hoc methods, we introduce a hierarchical Bayesian data selection method. This two-stage approach selects data per experiment, and integrates data weightings into the Bayesian mixed effects model. Our results indicate that the CDET is stiffer than the SDFT, likely due to a higher collagen volume fraction. The modes of the parameter distributions yield estimates of the product of the collagen volume fraction and Young's modulus as 811.5 MPa for the SDFT and 1430.2 MPa for the CDET. This suggests that positional tendons have stiffer collagen fibrils and/or higher collagen volume density than energy-storing tendons.

stat.AP↗

arXiv2022

Efficient forecasting and uncertainty quantification for large scale account level Monte Carlo models of debt recovery

We consider the problem of forecasting debt recovery from large portfolios of non-performing unsecured consumer loans under management. The state of the art in industry is to use stochastic processes to approximately model payment behaviour of individual customers based on several covariates, including credit scores and payment history. Monte Carlo simulation of these stochastic processes can enable forecasting of the possible returns from portfolios of defaulted debt, and the quantification of uncertainty. Despite the fact that the individual-level models are relatively simple, it is challenging to carry out simulations at the portfolio level because of the very large number of accounts. The accounts are also heterogeneous, with a broad range of values for the collection variances. We aim to solve two main problems: efficient allocation of computational resources in the simulations to estimate the likely collections as precisely as possible, and quantification of the uncertainty in the forecasts. We show that under certain conditions, robust estimators of population-level variance can be constructed by summing over coarse unbiased estimators of the variance of individual accounts. The proposed methods are demonstrated through application to a model which shares key features with those that are used in practice.

stat.CO↗

arXiv2020

NuZZ: numerical Zig-Zag sampling for general models

Markov chain Monte Carlo (MCMC) is a key algorithm in computational statistics, and as datasets grow larger and models grow more complex, many popular MCMC algorithms become too computationally expensive to be practical. Recent progress has been made on this problem through development of MCMC algorithms based on Piecewise Deterministic Markov Processes (PDMPs), irreversible processes that can be engineered to converge at a rate which is independent of the size of data. While there has understandably been a surge of theoretical studies following these results, PDMPs have so far only been implemented for models where certain gradients can be bounded, which is not possible in many statistical contexts. Focusing on the Zig-Zag process, we present the Numerical Zig-Zag (NuZZ) algorithm, which is applicable to general statistical models without the need for bounds on the gradient of the log posterior. This allows us to perform numerical experiments on: (i) how the Zig-Zag dynamics behaves on some test problems with common challenging features; and (ii) how the error between the target and sampled distributions evolves as a function of computational effort for different MCMC algorithms including NuZZ. Moreover, due to the specifics of the NuZZ algorithms, we are able to give an explicit bound on the Wasserstein distance between the exact posterior and its numerically perturbed counterpart in terms of the user-specified numerical tolerances of NuZZ.

stat.ME↗

arXiv2018

Surrogate Accelerated Bayesian Inversion for the Determination of the Thermal Diffusivity of a Material

Determination of the thermal properties of a material is an important task in many scientific and engineering applications. How a material behaves when subjected to high or fluctuating temperatures can be critical to the safety and longevity of a system's essential components. The laser flash experiment is a well-established technique for indirectly measuring the thermal diffusivity, and hence the thermal conductivity, of a material. In previous works, optimization schemes have been used to find estimates of the thermal conductivity and other quantities of interest which best fit a given model to experimental data. Adopting a Bayesian approach allows for prior beliefs about uncertain model inputs to be conditioned on experimental data to determine a posterior distribution, but probing this distribution using sampling techniques such as Markov chain Monte Carlo methods can be incredibly computationally intensive. This difficulty is especially true for forward models consisting of time-dependent partial differential equations. We pose the problem of determining the thermal conductivity of a material via the laser flash experiment as a Bayesian inverse problem in which the laser intensity is also treated as uncertain. We introduce a parametric surrogate model that takes the form of a stochastic Galerkin finite element approximation, also known as a generalized polynomial chaos expansion, and show how it can be used to sample efficiently from the approximate posterior distribution. This approach gives access not only to the sought-after estimate of the thermal conductivity but also important information about its relationship to the laser intensity, and information for uncertainty quantification. We also investigate the effects of the spatial profile of the laser on the estimated posterior distribution for the thermal conductivity.

math.NA↗

arXiv2015

Ensemble Transport Adaptive Importance Sampling

Markov chain Monte Carlo methods are a powerful and commonly used family of numerical methods for sampling from complex probability distributions. As applications of these methods increase in size and complexity, the need for efficient methods increases. In this paper, we present a particle ensemble algorithm. At each iteration, an importance sampling proposal distribution is formed using an ensemble of particles. A stratified sample is taken from this distribution and weighted under the posterior, a state-of-the-art ensemble transport resampling method is then used to create an evenly weighted sample ready for the next iteration. We demonstrate that this ensemble transport adaptive importance sampling (ETAIS) method outperforms MCMC methods with equivalent proposal distributions for low dimensional problems, and in fact shows better than linear improvements in convergence rates with respect to the number of ensemble members. We also introduce a new resampling strategy, multinomial transformation (MT), which while not as accurate as the ensemble transport resampler, is substantially less costly for large ensemble sizes, and can then be used in conjunction with ETAIS for complex problems. We also focus on how algorithmic parameters regarding the mixture proposal can be quickly tuned to optimise performance. In particular, we demonstrate this methodology's superior sampling for multimodal problems, such as those arising from inference for mixture models, and for problems with expensive likelihoods requiring the solution of a differential equation, for which speed-ups of orders of magnitude are demonstrated. Likelihood evaluations of the ensemble could be computed in a distributed manner, suggesting that this methodology is a good candidate for parallel Bayesian computations.

math.NA↗

arXiv2015

Constrained Approximation of Effective Generators for Multiscale Stochastic Reaction Networks and Application to Conditioned Path Sampling

Efficient analysis and simulation of multiscale stochastic systems of chemical kinetics is an ongoing area for research, and is the source of many theoretical and computational challenges. In this paper, we present a significant improvement to the constrained approach, which is a method for computing effective dynamics of slowly changing quantities in these systems, but which does not rely on the quasi-steady-state assumption (QSSA). The QSSA can cause errors in the estimation of effective dynamics for systems where the difference in timescales between the "fast" and "slow" variables is not so pronounced. This new application of the constrained approach allows us to compute the effective generator of the slow variables, without the need for expensive stochastic simulations. This is achieved by finding the null space of the generator of the constrained system. For complex systems where this is not possible, or where the constrained subsystem is itself multiscale, the constrained approach can then be applied iteratively. This results in breaking the problem down into finding the solutions to many small eigenvalue problems, which can be efficiently solved using standard methods. Since this methodology does not rely on the quasi steady-state assumption, the effective dynamics that are approximated are highly accurate, and in the case of systems with only monomolecular reactions, are exact. We will demonstrate this with some numerics, and also use the effective generators to sample paths of the slow variables which are conditioned on their endpoints, a task which would be computationally intractable for the generator of the full system.

math.NA↗

arXiv2014

Error Analysis of Diffusion Approximation Methods for Multiscale Systems in Reaction Kinetics

Several different methods exist for efficient approximation of paths in multiscale stochastic chemical systems. Another approach is to use bursts of stochastic simulation to estimate the parameters of a stochastic differential equation approximation of the paths. In this paper, multiscale methods for approximating paths are used to formulate different strategies for estimating the dynamics by diffusion processes. We then analyse how efficient and accurate these methods are in a range of different scenarios, and compare their respective advantages and disadvantages to other methods proposed to analyse multiscale chemical networks.

math.NA↗