arXiv subjects

Samuel Livingstone

Publications and source records attributed to Samuel Livingstone.

At least 19 recordsLinked to original sources

arXiv2026

Pathwise skew-symmetric discretisation for SDEs with superlinear drift

The skew-symmetric discretisation has recently been proposed as a new robust simulation method for weakly approximating stochastic differential equations (SDEs) with non-globally Lipschitz drift. This work develops a pathwise version of the scheme by representing the noise increment as a skew-normal distribution and coupling it with the driving Brownian increments, thereby enabling its use in the multilevel Monte Carlo (MLMC) framework. Under suitable conditions, we establish strong convergence of order 1/2 in $L^2$. Subsequently, the associated MLMC estimator is shown to have computational complexity ${O} \bigl(\varepsilon^{-2} (\log (1/\varepsilon))^2 \bigr)$ to achieve a mean-squared error $\varepsilon^2$. We then analytically compare the proposed scheme with the tamed Euler scheme, another benchmark for robust discretisation. Under a strong inward-drift regime with the current state being far from the stable region, we show that the probability of moving in the wrong direction tends to vanish in the skew-symmetric scheme, whereas the tamed Euler scheme makes such moves with a non-trivial probability. Furthermore, in the MLMC setting, employing a one-dimensional stochastic Ginzburg-Landau model, we specify the range of step sizes for which the asymptotic variance of the coupled level difference obtained via the pathwise skew-symmetric scheme is lower than that obtained via the tamed Euler scheme. Numerical experiments on several model examples support the theoretical rate of strong convergence and demonstrate the stability and effectiveness of the resulting MLMC in the superlinear drift setting.

math.NA↗

arXiv2026

High-dimensional Adaptive MCMC with Reduced Computational Complexity

We propose an adaptive MCMC method that learns a linear preconditioner which is dense in its off-diagonal elements but sparse in its parametrisation. Due to this sparsity, we achieve a per-iteration computational complexity of $O(m^2d)$ for a user-determined parameter $m$, compared with the $O(d^2)$ complexity of existing adaptive strategies that can capture correlation information from the target. Diagonal preconditioning has an $O(d)$ per-iteration complexity, but is known to fail in the case that the target distribution is highly correlated, see \citet[Section 3.5]{hird2025a}. Our preconditioner is constructed using eigeninformation from the target covariance which we infer using online principal components analysis on the MCMC chain. It is composed of a diagonal matrix and a product of carefully chosen reflection matrices. On various numerical tests we show that it outperforms diagonal preconditioning in terms of absolute performance, and that it outperforms traditional dense preconditioning and multiple diagonal plus low-rank alternatives in terms of time-normalised performance.

stat.CO↗

arXiv2026

A note on diffusive/random-walk behaviour in Metropolis--Hastings algorithms

We prove a general result that if a Metropolis--Hastings algorithm has a proposal that is not geometrically ergodic and the acceptance rate approaches unity at a suitable rate as the state variable becomes large, then the Metropolised chain will also not be geometrically ergodic. Our conditions seem stronger than might be expected, but are shown to be necessary through a counterexample. We then turn our attention to the random walk and guided walk Metropolis algorithms. We show that if the target distribution has polynomial tails the latter converges at twice the polynomial rate of the former, but that if instead the target distribution has strictly convex potential then the random walk Metropolis behaves as a $1/2$-lazy version of the guided walk Metropolis when the state variable is large, and therefore moves at a similar (ballistic) speed.

stat.CO↗

arXiv2026

On randomized step sizes in Metropolis-Hastings algorithms

The performance of Metropolis-Hastings algorithms is highly sensitive to the choice of step size, and miss-specification can lead to severe loss of efficiency. We study algorithms with randomized step sizes, considering both auxiliary-variable and marginalized constructions. We show that algorithms with a randomized step size inherit weak Poincar\'e inequalities/spectral gaps from their fixed-step-size counterparts under minimal conditions, and that the marginalized kernel should always be preferred in terms of asymptotic variance to the auxiliary-variable choice if it is implementable. In addition we show that both types of randomization make an algorithm robust to tuning, meaning that spectral gaps decay polynomially as the step size is increasingly poorly chosen. We further show that step-size randomization often preserves high-dimensional scaling limits and algorithmic complexity, while increasing the optimal acceptance rate for Langevin and Hamiltonian samplers when an Exponential or Uniform distribution is chosen to randomize the step size. Theoretical results are complemented with a numerical study on challenging benchmarks such as Poisson regression, Neal's funnel and the Rosenbrock (banana) distribution.

stat.CO↗

arXiv2025

Diffusion piecewise exponential models for survival extrapolation using Piecewise Deterministic Monte Carlo

The piecewise exponential model is a flexible non-parametric approach for time-to-event data, but extrapolation beyond final observation times typically relies on random walk priors and deterministic knot locations, resulting in unrealistic long-term hazards. We introduce the diffusion piecewise exponential model, a prior framework consisting of a discretised diffusion for the hazard, that can encode a wide variety of information about the long-term behaviour of the hazard, time changed by a Poisson process prior for knot locations. This allows the behaviour of the hazard in the observation period to be combined with prior information to inform extrapolations. Efficient posterior sampling is achieved using Piecewise Deterministic Markov Processes, whereby we extend existing approaches using sticky dynamics from sampling spike-and-slab distributions to more general transdimensional posteriors. We focus on applications in Health Technology Assessment, where the need to compute mean survival requires hazard functions to be extrapolated beyond the observation period, showcasing performance on datasets for Colon cancer and Leukaemia patients.

stat.ME↗

arXiv2025

Foundations of locally-balanced Markov processes

We formally introduce and study locally-balanced Markov jump processes (LBMJPs) defined on a general state space. These continuous-time stochastic processes with a user-specified limiting distribution are designed for sampling in settings involving discrete parameters and/or non-smooth distributions, addressing limitations of other processes such as the overdamped Langevin diffusion. The paper establishes the well-posedness, non-explosivity, and ergodicity of LBMJPs under mild conditions. We further explore regularity properties such as the Feller property and characterise the weak generator of the process. We then derive conditions for exponential ergodicity via spectral gaps and establish comparison theorems for different balancing functions. In particular we show an equivalence between the spectral gaps of Metropolis--Hastings algorithms and LBMJPs with bounded balancing function, but show that LBMJPs can exhibit uniform ergodicity on unbounded state spaces when the balancing function is unbounded, even when the limiting distribution is not sub-Gaussian. We also establish a diffusion limit for an LBMJP in the small jump limit, and discuss applications to Monte Carlo sampling and non-reversible extensions of the processes.

math.PR↗

arXiv2024

Averaging polyhazard models using Piecewise deterministic Monte Carlo with applications to data with long-term survivors

Polyhazard models are a class of flexible parametric models for modelling survival over extended time horizons. Their additive hazard structure allows for flexible, non-proportional hazards whose characteristics can change over time while retaining a parametric form, which allows for survival to be extrapolated beyond the observation period of a study. Significant user input is required, however, in selecting the number of latent hazards to model, their distributions and the choice of which variables to associate with each hazard. The resulting set of models is too large to explore manually, limiting their practical usefulness. Motivated by applications to stroke survivor and kidney transplant patient survival times we extend the standard polyhazard model through a prior structure allowing for joint inference of parameters and structural quantities, and develop a sampling scheme that utilises state-of-the-art Piecewise Deterministic Markov Processes to sample from the resulting transdimensional posterior with minimal user tuning.

stat.ME↗

arXiv2024

Skew-symmetric schemes for stochastic differential equations with non-Lipschitz drift: an unadjusted Barker algorithm

We propose a new simple and explicit numerical scheme for time-homogeneous stochastic differential equations. The scheme is based on sampling increments at each time step from a skew-symmetric probability distribution, with the level of skewness determined by the drift and volatility of the underlying process. We show that as the step-size decreases the scheme converges weakly to the diffusion of interest. We then consider the problem of simulating from the limiting distribution of an ergodic diffusion process using the numerical scheme with a fixed step-size. We establish conditions under which the numerical scheme converges to equilibrium at a geometric rate, and quantify the bias between the equilibrium distributions of the scheme and of the true diffusion process. Notably, our results do not require a global Lipschitz assumption on the drift, in contrast to those required for the Euler--Maruyama scheme for long-time simulation at fixed step-sizes. Our weak convergence result relies on an extension of the theory of Milstein \& Tretyakov to stochastic differential equations with non-Lipschitz drift, which could also be of independent interest. We support our theoretical results with numerical simulations.

math.PR↗

arXiv2023

Quantifying the effectiveness of linear preconditioning in Markov chain Monte Carlo

We study linear preconditioning in Markov chain Monte Carlo. We consider the class of well-conditioned distributions, for which several mixing time bounds depend on the condition number $\kappa$. First we show that well-conditioned distributions exist for which $\kappa$ can be arbitrarily large and yet no linear preconditioner can reduce it. We then impose two sets of extra assumptions under which a linear preconditioner can significantly reduce $\kappa$. For the random walk Metropolis we further provide upper and lower bounds on the spectral gap with tight $1/\kappa$ dependence. This allows us to give conditions under which linear preconditioning can provably increase the gap. We then study popular preconditioners such as the covariance, its diagonal approximation, the hessian at the mode, and the QR decomposition. We show conditions under which each of these reduce $\kappa$ to near its minimum. We also show that the diagonal approach can in fact \textit{increase} the condition number. This is of interest as diagonal preconditioning is the default choice in well-known software packages. We conclude with a numerical study comparing preconditioners in different models, and showing how proper preconditioning can greatly reduce compute time in Hamiltonian Monte Carlo.

stat.CO↗

arXiv2023

Structure Learning with Adaptive Random Neighborhood Informed MCMC

In this paper, we introduce a novel MCMC sampler, PARNI-DAG, for a fully-Bayesian approach to the problem of structure learning under observational data. Under the assumption of causal sufficiency, the algorithm allows for approximate sampling directly from the posterior distribution on Directed Acyclic Graphs (DAGs). PARNI-DAG performs efficient sampling of DAGs via locally informed, adaptive random neighborhood proposal that results in better mixing properties. In addition, to ensure better scalability with the number of nodes, we couple PARNI-DAG with a pre-tuning procedure of the sampler's parameters that exploits a skeleton graph derived through some constraint-based or scoring-based algorithms. Thanks to these novel features, PARNI-DAG quickly converges to high-probability regions and is less likely to get stuck in local modes in the presence of high correlation between nodes in high-dimensional settings. After introducing the technical novelties in PARNI-DAG, we empirically demonstrate its mixing efficiency and accuracy in learning DAG structures on a variety of experiments.

cs.LG↗

arXiv2023

Adaptive MCMC for Bayesian variable selection in generalised linear models and survival models

Developing an efficient computational scheme for high-dimensional Bayesian variable selection in generalised linear models and survival models has always been a challenging problem due to the absence of closed-form solutions for the marginal likelihood. The RJMCMC approach can be employed to samples model and coefficients jointly, but effective design of the transdimensional jumps of RJMCMC can be challenge, making it hard to implement. Alternatively, the marginal likelihood can be derived using data-augmentation scheme e.g. Polya-gamma data argumentation for logistic regression) or through other estimation methods. However, suitable data-augmentation schemes are not available for every generalised linear and survival models, and using estimations such as Laplace approximation or correlated pseudo-marginal to derive marginal likelihood within a locally informed proposal can be computationally expensive in the "large n, large p" settings. In this paper, three main contributions are presented. Firstly, we present an extended Point-wise implementation of Adaptive Random Neighbourhood Informed proposal (PARNI) to efficiently sample models directly from the marginal posterior distribution in both generalised linear models and survival models. Secondly, in the light of the approximate Laplace approximation, we also describe an efficient and accurate estimation method for the marginal likelihood which involves adaptive parameters. Additionally, we describe a new method to adapt the algorithmic tuning parameters of the PARNI proposal by replacing the Rao-Blackwellised estimates with the combination of a warm-start estimate and an ergodic average. We present numerous numerical results from simulated data and 8 high-dimensional gene fine mapping data-sets to showcase the efficiency of the novel PARNI proposal compared to the baseline add-delete-swap proposal.

stat.ME↗

arXiv2022

Sampling algorithms in statistical physics: a guide for statistics and machine learning

We discuss several algorithms for sampling from unnormalized probability distributions in statistical physics, but using the language of statistics and machine learning. We provide a self-contained introduction to some key ideas and concepts of the field, before discussing three well-known problems: phase transitions in the Ising model, the melting transition on a two-dimensional plane and simulation of an all-atom model for liquid water. We review the classical Metropolis, Glauber and molecular dynamics sampling algorithms before discussing several more recent approaches, including cluster algorithms, novel variations of hybrid Monte Carlo and Langevin dynamics and piece-wise deterministic processes such as event chain Monte Carlo. We highlight cross-over with statistics and machine learning throughout and present some results on event chain Monte Carlo and sampling from the Ising model using tools from the statistics literature. We provide a simulation study on the Ising and XY models, with reproducible code freely available online, and following this we discuss several open areas for interaction between the disciplines that have not yet been explored and suggest avenues for doing so.

stat.CO↗

arXiv2022

A Bayesian hierarchical model for improving exercise rehabilitation in mechanically ventilated ICU patients

Patients who are mechanically ventilated in the intensive care unit (ICU) participate in exercise as a component of their rehabilitation to ameliorate the long-term impact of critical illness on their physical function. The effective implementation of these programmes is hindered, however, by the lack of a scientific method for quantifying an individual patient's exercise intensity level in real time, which results in a broad one-size-fits-all approach to rehabilitation and sub-optimal patient outcomes. In this work we have developed a Bayesian hierarchical model with temporally correlated latent Gaussian processes to predict $\dot VO_2$, a physiological measure of exercise intensity, using readily available physiological data. Inference was performed using Integrated Nested Laplace Approximation. For practical use by clinicians $\dot VO_2$ was classified into exercise intensity categories. Internal validation using leave-one-patient-out cross-validation was conducted based on these classifications, and the role of probabilistic statements describing the classification uncertainty was investigated.

stat.AP↗

arXiv2022

Optimal design of the Barker proposal and other locally-balanced Metropolis-Hastings algorithms

We study the class of first-order locally-balanced Metropolis--Hastings algorithms introduced in Livingstone & Zanella (2021). To choose a specific algorithm within the class the user must select a balancing function $g:\mathbb{R} \to \mathbb{R}$ satisfying $g(t) = tg(1/t)$, and a noise distribution for the proposal increment. Popular choices within the class are the Metropolis-adjusted Langevin algorithm and the recently introduced Barker proposal. We first establish a universal limiting optimal acceptance rate of 57% and scaling of $n^{-1/3}$ as the dimension $n$ tends to infinity among all members of the class under mild smoothness assumptions on $g$ and when the target distribution for the algorithm is of the product form. In particular we obtain an explicit expression for the asymptotic efficiency of an arbitrary algorithm in the class, as measured by expected squared jumping distance. We then consider how to optimise this expression under various constraints. We derive an optimal choice of noise distribution for the Barker proposal, optimal choice of balancing function under a Gaussian noise distribution, and optimal choice of first-order locally-balanced algorithm among the entire class, which turns out to depend on the specific target distribution. Numerical simulations confirm our theoretical findings and in particular show that a bi-modal choice of noise distribution in the Barker proposal gives rise to a practical algorithm that is consistently more efficient than the original Gaussian version.

stat.CO↗

arXiv2021

Adaptive random neighbourhood informed Markov chain Monte Carlo for high-dimensional Bayesian variable Selection

We introduce a framework for efficient Markov Chain Monte Carlo (MCMC) algorithms targeting discrete-valued high-dimensional distributions, such as posterior distributions in Bayesian variable selection (BVS) problems. We show that many recently introduced algorithms, such as the locally informed sampler and the Adaptively Scaled Individual adaptation sampler (ASI), can be viewed as particular cases within the framework. We then describe a novel algorithm, the Adaptive Random Neighbourhood Informed sampler (ARNI), by combining ideas from both of these existing approaches. We show using several examples of both real and simulated datasets that a computationally efficient point-wise implementation (PARNI) leads to relatively more reliable inferences on a range of variable selection problems, particularly in the very large $p$ setting.

stat.CO↗

arXiv2020

A fresh take on 'Barker dynamics' for MCMC

We study a recently introduced gradient-based Markov chain Monte Carlo method based on 'Barker dynamics'. We provide a full derivation of the method from first principles, placing it within a wider class of continuous-time Markov jump processes. We then evaluate the Barker approach numerically on a challenging ill-conditioned logistic regression example with imbalanced data, showing in particular that the algorithm is remarkably robust to irregularity (in this case a high degree of skew) in the target distribution.

stat.CO↗

arXiv2020

The Barker proposal: combining robustness and efficiency in gradient-based MCMC

There is a tension between robustness and efficiency when designing Markov chain Monte Carlo (MCMC) sampling algorithms. Here we focus on robustness with respect to tuning parameters, showing that more sophisticated algorithms tend to be more sensitive to the choice of step-size parameter and less robust to heterogeneity of the distribution of interest. We characterise this phenomenon by studying the behaviour of spectral gaps as an increasingly poor step-size is chosen for the algorithm. Motivated by these considerations, we propose a novel and simple gradient-based MCMC algorithm, inspired by the classical Barker accept-reject rule, with improved robustness properties. Extensive theoretical results, dealing with robustness to tuning, geometric ergodicity and scaling with dimension, suggest that the novel scheme combines the robustness of simple schemes with the efficiency of gradient-based ones. We show numerically that this type of robustness is particularly beneficial in the context of adaptive MCMC, giving examples where our proposed scheme significantly outperforms state-of-the-art alternatives.

stat.CO↗

arXiv2019

Peskun-Tierney ordering for Markov chain and process Monte Carlo: beyond the reversible scenario

Historically time-reversibility of the transitions or processes underpinning Markov chain Monte Carlo methods (MCMC) has played a key rôle in their development, while the self-adjointness of associated operators together with the use of classical functional analysis techniques on Hilbert spaces have led to powerful and practically successful tools to characterize and compare their performance. Similar results for algorithms relying on nonreversible Markov processes are scarce. We show that for a type of nonreversible Monte Carlo Markov chains and processes, of current or renewed interest in the Physics and Statistical literatures, it is possible to develop comparison results which closely mirror those available in the reversible scenario. We show that these results shed light on earlier literature, proving some conjectures and strengthening some earlier results.

math.PR↗