arXiv subjects

Nick Whiteley

Publications and source records attributed to Nick Whiteley.

At least 37 records · Page 2Linked to original sources

arXiv2021

Inference in Stochastic Epidemic Models via Multinomial Approximations

We introduce a new method for inference in stochastic epidemic models which uses recursive multinomial approximations to integrate over unobserved variables and thus circumvent likelihood intractability. The method is applicable to a class of discrete-time, finite-population compartmental models with partial, randomly under-reported or missing count observations. In contrast to state-of-the-art alternatives such as Approximate Bayesian Computation techniques, no forward simulation of the model is required and there are no tuning parameters. Evaluating the approximate marginal likelihood of model parameters is achieved through a computationally simple filtering recursion. The accuracy of the approximation is demonstrated through analysis of real and simulated data using a model of the 1995 Ebola outbreak in the Democratic Republic of Congo. We show how the method can be embedded within a Sequential Monte Carlo approach to estimating the time-varying reproduction number of COVID-19 in Wuhan, China, recently published by Kucharski et al. 2020.

stat.ME↗

arXiv2021

Dimension-free Wasserstein contraction of nonlinear filters

For a class of partially observed diffusions, conditions are given for the map from the initial condition of the signal to filtering distribution to be contractive with respect to Wasserstein distances, with rate which does not necessarily depend on the dimension of the state-space. The main assumptions are that the signal has affine drift and constant diffusion coefficient and that the likelihood functions are log-concave. Ergodic and nonergodic signals are handled in a single framework. Examples include linear-Gaussian, stochastic volatility, neural spike-train and dynamic generalized linear models. For these examples filter stability can be established without any assumptions on the observations.

math.ST↗

arXiv2020

Global consensus Monte Carlo

To conduct Bayesian inference with large data sets, it is often convenient or necessary to distribute the data across multiple machines. We consider a likelihood function expressed as a product of terms, each associated with a subset of the data. Inspired by global variable consensus optimisation, we introduce an instrumental hierarchical model associating auxiliary statistical parameters with each term, which are conditionally independent given the top-level parameters. One of these top-level parameters controls the unconditional strength of association between the auxiliary parameters. This model leads to a distributed MCMC algorithm on an extended state space yielding approximations of posterior expectations. A trade-off between computational tractability and fidelity to the original model can be controlled by changing the association strength in the instrumental model. We further propose the use of a SMC sampler with a sequence of association strengths, allowing both the automatic determination of appropriate strengths and for a bias correction technique to be applied. In contrast to similar distributed Monte Carlo algorithms, this approach requires few distributional assumptions. The performance of the algorithms is illustrated with a number of simulated examples.

stat.CO↗

arXiv2020

Negative association, ordering and convergence of resampling methods

We study convergence and convergence rates for resampling schemes. Our first main result is a general consistency theorem based on the notion of negative association, which is applied to establish the almost-sure weak convergence of measures output from Kitagawa's (1996) stratified resampling method. Carpenter et al's (1999) systematic resampling method is similar in structure but can fail to converge depending on the order of the input samples. We introduce a new resampling algorithm based on a stochastic rounding technique of Srinivasan (2001), which shares some attractive properties of systematic resampling, but which exhibits negative association and therefore converges irrespective of the order of the input samples. We confirm a conjecture made by Kitagawa (1996) that ordering input samples by their states in $\mathbb{R}$ yields a faster rate of convergence; we establish that when particles are ordered using the Hilbert curve in $\mathbb{R}^d$, the variance of the resampling error is ${\scriptscriptstyle\mathcal{O}}(N^{-(1+1/d)})$ under mild conditions, where $N$ is the number of particles. We use these results to establish asymptotic properties of particle algorithms based on resampling schemes that differ from multinomial resampling.

stat.CO↗

arXiv2019

Dynamic time series clustering via volatility change-points

This note outlines a method for clustering time series based on a statistical model in which volatility shifts at unobserved change-points. The model accommodates some classical stylized features of returns and its relation to GARCH is discussed. Clustering is performed using a probability metric evaluated between posterior distributions of the most recent change-point associated with each series. This implies series are grouped together at a given time if there is evidence the most recent shifts in their respective volatilities were coincident or closely timed. The clustering method is dynamic, in that groupings may be updated in an online manner as data arrive. Numerical results are given analyzing daily returns of constituents of the S&P 500.

stat.ME↗

arXiv2018

Parallelising Particle Filters with Butterfly Interactions

Bootstrap particle filter (BPF) is the corner stone of many popular algorithms used for solving inference problems involving time series that are observed through noisy measurements in a non-linear and non-Gaussian context. The long term stability of BPF arises from particle interactions which in the context of modern parallel computing systems typically means that particle information needs to be communicated between processing elements, which makes parallel implementation of BPF nontrivial. In this paper we show that it is possible to constrain the interactions in a way which, under some assumptions, enables the reduction of the cost of communicating the particle information while still preserving the consistency and the long term stability of the BPF. Numerical experiments demonstrate that although the imposed constraints introduce additional error, the proposed method shows potential to be the method of choice in certain settings.

stat.CO↗

arXiv2018

Sampling normalizing constants in high dimensions using inhomogeneous diffusions

Motivated by the task of computing normalizing constants and importance sampling in high dimensions, we study the dimension dependence of fluctuations for additive functionals of time-inhomogeneous Langevin-type diffusions on $\mathbb{R}^{d}$. The main results are nonasymptotic variance and bias bounds, and a central limit theorem in the $d\to\infty$ regime. We demonstrate that a temporal discretization inherits the fluctuation properties of the underlying diffusion, which are controlled at a computational cost growing at most polynomially with $d$. The key steps include establishing Poincaré inequalities for time-marginal distributions of the diffusion and nonasymptotic bounds on deviation from Gaussianity in a martingale central limit theorem.

math.ST↗

arXiv2016

Calculating principal eigen-functions of non-negative integral kernels: particle approximations and applications

Often in applications such as rare events estimation or optimal control it is required that one calculates the principal eigen-function and eigen-value of a non-negative integral kernel. Except in the finite-dimensional case, usually neither the principal eigen-function nor the eigen-value can be computed exactly. In this paper, we develop numerical approximations for these quantities. We show how a generic interacting particle algorithm can be used to deliver numerical approximations of the eigen-quantities and the associated so-called "twisted" Markov kernel as well as how these approximations are relevant to the aforementioned applications. In addition, we study a collection of random integral operators underlying the algorithm, address some of their mean and path-wise properties, and obtain $L_{r}$ error estimates. Finally, numerical examples are provided in the context of importance sampling for computing tail probabilities of Markov chains and computing value functions for a class of stochastic optimal control problems.

stat.CO↗

arXiv2016

An algorithm for approximating the second moment of the normalizing constant estimate from a particle filter

We propose a new algorithm for approximating the non-asymptotic second moment of the marginal likelihood estimate, or normalizing constant, provided by a particle filter. The computational cost of the new method is $O(M)$ per time step, independently of the number of particles $N$ in the particle filter, where $M$ is a parameter controlling the quality of the approximation. This is in contrast to $O(MN)$ for a simple averaging technique using $M$ i.i.d. replicates of a particle filter with $N$ particles. We establish that the approximation delivered by the new algorithm is unbiased, strongly consistent and, under standard regularity conditions, increasing $M$ linearly with time is sufficient to prevent growth of the relative variance of the approximation, whereas for the simple averaging technique it can be necessary to increase $M$ exponentially with time in order to achieve the same effect. Numerical examples illustrate performance in the context of a stochastic Lotka\textendash Volterra system and a simple AR(1) model.

stat.ME↗

arXiv2016

Variance estimation in the particle filter

This paper concerns numerical assessment of Monte Carlo error in particle filters. We show that by keeping track of certain key features of the genealogical structure arising from resampling operations, it is possible to estimate variances of a number of standard Monte Carlo approximations which particle filters deliver. All our estimators can be computed from a single run of a particle filter with no further simulation. We establish that as the number of particles grows, our estimators are weakly consistent for asymptotic variances of the Monte Carlo approximations and some of them are also non-asymptotically unbiased. The asymptotic variances can be decomposed into terms corresponding to each time step of the algorithm, and we show how to consistently estimate each of these terms. When the number of particles may vary over time, this allows approximation of the asymptotically optimal allocation of particle numbers.

stat.CO↗

arXiv2016

Fluctuations, stability and instability of a distributed particle filter with local exchange

We study a distributed particle filter proposed by Bolić et al.~(2005). This algorithm involves $m$ groups of $M$ particles, with interaction between groups occurring through a "local exchange" mechanism. We establish a central limit theorem in the regime where $M$ is fixed and $m\to\infty$. A formula we obtain for the asymptotic variance can be interpreted in terms of colliding Markov chains, enabling analytic and numerical evaluations of how the asymptotic variance behaves over time, with comparison to a benchmark algorithm consisting of $m$ independent particle filters. We prove that subject to regularity conditions, when $m$ is fixed both algorithms converge time-uniformly at rate $M^{-1/2}$. Through use of our asymptotic variance formula we give counter-examples satisfying the same regularity conditions to show that when $M$ is fixed neither algorithm, in general, converges time-uniformly at rate $m^{-1/2}$.

stat.ME↗

arXiv2016

An Introduction to Twisted Particle Filters and Parameter Estimation in Non-linear State-space Models

Twisted particle filters are a class of sequential Monte Carlo methods recently introduced by Whiteley and Lee to improve the efficiency of marginal likelihood estimation in state-space models. The purpose of this article is to extend the twisted particle filtering methodology, establish accessible theoretical results which convey its rationale, and provide a demonstration of its practical performance within particle Markov chain Monte Carlo for estimating static model parameters. We derive twisted particle filters that incorporate systematic or multinomial resampling and information from historical particle states, and a transparent proof which identifies the optimal algorithm for marginal likelihood estimation. We demonstrate how to approximate the optimal algorithm for nonlinear state-space models with Gaussian noise and we apply such approximations to two examples: a range and bearing tracking problem and an indoor positioning problem with Bluetooth signal strength measurements. We demonstrate improvements over standard algorithms in terms of variance of marginal likelihood estimates and Markov chain autocorrelation for given CPU time, and improved tracking performance using estimated parameters.

stat.CO↗

arXiv2016

On the role of interaction in sequential Monte Carlo algorithms

We introduce a general form of sequential Monte Carlo algorithm defined in terms of a parameterized resampling mechanism. We find that a suitably generalized notion of the Effective Sample Size (ESS), widely used to monitor algorithm degeneracy, appears naturally in a study of its convergence properties. We are then able to phrase sufficient conditions for time-uniform convergence in terms of algorithmic control of the ESS, in turn achievable by adaptively modulating the interaction between particles. This leads us to suggest novel algorithms which are, in senses to be made precise, provably stable and yet designed to avoid the degree of interaction which hinders parallelization of standard algorithms. As a byproduct, we prove time-uniform convergence of the popular adaptive resampling particle filter.

stat.CO↗

arXiv2016

Perfect sampling for nonhomogeneous Markov chains and hidden Markov models

We obtain a perfect sampling characterization of weak ergodicity for backward products of finite stochastic matrices, and equivalently, simultaneous tail triviality of the corresponding nonhomogeneous Markov chains. Applying these ideas to hidden Markov models, we show how to sample exactly from the finite-dimensional conditional distributions of the signal process given infinitely many observations, using an algorithm which requires only an almost surely finite number of observations to actually be accessed. A notion of "successful" coupling is introduced and its occurrence is characterized in terms of conditional ergodicity properties of the hidden Markov model and related to the stability of nonlinear filters.

math.ST↗

arXiv2015

Stability with respect to initial conditions in V-norm for nonlinear filters with ergodic observations

We establish conditions for an exponential rate of forgetting of the initial distribution of nonlinear filters in $V$-norm, path-wise along almost all observation sequences. In contrast to previous works, our results allow for unbounded test functions. The analysis is conducted in an general setup involving nonnegative kernels in a random environment which allows treatment of filters and prediction filters in a single framework. The main result is illustrated on two examples, the first showing that a total variation norm stability result obtained by Douc et al. (2009) can be extended to $V$-norm without any additional assumptions, the second concerning a situation in which forgetting of the initial condition holds in $V$-norm for the filters, but the $V$-norm of each prediction filter is infinite.

stat.CO↗

arXiv2014

A hidden Markov model for decoding and the analysis of replay in spike trains

We present a hidden Markov model that describes variation in an animal's position associated with varying levels of activity in action potential spike trains of individual place cell neurons. The model incorporates a coarse-graining of position, which we find to be a more parsimonious description of the system than other models. We use a sequential Monte Carlo algorithm for Bayesian inference of model parameters, including the state space dimension, and we explain how to estimate position from spike train observations (decoding). We obtain greater accuracy over other methods in the conditions of high temporal resolution and small neuronal sample size. We also present a novel, model-based approach to the study of replay: the expression of spike train activity related to behaviour during times of motionlessness or sleep, thought to be integral to the consolidation of long-term memories. We demonstrate how we can detect the time, information content and compression rate of replay events in simulated and real hippocampal data recorded from rats in two different environments, and verify the correlation between the times of detected replay events and of sharp wave/ripples in the local field potential.

stat.AP↗

arXiv2014

Butterfly resampling: asymptotics for particle filters with constrained interactions

We generalize the elementary mechanism of sampling with replacement $N$ times from a weighted population of size $N$, by introducing auxiliary variables and constraints on conditional independence characterised by modular congruence relations. Motivated by considerations of parallelism, a convergence study reveals how sparsity of the mechanism's conditional independence graph is related to fluctuation properties of particle filters which use it for resampling, in some cases exhibiting exotic scaling behaviour. The proofs involve detailed combinatorial analysis of conditional independence graphs.

stat.ME↗

arXiv2014

Forest resampling for distributed sequential Monte Carlo

This paper brings explicit considerations of distributed computing architectures and data structures into the rigorous design of Sequential Monte Carlo (SMC) methods. A theoretical result established recently by the authors shows that adapting interaction between particles to suitably control the Effective Sample Size (ESS) is sufficient to guarantee stability of SMC algorithms. Our objective is to leverage this result and devise algorithms which are thus guaranteed to work well in a distributed setting. We make three main contributions to achieve this. Firstly, we study mathematical properties of the ESS as a function of matrices and graphs that parameterize the interaction amongst particles. Secondly, we show how these graphs can be induced by tree data structures which model the logical network topology of an abstract distributed computing environment. Thirdly, we present efficient distributed algorithms that achieve the desired ESS control, perform resampling and operate on forests associated with these trees.

stat.CO↗