arXiv subjects

Kari Heine

Publications and source records attributed to Kari Heine.

10 recordsLinked to original sources

arXiv2023

Augmented Island Resampling Particle Filters for Particle Markov Chain Monte Carlo

In modern days, the ability to carry out computations in parallel is key to efficient implementations of computationally intensive algorithms. This paper investigates the applicability of the previously proposed Augmented Island Resampling Particle Filter (AIRPF) -- an algorithm designed for parallel implementations -- to particle Markov Chain Monte Carlo (PMCMC). We show that AIRPF produces a non-negative unbiased estimator of the marginal likelihood and hence is suitable for PMCMC. We also prove stability properties, similar to those of the $α$SMC algorithm, for AIRPF. This implies that the error of AIRPF can be bounded uniformly in time by controlling the effective number of filters, which in turn can be done by adaptively constraining the interactions between filters. We demonstrate the superiority of AIRPF over independent Bootstrap Particle Filters, not only numerically, but also theoretically. To this end, we extend the previously proposed collision analysis approach to derive an explicit expression for the variance of the marginal likelihood estimate. This expression admits exact evaluation of the variance in some simple scenarios as we shall also demonstrate.

stat.CO↗

arXiv2022

Multilevel Bootstrap Particle Filter

We consider situations where the applicability of sequential Monte Carlo particle filters is compromised due to the expensive evaluation of the particle weights. To alleviate this problem, we propose a new particle filter algorithm based on the multilevel approach. We show that the resulting multilevel bootstrap particle filter (MLBPF) retains the strong law of large numbers as well as the central limit theorem of classical particle filters under mild conditions. Our numerical experiments demonstrate up to 85\% reduction in computation time compared to the classical bootstrap particle filter, in certain settings. While it should be acknowledged that this reduction is highly application dependent, and a similar gain should not be expected for all applications across the board, we believe that this substantial improvement in certain settings makes MLBPF an important addition to the family of sequential Monte Carlo methods.

stat.CO↗

arXiv2020

Online Inference with Multi-modal Likelihood Functions

Let $(Y_t)_{t\geq 1}$ be a sequence of i.i.d.\ observations and $\{f_θ,θ\in \mathbb{R}^d\}$ be a parametric model. We introduce a new online algorithm for computing a sequence $(\hatθ_t)_{t\geq 1}$ which is shown to converge almost surely to $\text{argmax}_{θ\in \mathbb{R}^d}\mathbb{E}[\log f_θ(Y_1)]$ at rate $ \mathcal{O}(\log (t)^{(1+\varepsilon)/2}t^{-1/2})$, with $\varepsilon>0$ a user specified parameter. This convergence result is obtained under standard conditions on the statistical model and, most notably, we allow the mapping $θ\mapsto \mathbb{E}[\log f_θ(Y_1)]$ to be multi-modal. However, the computational cost to process each observation grows exponentially with the dimension of $θ$, which makes the proposed approach applicable to low or moderate dimensional problems only. We also derive a version of the estimator $\hatθ_t$ which is well suited to Student-t linear regression models. The corresponding estimator of the regression coefficients is robust to the presence of outliers, as shown by experiments on simulated and real data, and thus, as a by-product of this work, we obtain a new online and adaptive robust estimation method for linear regression models.

math.ST↗

arXiv2018

Parallelising Particle Filters with Butterfly Interactions

Bootstrap particle filter (BPF) is the corner stone of many popular algorithms used for solving inference problems involving time series that are observed through noisy measurements in a non-linear and non-Gaussian context. The long term stability of BPF arises from particle interactions which in the context of modern parallel computing systems typically means that particle information needs to be communicated between processing elements, which makes parallel implementation of BPF nontrivial. In this paper we show that it is possible to constrain the interactions in a way which, under some assumptions, enables the reduction of the cost of communicating the particle information while still preserving the consistency and the long term stability of the BPF. Numerical experiments demonstrate that although the imposed constraints introduce additional error, the proposed method shows potential to be the method of choice in certain settings.

stat.CO↗

arXiv2018

Bridging trees for posterior inference on Ancestral Recombination Graphs

We present a new Markov chain Monte Carlo algorithm, implemented in software Arbores, for inferring the history of a sample of DNA sequences. Our principal innovation is a bridging procedure, previously applied only for simple stochastic processes, in which the local computations within a bridge can proceed independently of the rest of the DNA sequence, facilitating large-scale parallelisation.

stat.CO↗

arXiv2016

Fluctuations, stability and instability of a distributed particle filter with local exchange

We study a distributed particle filter proposed by Bolić et al.~(2005). This algorithm involves $m$ groups of $M$ particles, with interaction between groups occurring through a "local exchange" mechanism. We establish a central limit theorem in the regime where $M$ is fixed and $m\to\infty$. A formula we obtain for the asymptotic variance can be interpreted in terms of colliding Markov chains, enabling analytic and numerical evaluations of how the asymptotic variance behaves over time, with comparison to a benchmark algorithm consisting of $m$ independent particle filters. We prove that subject to regularity conditions, when $m$ is fixed both algorithms converge time-uniformly at rate $M^{-1/2}$. Through use of our asymptotic variance formula we give counter-examples satisfying the same regularity conditions to show that when $M$ is fixed neither algorithm, in general, converges time-uniformly at rate $m^{-1/2}$.

stat.ME↗

arXiv2016

On the role of interaction in sequential Monte Carlo algorithms

We introduce a general form of sequential Monte Carlo algorithm defined in terms of a parameterized resampling mechanism. We find that a suitably generalized notion of the Effective Sample Size (ESS), widely used to monitor algorithm degeneracy, appears naturally in a study of its convergence properties. We are then able to phrase sufficient conditions for time-uniform convergence in terms of algorithmic control of the ESS, in turn achievable by adaptively modulating the interaction between particles. This leads us to suggest novel algorithms which are, in senses to be made precise, provably stable and yet designed to avoid the degree of interaction which hinders parallelization of standard algorithms. As a byproduct, we prove time-uniform convergence of the popular adaptive resampling particle filter.

stat.CO↗

arXiv2015

An Introduction to Twisted Particle Filters and Parameter Estimation in Non-linear State-space Models

Twisted particle filters are a class of sequential Monte Carlo methods recently introduced by Whiteley and Lee to improve the efficiency of marginal likelihood estimation in state-space models. The purpose of this article is to extend the twisted particle filtering methodology, establish accessible theoretical results which convey its rationale, and provide a demonstration of its practical performance within particle Markov chain Monte Carlo for estimating static model parameters. We derive twisted particle filters that incorporate systematic or multinomial resampling and information from historical particle states, and a transparent proof which identifies the optimal algorithm for marginal likelihood estimation. We demonstrate how to approximate the optimal algorithm for nonlinear state-space models with Gaussian noise and we apply such approximations to two examples: a range and bearing tracking problem and an indoor positioning problem with Bluetooth signal strength measurements. We demonstrate improvements over standard algorithms in terms of variance of marginal likelihood estimates and Markov chain autocorrelation for given CPU time, and improved tracking performance using estimated parameters.

stat.CO↗

arXiv2015

Bayesian Inference for Duplication-Mutation with Complementarity Network Models

We observe an undirected graph $G$ without multiple edges and self-loops, which is to represent a protein-protein interaction (PPI) network. We assume that $G$ evolved under the duplication-mutation with complementarity (DMC) model from a seed graph, $G_0$, and we also observe the binary forest $Γ$ that represents the duplication history of $G$. A posterior density for the DMC model parameters is established, and we outline a sampling strategy by which one can perform Bayesian inference; that sampling strategy employs a particle marginal Metropolis-Hastings (PMMH) algorithm. We test our methodology on numerical examples to demonstrate a high accuracy and precision in the inference of the DMC model's mutation and homodimerization parameters.

stat.CO↗

arXiv2014

Butterfly resampling: asymptotics for particle filters with constrained interactions

We generalize the elementary mechanism of sampling with replacement $N$ times from a weighted population of size $N$, by introducing auxiliary variables and constraints on conditional independence characterised by modular congruence relations. Motivated by considerations of parallelism, a convergence study reveals how sparsity of the mechanism's conditional independence graph is related to fluctuation properties of particle filters which use it for resampling, in some cases exhibiting exotic scaling behaviour. The proofs involve detailed combinatorial analysis of conditional independence graphs.

stat.ME↗