arXiv subjects

Krzysztof Łatuszyński

Publications and source records attributed to Krzysztof Łatuszyński.

16 recordsLinked to original sources

arXiv2026

A debiased Bernoulli factory and unbiased estimation of a probability

Given a known function $f : [0, 1] \to (0, 1)$ and a random but almost surely finite number of independent, Ber$(x)$-distributed random variables with unknown $x \in [0, 1]$, we prove the existence of an unbiased, $[0, 1]$-valued estimator of the probability $f(x) \in (0, 1)$. Our estimator is based on so-called debiasing, or randomly truncating a telescopic series of consistent estimators. Debiased estimators of a probability are not typically constrained to $[0, 1]$, or even bounded, even when all consistent estimators used as inputs are. We show that constructing the series of consistent estimators from the coefficients of a particular Bernoulli factory yields provable boundedness provided $f \in C^ρ[0, 1]$ for $ρ> 3$. Our result can be thought of as a novel Bernoulli factory with the appealing property that the required number of Ber$(x)$-distributed random variates is independent of their outcomes.

math.PR↗

arXiv2025

Exact Bayesian inference for Markov switching diffusions

We develop the first exact Bayesian methodology for the problem of inference in discretely observed regime switching diffusions. Switching diffusion models extend ordinary diffusions by allowing for jumps in instantaneous drift and volatility. The jumps are driven by a latent, continuous time Markov switching process. We address the problem through an MCMC and an MCEM algorithm that target the exact posterior of diffusion parameters and the latent regime process. The algorithms are exact in the sense that they target the correct posterior distribution of the continuous model, so that the errors are due to Monte Carlo only. We illustrate the method on numerical examples, including an empirical analysis of the method's scalability in the length of the time series, and find that it is comparable in computational cost with discrete approximations while avoiding their shortcomings.

stat.CO↗

arXiv2025

MCMC for multi-modal distributions

We explain the fundamental challenges of sampling from multimodal distributions, particularly for high-dimensional problems. We present the major types of MCMC algorithms that are designed for this purpose, including parallel tempering, mode jumping and Wang-Landau, as well as several state-of-the-art approaches that have recently been proposed. We demonstrate these methods using both synthetic and real-world examples of multimodal distributions with discrete or continuous state spaces.

stat.CO↗

arXiv2024

Solidarity of Gibbs Samplers: the spectral gap

Gibbs samplers are preeminent Markov chain Monte Carlo algorithms used in computational physics and statistical computing. Yet, their most fundamental properties, such as relations between convergence characteristics of their various versions, are not well understood. In this paper we prove the solidarity of their spectral gaps: if any of the random scan or $d!$ deterministic scans has a~spectral gap then all of them have. Our methods rely on geometric interpretation of the Gibbs samplers as alternating projection algorithms and analysis of the rate of convergence in the von Neumann--Halperin method of cyclic alternating projections. In addition, we provide a quantitative result: if the spectral gap of the random scan Gibbs sampler scales polynomially with dimension, so does the spectral gap of any of the deterministic scans.

stat.CO↗

arXiv2024

Stereographic Markov Chain Monte Carlo

High-dimensional distributions, especially those with heavy tails, are notoriously difficult for off-the-shelf MCMC samplers: the combination of unbounded state spaces, diminishing gradient information, and local moves results in empirically observed ``stickiness'' and poor theoretical mixing properties -- lack of geometric ergodicity. In this paper, we introduce a new class of MCMC samplers that map the original high-dimensional problem in Euclidean space onto a sphere and remedy these notorious mixing problems. In particular, we develop random-walk Metropolis type algorithms as well as versions of the Bouncy Particle Sampler that are uniformly ergodic for a large class of light and heavy-tailed distributions and also empirically exhibit rapid convergence in high dimensions. In the best scenario, the proposed samplers can enjoy the ``blessings of dimensionality'' that the convergence is faster in higher dimensions.

stat.CO↗

arXiv2024

Convergence of hybrid slice sampling via spectral gap

It is known that the simple slice sampler has robust convergence properties, however the class of problems where it can be implemented is limited. In contrast, we consider hybrid slice samplers which are easily implementable and where another Markov chain approximately samples the uniform distribution on each slice. Under appropriate assumptions on the Markov chain on the slice we show a lower bound and an upper bound of the spectral gap of the hybrid slice sampler in terms of the spectral gap of the simple slice sampler. An immediate consequence of this is that spectral gap and geometric ergodicity of the hybrid slice sampler can be concluded from spectral gap and geometric ergodicity of its simple version which is very well understood. These results indicate that robustness properties of the simple slice sampler are inherited by (appropriately designed) easily implementable hybrid versions. We apply the developed theory and analyse a number of specific algorithms such as the stepping-out shrinkage slice sampling, hit-and-run slice sampling on a class of multivariate targets and an easily implementable combination of both procedures on multidimensional bimodal densities.

stat.ME↗

arXiv2024

Bernoulli factories and duality in Wright-Fisher and Allen-Cahn models of population genetics

Mathematical models of genetic evolution often come in pairs, connected by a so-called duality relation. The most seminal example are the Wright-Fisher diffusion and the Kingman coalescent, where the former describes the stochastic evolution of neutral allele frequencies in a large population forwards in time, and the latter describes the genetic ancestry of randomly sampled individuals from the population backwards in time. As well as providing a richer description than either model in isolation, duality often yields equations satisfied by quantities of interest. We employ the so-called Bernoulli factory - a celebrated tool in simulation-based computing - to derive duality relations for broad classes of genetics models. As concrete examples, we present Wright-Fisher diffusions with general drift functions, and Allen-Cahn equations with general, nonlinear forcing terms. The drift and forcing functions can be interpreted as the action of frequency-dependent selection. To our knowledge, this work is the first time a connection has been drawn between Bernoulli factories and duality in models of population genetics.

math.PR↗

arXiv2022

Optimal Scaling of MCMC Beyond Metropolis

The problem of optimally scaling the proposal distribution in a Markov chain Monte Carlo algorithm is critical to the quality of the generated samples. Much work has gone into obtaining such results for various Metropolis-Hastings (MH) algorithms. Recently, acceptance probabilities other than MH are being employed in problems with intractable target distributions. There is little resource available on tuning the Gaussian proposal distributions for this situation. We obtain optimal scaling results for a general class of acceptance functions, which includes Barker's and Lazy-MH. In particular, optimal values for the Barker's algorithm are derived and found to be significantly different from that obtained for the MH algorithm. Our theoretical conclusions are supported by numerical simulations indicating that when the optimal proposal variance is unknown, tuning to the optimal acceptance probability remains an effective strategy.

stat.CO↗

arXiv2021

Efficient Bernoulli factory MCMC for intractable posteriors

Accept-reject based Markov chain Monte Carlo (MCMC) algorithms have traditionally utilised acceptance probabilities that can be explicitly written as a function of the ratio of the target density at the two contested points. This feature is rendered almost useless in Bayesian posteriors with unknown functional forms. We introduce a new family of MCMC acceptance probabilities that has the distinguishing feature of not being a function of the ratio of the target density at the two points. We present two stable Bernoulli factories that generate events within this class of acceptance probabilities. The efficiency of our methods rely on obtaining reasonable local upper or lower bounds on the target density and we present two classes of problems where such bounds are viable: Bayesian inference for diffusions and MCMC on constrained spaces. The resulting portkey Barker's algorithms are exact and computationally more efficient that the current state-of-the-art.

stat.CO↗

arXiv2019

A Framework for Adaptive MCMC Targeting Multimodal Distributions

We propose a new Monte Carlo method for sampling from multimodal distributions. The idea of this technique is based on splitting the task into two: finding the modes of a target distribution $π$ and sampling, given the knowledge of the locations of the modes. The sampling algorithm relies on steps of two types: local ones, preserving the mode; and jumps to regions associated with different modes. Besides, the method learns the optimal parameters of the algorithm while it runs, without requiring user intervention. Our technique should be considered as a flexible framework, in which the design of moves can follow various strategies known from the broad MCMC literature. In order to design an adaptive scheme that facilitates both local and jump moves, we introduce an auxiliary variable representing each mode and we define a new target distribution $\tildeπ$ on an augmented state space $\mathcal{X}~\times~\mathcal{I}$, where $\mathcal{X}$ is the original state space of $π$ and $\mathcal{I}$ is the set of the modes. As the algorithm runs and updates its parameters, the target distribution $\tildeπ$ also keeps being modified. This motivates a new class of algorithms, Auxiliary Variable Adaptive MCMC. We prove general ergodic results for the whole class before specialising to the case of our algorithm.

stat.CO↗

arXiv2017

Barker's algorithm for Bayesian inference with intractable likelihoods

In this expository paper we abstract and describe a simple MCMC scheme for sampling from intractable target densities. The approach has been introduced in Gonçalves et al. (2017a) in the specific context of jump-diffusions, and is based on the Barker's algorithm paired with a simple Bernoulli factory type scheme, the so called 2-coin algorithm. In many settings it is an alternative to standard Metropolis-Hastings pseudo-marginal method for simulating from intractable target densities. Although Barker's is well-known to be slightly less efficient than Metropolis-Hastings, the key advantage of our approach is that it allows to implement the "marginal Barker's" instead of the extended state space pseudo-marginal Metropolis-Hastings, owing to the special form of the accept/reject probability. We shall illustrate our methodology in the context of Bayesian inference for discretely observed Wright-Fisher family of diffusions.

stat.CO↗

arXiv2015

Stability of adversarial Markov chains, with an application to adaptive MCMC algorithms

We consider whether ergodic Markov chains with bounded step size remain bounded in probability when their transitions are modified by an adversary on a bounded subset. We provide counterexamples to show that the answer is no in general, and prove theorems to show that the answer is yes under various additional assumptions. We then use our results to prove convergence of various adaptive Markov chain Monte Carlo algorithms.

math.PR↗

arXiv2015

Bayesian computation: a perspective on the current state, and sampling backwards and forwards

The past decades have seen enormous improvements in computational inference based on statistical models, with continual enhancement in a wide range of computational tools, in competition. In Bayesian inference, first and foremost, MCMC techniques continue to evolve, moving from random walk proposals to Langevin drift, to Hamiltonian Monte Carlo, and so on, with both theoretical and algorithmic inputs opening wider access to practitioners. However, this impressive evolution in capacity is confronted by an even steeper increase in the complexity of the models and datasets to be addressed. The difficulties of modelling and then handling ever more complex datasets most likely call for a new type of tool for computational inference that dramatically reduce the dimension and size of the raw data while capturing its essential aspects. Approximate models and algorithms may thus be at the core of the next computational revolution.

stat.CO↗

arXiv2014

Perfect simulation using atomic regeneration with application to Sequential Monte Carlo

Consider an irreducible, Harris recurrent Markov chain of transition kernel Π and invariant probability measure π. If Π satisfies a minorization condition, then the split chain allows the identification of regeneration times which may be exploited to obtain perfect samples from π. Unfortunately, many transition kernels associated with complex Markov chain Monte Carlo algorithms are analytically intractable, so establishing a minorization condition and simulating the split chain is challenging, if not impossible. For uniformly ergodic Markov chains with intractable transition kernels, we propose two efficient perfect simulation procedures of similar expected running time which are instances of the multigamma coupler and an imputation scheme. These algorithms overcome the intractability of the kernel by introducing an artificial atom and using a Bernoulli factory. We detail an application of these procedures when Π is the recently introduced iterated conditional Sequential Monte Carlo kernel. We additionally provide results on the general applicability of the methodology, and how Sequential Monte Carlo methods may be used to facilitate perfect simulation and/or unbiased estimation of expectations with respect to the stationary distribution of a non-uniformly ergodic Markov chain.

stat.CO↗

arXiv2013

Nonasymptotic bounds on the estimation error of MCMC algorithms

We address the problem of upper bounding the mean square error of MCMC estimators. Our analysis is nonasymptotic. We first establish a general result valid for essentially all ergodic Markov chains encountered in Bayesian computation and a possibly unbounded target function $f$. The bound is sharp in the sense that the leading term is exactly $σ_{\mathrm {as}}^2(P,f)/n$, where $σ_{\mathrm{as}}^2(P,f)$ is the CLT asymptotic variance. Next, we proceed to specific additional assumptions and give explicit computable bounds for geometrically and polynomially ergodic Markov chains under quantitative drift conditions. As a corollary, we provide results on confidence estimation.

stat.ME↗

arXiv2013

Adaptive Gibbs samplers and related MCMC methods

We consider various versions of adaptive Gibbs and Metropolis-within-Gibbs samplers, which update their selection probabilities (and perhaps also their proposal distributions) on the fly during a run by learning as they go in an attempt to optimize the algorithm. We present a cautionary example of how even a simple-seeming adaptive Gibbs sampler may fail to converge. We then present various positive results guaranteeing convergence of adaptive Gibbs samplers under certain conditions.

stat.CO↗