Scalable Bernoulli Factory MCMC for Intractable Marginalised Posteriors

Abstract

Bernoulli factory MCMC algorithms implement accept-reject Markov chains without explicit computation of acceptance probabilities, and are used to target posterior distributions associated with intractable likelihood models. Intractable likelihoods naturally arise in continuous-time models and mixture distributions, or from the marginalisation of a tractable augmented model. Bernoulli factory MCMC algorithms often mix better than alternatives that target a tractable augmented posterior. However, we show that their computational performance typically deteriorates exponentially with data size, as happens in naively implemented pseudo-marginal algorithms. To address this, we propose a simple divide-and-conquer Bernoulli factory MCMC algorithm and prove that it has polynomial complexity for factorizing models, with the exact degree depending on the existence of efficient unbiased estimators of the intractable likelihood ratio. We demonstrate the effectiveness of our approach with applications to Bayesian inference in two intractable likelihood models, and observe respective polynomial cost of degree 1.2 and 1 in the data size.