The Block Pseudo-Marginal Sampler

Abstract

The pseudo-marginal (PM) approach is increasingly used for Bayesian inference in statistical models, where the likelihood is intractable but can be estimated unbiasedly. %Examples include random effect models, state-space models and data subsampling in big-data settings. Deligiannidis et al. (2016) show how the PM approach can be made much more efficient by correlating the underlying Monte Carlo (MC) random numbers used to form the estimate of the likelihood at the current and proposed values of the unknown parameters. Their approach greatly speeds up the standard PM algorithm, as it requires a much smaller number of samples or particles to form the optimal likelihood estimate. Our paper presents an alternative implementation of the correlated PM approach, called the block PM, which divides the underlying random numbers into blocks so that the likelihood estimates for the proposed and current values of the parameters only differ by the random numbers in one block. We show that this implementation of the correlated PM can be much more efficient for some specific problems than the implementation in Deligiannidis et al. (2016); for example when the likelihood is estimated by subsampling or the likelihood is a product of terms each of which is given by an integral which can be estimated unbiasedly by randomised quasi-Monte Carlo. Our article provides methodology and guidelines for efficiently implementing the block PM. A second advantage of the the block PM is that it provides a direct way to control the correlation between the logarithms of the estimates of the likelihood at the current and proposed values of the parameters than the implementation in Deligiannidis et al. (2016). We obtain methods and guidelines for selecting the optimal number of samples based on idealized but realistic assumptions.