Exact Sampling of Gibbs Measures with Estimated Losses

Abstract

A popular strategy for ameliorating some of the shortcomings of Bayesian posterior inference is to instead target a Gibbs measure based on losses that connect a parameter of interest to observed data, and which are known to be robust to misspecification. Existing theory for these procedures treats these losses as being analytically available, but in many situations these losses must be stochastically estimated using pseudo-observations. In these settings, and even under strong assumptions, we show that posteriors based on standard Markov chain Monte Carlo (MCMC) algorithms exhibit strong dependence on the number of these pseudo-observations, and require utilizing a diverging number of pseudo-observations to ensure posterior concentration. To remedy this issue, we introduce a modified piecewise deterministic Markov process (PDMP) sampler, and formally show that its posterior draws have no dependence on the number of pseudo-observations used to estimate the loss within a Gibbs measure. We verify the practical utility of this approach on five examples spanning intractable likelihoods and intractable losses.