Random construction of interpolating sets for high dimensional integration

Abstract

Many high dimensional integrals can be reduced to the problem of finding the relative measures of two sets. Often one set will be exponentially larger than the other, making it difficult to compare the sizes. A standard method of dealing with this problem is to interpolate between the sets with a sequence of nested sets where neighboring sets have relative measures bounded above by a constant. Choosing such a well balanced sequence can be very difficult in practice. Here a new approach that automatically creates such sets is presented. These well balanced sets allow for faster approximation algorithms for integrals and sums, and better tempering and annealing Markov chains for generating random samples. Applications such as finding the partition function of the Ising model and normalizing constants for posterior distributions in Bayesian methods are discussed.