Sequential Exchange Monte Carlo: A Sampling Method for Bayesian Data Analysis without Parameter Tuning

Abstract

Bayesian data analysis is widely used across many disciplines, and representative examples in materials science include spectral analysis and sparse modeling. In such applications, the underlying models often become complex and yield multimodal posterior distributions, making efficient sampling from multimodal distributions essential. Replica exchange Monte Carlo has been commonly employed for this purpose; however, its performance strongly depends on difficult parameter tuning, such as the design of the inverse temperature. In this study, we comparatively investigate sampling algorithms that require fewer tuning parameters for Bayesian data analysis in materials science. Specifically, we compare three approaches: non-reversible parallel tempering (NRPT), sequential Monte Carlo samplers (SMCS), and a newly proposed method, sequential exchange Monte Carlo (SEMC). Our results indicate that NRPT can require computational time for parameter tuning, while SMCS requires careful adjustment of the number of MCMC steps at each temperature level. In contrast, SEMC achieves robust convergence across a range of problem settings without additional tuning, demonstrating its practicality for Bayesian inference.