Bayesian Empirical Bayes: Simultaneous Inference from Probabilistic Symmetries

Abstract

Empirical Bayes (EB) improves the accuracy of simultaneous inference "by learning from the experience of others" (Efron, 2012). Classical EB theory focuses on latent variables that are iid draws from a fitted prior (Efron, 2019). Modern applications, however, feature complex structure, like arrays, spatial processes, or covariates. We propose a generalized approach to empirical Bayes based on probabilistic symmetry. Our method pairs a simultaneous inference problem with an unknown prior to a symmetry assumption on the joint distribution of the latent variables. Each symmetry implies an ergodic decomposition, which we use to derive a corresponding empirical Bayes method. We call this method Bayesian empirical Bayes (BEB). BEB recovers classical empirical Bayes methods, which implicitly assume exchangeability. We extend EB to other probabilistic symmetries: (i) EB matrix recovery for arrays and graphs; (ii) covariate-informed EB for conditional data; and (iii) EB spatial regression under shift invariance. We develop scalable algorithms based on variational inference and neural networks. In simulations, BEB outperforms existing denoising methods. On real data, we demonstrate BEB on cancer gene-expression and brain-connectivity matrices and NYC air-quality data.