Sharp regret-Hellinger bounds for Gaussian empirical Bayes via polynomial approximation

Abstract

A central problem in the theory of empirical Bayes is to control the regret (excess risk) of a learned Bayes rule by the Hellinger distance between the estimated and true marginal densities. In the normal means model, the classical result of Jiang and Zhang (2009) achieves this only after regularizing the Bayes rule and incurs an extraneous cubic logarithmic factor through a delicate recursive argument. This paper introduces a new technique, based on polynomial approximation and Bernstein-type inequalities for weighted $L_2$ norms, that bounds the unregularized regret directly. The method is conceptually simpler and yields sharper, sometimes optimal, regret bounds. For compactly supported priors, we prove the sharp bound that the regret is $O(\epsilon^2 \frac{\log(1/\epsilon)}{\log\log(1/\epsilon)})$, where $\epsilon$ is the Hellinger distance between the marginal densities. The same method also extends to priors with exponential tails. Conversely, we show that regularization is genuinely necessary for heavy-tailed priors under only bounded moment assumptions. As statistical consequences, we obtain improved regret bounds for the nonparametric maximum likelihood estimator (NPMLE). Notably, for compactly supported priors, by determining the optimal Hellinger rate of mixture density estimation, we show that the optimal regret for sample size $n$ scales as $\Theta(\frac{1}{n}(\frac{\log n}{\log\log n})^2)$, attained by the NPMLE within a $\log\log n$ factor.