Improved Concentration for Mean Estimators via Shrinkage

Abstract

We study a class of robust mean estimators $\widehat{\mu}$ obtained by adaptively shrinking the weights of sample points far from a base estimator $\widehat{\kappa}$. Given a data-dependent scaling factor $\widehat{\alpha}$ and a weighting function $w:[0, \infty) \to [0,1]$, we let $\widehat{\mu}=\widehat{\kappa} + \frac{1}{n}\sum_{i=1}^n(X_i - \widehat{\kappa})w(\widehat{\alpha}|X_i-\widehat{\kappa}|)$. We prove that, under mild assumptions over $w$, these estimators achieve stronger concentration bounds than the base estimate $\widehat{\kappa}$, including sub-Gaussian guarantees. This framework unifies and extends several existing approaches to robust mean estimation in $\R$, and can also be generalized to the multivariate setting. Through numerical experiments, we show that our shrinking approach translates to faster concentration, even for small sample sizes.