Uniform mean estimation via generic chaining

Abstract

We introduce an empirical functional $\Psi$ that is an optimal uniform mean estimator: Let $F\subset L_2(\mu)$ be a class of mean zero functions, $u$ is a real valued function, and $X_1,\dots,X_N$ are independent, distributed according to $\mu$. We show that under minimal assumptions, with $\mu^{\otimes N}$ exponentially high probability, \[ \sup_{f\in F} |\Psi(X_1,\dots,X_N,f) - \mathbb{E} u(f(X))| \leq c R(F) \frac{ \mathbb{E} \sup_{f\in F } |G_f| }{\sqrt N}, \] where $(G_f)_{f\in F}$ is the gaussian processes indexed by $F$ and $R(F)$ is an appropriate notion of `diameter' of the class $\{u(f(X)) : f\in F\}$. The fact that such a bound is possible is surprising, and it leads to the solution of various key problems in high dimensional probability and high dimensional statistics. The construction is based on combining Talagrand's generic chaining mechanism with optimal mean estimation procedures for a single real-valued random variable.