Model selection in a sparse heterogeneous framework

Abstract

We consider a Gaussian sequence space model $X_{\lambda}=f_{\lambda} + \xi_{\lambda},$ where $\xi $ has a diagonal covariance matrix $\Sigma=\diag(\sigma_\lambda ^2)$. We consider the situation where the parameter vector $(f_{\lambda})$ is sparse. Our goal is to estimate the unknown parameter by a model selection approach. The heterogenous case is much more involved than the direct model. Indeed, there is no more symmetry inside the stochastic process that one needs to control since each empirical coefficient has its own variance. The problem and the penalty do not only depend on the number of coefficients that one selects, but also on their position. This appears also in the minimax bounds where the worst coefficients will go to the larger variances. However, with a careful and explicit choice of the penalty we are able to select the correct coefficients and get a sharp non-asymptotic control of the risk of our procedure. Some simulation results are provided.