Non-asymptotic model selection for linear non least-squares estimation in regression models and inverse problems

Abstract

We propose to address the common problem of linear estimation in linear statistical models by using a model selection approach via penalization. Depending then on the framework in which the linear statistical model is considered namely the regression framework or the inverse problem framework, a data-driven model selection criterion is obtained either under general assumptions, or under the mild assumption of model identifiability respectively. The proposed approach was stimulated by the important recent non-asymptotic model selection results due to Birgé and Massart mainly (Birge and Massart 2007), and our results in this paper, like theirs, are non-asymptotic and turn to be sharp. Our main contribution in this paper resides in the fact that these linear estimators are not necessarily least-squares estimators but can be any linear estimators. The proposed approach finds therefore potential applications in countless fields of engineering and applied science (image science, signal processing,applied statistics, coding, to name a few) in which one is interested in recovering some unknown vector quantity of interest as the one, for example, which achieves the best trade-off between a term of fidelity to data, and a term of regularity or/and parsimony of the solution. The proposed approach provides then such applications with an interesting model selection framework that allows them to achieve such a goal.