Robust and sparse estimators for linear regression models

Abstract

Penalized regression estimators are a popular tool for the analysis of sparse and high-dimensional data sets. However, penalized regression estimators defined using an unbounded loss function can be very sensitive to the presence of outlying observations, especially high leverage outliers. Moreover, it can be particularly challenging to detect outliers in high-dimensional data sets. Thus, robust estimators for sparse and high-dimensional linear regression models are in need. In this paper, we study the robust and asymptotic properties of MM-Bridge and adaptive MM-Bridge estimators: $\ell_q$-penalized MM-estimators of regression and MM-estimators with an adaptive $\ell_t$ penalty. For the case of a fixed number of covariates, we derive the asymptotic distribution of MM-Bridge estimators for all $q>0$. We prove that for $q<1$ MM-Bridge estimators can have the oracle property defined in Fan and Li (2001). We prove that for all $t\leq 1$ adaptive MM-Bridge estimators can have the oracle property. The advantages of our proposed estimators are demonstrated through an extensive simulation study and the analysis of a real high-dimensional data set.