Robust Best Subset Selection via Fast Approximate MM-Estimation
Best subset selection procedures typically rely on a squared error loss, where a small number of outlying observations may distort the entire solution path. Replacing this loss with a robust alternative, such as the MM-estimator robust model selection criterion, is computationally prohibitive, as it requires an iterative fit across near-exhaustive candidate model spaces. To preserve the robust properties of the MM-estimator while avoiding its repeated iterative computation, we introduce FAMM, a Fast Approximate MM-Estimator based on a weighted least squares fit with weights obtained from a full data MM-estimator. Although the resulting estimator is no longer an MM-estimator, we prove that model selection consistency is retained when aligning the weights of the selection criterion with those of the estimator. We embed FAMM within COMBSS, whose continuous relaxation of the discrete best subset search enables scalable best subset selection for large numbers of predictor variables. Consequently, robust model selection and best subset selection with large and contaminated datasets are made feasible.