Robust and sparse estimators for linear regression models

Abstract

Penalized regression estimators are popular tools for the analysis of sparse and high-dimensional models. However, penalized regression estimators defined using an unbounded loss function can be very sensitive to the presence of outlying observations, especially to high leverage outliers. The robust and asymptotic properties of ℓ1-penalized MM-estimators and MM-estimators with an adaptive ℓ1 penalty are studied. For the case of a fixed number of covariates, the asymptotic distribution of the estimators is derived and it is proven that for the case of an adaptive ℓ1 penalty, the resulting estimator can have the oracle property. The advantages of the proposed estimators are demonstrated through an extensive simulation study and the analysis of real data sets. The proofs of the theoretical results are available in the Supplementary material to this article (see Appendix A).