Robust variable selection and estimation via adaptive elastic net S-estimators for linear regression

Abstract

Heavy-tailed error distributions and predictors with anomalous values are ubiquitous in high-dimensional regression problems and can seriously jeopardize the validity of statistical analyses if not properly addressed. For more reliable variable selection and prediction under these adverse conditions, adaptive PENSE, a new robust regularized regression estimator, is proposed. Adaptive PENSE yields reliable variable selection and coefficient estimates even under aberrant contamination in the predictors or residuals. It is shown that the adaptive penalty leads to more robust and reliable variable selection than other penalties, particularly in the presence of gross outliers in the predictor space. It is further demonstrated that adaptive PENSE has strong variable selection properties and that it possesses the oracle property even under heavy-tailed errors and without the need to estimate the error scale. Numerical studies on simulated and real data sets highlight the superior finite-sample performance in a vast range of settings compared to other robust regularized estimators in the case of contaminated samples. An R package implementing a fast algorithm for computing the proposed method and additional simulation results are provided in the supplementary materials.