A fast robust best subset regression

Abstract

In this paper, we present a novel approach called the Fast Robust Best Subset Regression (FRBSR) procedure, specifically designed to tackle outliers in both covariates and response variables. Through the incorporation of C-steps, the FRBSR procedure not only enhances the robustness of the estimation process, but also relaxes the stringent assumptions imposed by the Enhanced Support Detection and Root Finding (ESDAR, Huang et al. (2018, 2021)) algorithm regarding the distribution of covariates and response variables. More specifically, we propose additional techniques, namely reweighted least squares (REWLS) and one-step weighted least squares (OSWLS), which build upon the L0 regularized least trimmed squares (LTS) method and aim to improve estimation efficiency. Compared to the majorization-minimization (MM, Liu et al. (2021)) principle for solving non-convex loss L2E, our proposed OSWLS requires only one step to compute the L0 regularized weighted least squares, resulting in reduced estimation error and computational complexity. Extensive simulations and real data analysis demonstrate the superior performance of our FRBSR procedure in terms of estimation accuracy and prediction compared to the latest robust methods.