A safe reinforced feature screening strategy for lasso based on feasible solutions

Abstract

As a popular method in machine learning, lasso performs regression and feature selection simultaneously. However, for large datasets, the training efficiency of lasso remains a challenge. Recently, an Enhanced screening rule via Dual Polytope Projection (EDPP) was proposed to substantially reduce the scale of lasso by deleting inactive features beforehand. However, EDPP may mistakenly discard active features in practice due to the unavailable optimal solutions. To solve this problem, a safe reinforced feature screening rule based on EDPP and feasible solutions (S-EDPP) is introduced in this paper. By utilizing feasible solutions and estimating a proper upper bound of the deviation, S-EDPP can be guaranteed to be safe both in theory and in practice. Theoretical analysis of the deviation term in S-EDPP is given to verify its efficiency. Furthermore, S-EDPP is also extended to accelerate the Elastic Net, which is a corrective method of lasso. Experiments on synthetic and real datasets verify that S-EDPP is a safe modification of EDPP and it gives superior performance than other existing safe rules.