Feature screening strategy for non-convex sparse logistic regression with log sum penalty

Abstract

L1 logistic regression is an efficient classification algorithm. Leveraging on the convexity of the model and the sparsity of L1 norm, techniques named safe screening rules can help accelerate solvers for high-dimensional data. Recently, Log Sum Penalty (LSP) provides better theoretical guarantees in identifying relevant variables, and sparse logistic regression with LSP has been proposed. However, due to the non-convexity of this model, the training process is time-consuming. Furthermore, the existing safe screening rules cannot be directly applied to accelerate it. To deal with this issue, in this paper, based on the iterative majorization minimization (MM) principle, we construct a novel method that can effectively save training time. To do this, we first design a feature screening strategy for the inner solver and then build another rule to propagate screened features between the iterations of MM. After that, we introduce a modified feature screening strategy to further accelerate the computational speed, which can obtain a smaller safe region thanks to reconstructing the strong global concavity bound. Moreover, our rules can be applied to other non-convex cases. Experiments on nine benchmark datasets verify the effectiveness and security of our algorithm.