Linearized alternating direction method of multipliers for elastic-net support vector machines

Abstract

In many high-dimensional datasets, the phenomenon that features are relevant often occurs. Elastic-net regularization is widely used in support vector machines (SVMs) because it can automatically perform feature selection and encourage highly correlated features to be selected or removed together. Recently, some effective algorithms have been proposed to solve the elastic-net SVMs with different convex loss functions, such as hinge, squared hinge, huberized hinge, pinball and huberized pinball. In this paper, we develop a linearized alternating direction method of multipliers (LADMM) algorithm to solve above elastic-net SVMs. In addition, our algorithm can be applied to solve some new elastic-net SVMs such as elastic-net least squares SVM. Compared with some existing algorithms, our algorithm has comparable or better performances in terms of computational cost and accuracy. Under mild conditions, we prove the convergence and derive convergence rate of our algorithm. Furthermore, numerical experiments on synthetic and real datasets demonstrate the feasibility and validity of the proposed algorithm.