A 1-norm regularized linear programming nonparallel hyperplane support vector machine for binary classification problems

Abstract

This research proposes a 1-norm regularized linear programming nonparallel hyperplane support vector machine (LNSVM) model to solve binary classification problems and enhance the robustness performance. Numerous nonparallel support vector machine (SVM) models have been studied with outstanding performance on classification tasks. However, most nonparallel SVM models require two independent models to determine hyperplanes. In addition, due to the involvement of the 2-norm terms, traditional SVM models may suffer from the lack of robustness to outliers and irrelevant features. Therefore, the LNSVM model is proposed by reformulating a typical nonparallel SVM model through the 1-norm regularization. By applying the exterior penalty theory, the proposed LNSVM model is converted to the dual exterior penalty problem, which is solved by the Newton–Armijo algorithm. The essential differences that distinguish the LNSVM model from other nonparallel SVM models are: (1) Different from typical nonparallel SVM models, which solve two quadratic programming (QP) problems, the proposed LNSVM model determines two nonparallel hyperplanes simultaneously by solving a single linear programming (LP) model; (2) The robustness performance of the proposed LNSVM model has been enhanced to tolerate noisy data through the involvement of 1-norm loss function, which can also eliminate redundant features by generating sparse solution during the training procedure. The performance of the proposed LNSVM model is tested through a comparison with state-of-art SVM-based classifiers using a synthetic dataset and 11 practical benchmark datasets. The experimental results show the superiority of the proposed LNSVM model, by achieving better classification performance regarding accuracy, sensitivity, specificity, and removing redundant features synchronously.