A stable variant of linex loss SVM for handling noise with reduced hyperparameters

Abstract

Support Vector Machine (SVM) primarily uses the hinge loss function with maximum margin. The boundary instances determine the separating hyperplanes. However, in real-world scenarios, noisy instances around the decision boundary can degrade the performance of hinge SVM. To address this issue, researchers introduced the pinball SVM, which uses all the instances to determine the separating hyperplane and has a discontinuous piecewise linear loss function. A more recent variant, the linex loss SVM with NAG, employs a combination of linear and exponential loss functions, resulting in an asymmetric and continuous loss function. Due to the employment of Nesterov accelerated gradient (NAG), the older linex SVM formulation was unstable. This study presents a novel noise-robust variant of linex SVM, termed linexSVM, which is stable in performance compared to the linex SVM with NAG. Additionally, the proposed linexSVM has fewer hyperparameters in the dual formulation than other noise-robust SVM variants, such as pinball SVM, ϵ-insensitive pinball SVM, and ϵ1,ϵ2 pinball SVM. We evaluated the performance of the proposed linexSVM, hinge SVM, and pinball SVM using the UCI dataset repository. The experimental results indicate that the proposed linexSVM outperforms hinge and pinball loss SVMs on datasets with and without noise.