Generalized Pinball Loss SVMs

Abstract

Abstract Support Vector Machine (SVM) is a well-known efficient classification technique which appropriately trade-offs between the training error and the generalization ability of the classifier. But SVM’s are known to be sensitive towards noise and not stable with respect to re-sampling. Recently, Huang et al. (2014) proposed a novel SVM formulation with pinball loss function (Pin-SVM) model to handle noise sensitivity and instability to re-sampling. The Pin-SVM model in its pure form loses sparsity and therefore Huang et al. (2014) proposed a new pinball SVM model termed as ϵ-insensitive zone Pin-SVM. The ϵ-insensitive zone Pin-SVM formulation requires the value of ϵ to be pre-specified. Taking motivation from these developments, we propose a modified (ϵ1, ϵ2)-insensitive zone Pin-SVM ((ϵ1, ϵ2)-Mod-Pin-SVM) model in which the asymmetric spread of insensitive zone is optimized and therefore it is data-driven. An interesting feature of our proposed (ϵ1, ϵ2)-Mod-Pin-SVM model is that it appropriately trade-offs the generalization ability, the sparsity as well as the noise insensitivity. The (ϵ1, ϵ2)-Mod-Pin-SVM model is very general and subsumes a number of popular SVM type models. Its relation with Minimal Complexity Machine (MCM) introduced by Jayadeva (2015) is interesting and novel. The experimental results on several benchmark datasets prove the effectiveness of our proposed (ϵ1, ϵ2)-Mod-Pin-SVM formulation to that of other state-of-the-art classification algorithms.