Universum parametric $$\nu $$-support vector regression for binary classification problems with its applications

Abstract

AbstractUniversum data sets, a collection of data sets that do not belong to any specific class in a classification problem, give previous information about data in the mathematical problem under consideration to enhance the classifiers’ generalization performance. Recently, some researchers have integrated Universum data into the existing models to propose new models which result in improved classification performance. Inspired by these Universum models, an efficient parametric$$ \nu $$ν-support vector regression with Universum data ($$ {\mathfrak {U}} $$UPar-$$ \nu $$ν-SVR) is proposed in this work. This method, which finds two non-parallel hyperplanes by solving one optimization problem and considers heteroscedastic noise, overcomes some common disadvantages of the previous methods. The$$ {\mathfrak {U}} $$UPar-$$ \nu $$ν-SVR includes unlabeled samples that don’t belong to any class in the training process, which results in a quadratic programming problem. Two approaches are proposed to solve this problem. The first approach derives the dual formulation using the Lagrangian function and KKT conditions. Furthermore, a least squares parametric$$ \nu $$ν-support vector regression with Universum data (named LS-$$ {\mathfrak {U}} $$UPar-$$ \nu $$ν-SVR) is suggested to further increase the generalization performance. The LS-$$ {\mathfrak {U}} $$UPar-$$ \nu $$ν-SVR solves a single system of linear equations, instead of addressing a quadratic programming problem in the dual formulation. Numerical experiments on artificial, UCI, credit card, NDC, and handwritten digit recognition data sets show that the suggested Universum model and its solving methodologies improve prediction accuracy.