Ramp-loss nonparallel support vector regression: Robust, sparse and scalable approximation

Abstract

Although the twin support vector regression (TSVR) has been extensively studied and diverse variants are successfully developed, when it comes to outlier-involved training set, the regression model can be wrongly driven towards the outlier points, yielding extremely poor generalization performance. To overcome such shortcoming, a Ramp-loss nonparallel support vector regression (RL-NPSVR) is proposed in this work. By adopting Ramp ε-insensitive loss function and another Ramp-type linear loss function, RL-NPSVR can not only explicitly filter noise and outlier suppression but also have an excellent sparseness. The non- convexity of RL-NPSVR is solved by concave–convex programming (CCCP). Because a regularized term is added into each primal problem by rigidly following the structural risk minimization (SRM) principle, CCCP actually solves a series of reconstructed convex optimizations which have the same formulation of dual problem as the standard SVR, so that computing inverse matrix is avoided and SMO-type fast algorithm can be used to accelerate the training process. Numerical experiments on various datasets have verified the effectiveness of our proposed RL-NPSVR in terms of outlier sensitivity, generalization ability, sparseness and scalability.