A Robust Least Squares Support Vector Machine Based on L∞-norm

Abstract

A family of norm-based least squares support vector machines (LSSVMs) is the effective statistical learning tool and has attracted considerable attentions. However, there are two critical problems: (1) LSSVM is ill-conditioned or singular when the sample size is much less than feature number and thus causes over-fitting for small sample size (SSS) problem, while other norm-based LSSVMs own slower training speed and cannot deal with large scale data. (2) Norm-based LSSVMs pay less attention to the edge points that are important for learning the final classifier. To overcome the above drawbacks, by replacing $$ L_{2}\text{-}norm $$ with $$ L_{\infty }\text{-}norm $$ in empirical loss, we construct a robust classifier, named $$ L_{\infty }\text{-}LSSVM $$ in short. After adjustment, $$ L_{\infty }\text{-}LSSVM $$ can detect edge points effectively and enhances the capability of the robustness and the generalization. Also, inspired by the idea of sequential minimal optimization (SMO), we design a novel SMO-typed iterative algorithm. The new algorithm not only guarantees the convergence of optimum solution in theory, but also owns the lower computational time and storage space when data size is large. Finally, extensive numerical experiments validate the above opinions again on four groups of artificial data, Non-i.i.d data: fault detection of railway turnout, four SSS datasets, two types of massive datasets and six benchmark datasets.