Construct a robust least squares support vector machine based on L[formula omitted]-norm and L[formula omitted]-norm

Abstract

Despite some Lp-norm LSSVMs own feature selection and prediction ability, they still suffer from two common issues. (i) They always ignore edge points because the L2-norm metric is used to measure the classification error of the training samples. The edge points are important in some practical application or on the datasets that are non-independent and identically distributed (non-i.i.d). (ii) They spend higher computational time and storage space for the large scale datasets. In order to solve the above two shortcomings while retaining the feature selection ability, we adopt L∞-norm to measure the classification error of training samples and still use Lp-norm (0<p<1) to measure the maximum margin between two parallel support planes, then obtain a novel LSSVM classifier, denoted as Lp-L∞-LSSVM. Our Lp-L∞-LSSVM owns three advantages: (1) L∞-norm on empirical risk ensures the effective recognition of edge points, thereby improving the robustness and generalization ability of the classifier. (2) Lp-norm on structural risk possess feature selection ability, whether for the linear or non-linear separable case and is suitable for the small samples size (SSS) problem. (3) Inspired by the sequential minimal optimization (SMO) algorithm, we designed an iterative heuristic algorithm by breaking the large quadratic programming problem (QPP) into a series of smallest possible QPPs, which can avoid high time-consuming. This algorithm not only ensures the convergence of optimum solution but also consumes lower computational time and storage space for large scale datasets. Finally, extensive numerical experiments once again verify the above opinions and show the outstanding classification performance and feature selection ability simultaneously.