Sparse Lq-norm least squares support vector machine with feature selection

Abstract

Least squares support vector machine (LS-SVM) is a popular hyperplane-based classifier and has attracted many attentions. However, it may suffer from singularity or ill-condition issue for the small sample size (SSS) problem where the sample size is much smaller than the number of features of a data set. Feature selection is an effective way to solve this problem. Motivated by this, in the paper, we propose a sparse Lq-norm least squares support vector machine (Lq-norm LS-SVM) with 0 < q < 1, where feature selection and prediction are performed simultaneously. Different from traditional LS-SVM, our Lq-norm LS-SVM minimizes the Lq-norm of weight and releases the least squares problem in primal space, resulting in that feature selection can be achieved effectively and small enough number of features can be selected by adjusting the parameters. Furthermore, our Lq-norm LS-SVM can be solved by an efficient iterative algorithm, which is proved to be convergent to a global optimal solution under some assumptions on the sparsity. The effectiveness of the proposed Lq-norm LS-SVM is validated via theoretical analysis as well as some illustrative numerical experiments.