A regression approach to LS-SVM and sparse realization based on fast subset selection

Abstract

The Least Squares Support Vector Machine (LS-SVM) is a modified SVM with a ridge regression cost function and equality constraints. It has been successfully applied in many classification problems. But, the common issue for LS-SVM is that it lacks sparseness, which is a serious drawback in its applications. To tackle this problem, a fast approach is proposed in this paper for developing sparse LS-SVM. First, a new regression solution is proposed for the LS-SVM which optimizes the same objective function for the conventional solution. Based on this, a new subset selection method is then adopted to realize the sparse approximation. Simulation results on different benchmark datasets i.e. Checkerboard, two Gaussian datasets, show that the proposed solution can achieve better objective value than conventional LS-SVM, and the proposed approach can achieve a more sparse LS-SVM than the conventional LS-SVM while provide comparable predictive classification accuracy. Additionally, the computational complexity is significantly decreased.