Fast pruning algorithm for multi-output LS-SVM and its application in chemical pattern classification

Abstract

Classification is an important problem in chemometrics field, and least squares support vector machine (LS-SVM), which is a simplified version of standard support vector machine (SVM), is a powerful tool for classification problems. However LS-SVM loses the sparseness of ordinary SVM, which would decrease its classification speed and limit its application to chemical data mining. To keep sparseness for LS-SVM, several pruning algorithms were proposed. But few of the existent pruning algorithms can be directly applied to an LS-SVM with multiple outputs. In this paper, a new fast pruning algorithm is proposed for a multi-output LS-SVM. The support vector (SV) selection is done by linear correlation analysis based on the principal component analysis (PCA) or the classification correlative component analysis (CCCA) of kernel matrix, then the information contained in non-SV data points is transferred to SV data points, so sparseness is imposed to LS-SVM. The tests on several real-life classification problems show that the proposed sparse LS-SVM not only has an equivalently good classification performance to the standard LS-SVM but also keeps a high sparseness in model structure. How the information transferring process ensures the performance stability during pruning and the difference of SV distribution between the LS-SVM pruned by Suykens' pruning algorithm and the proposed sparse LS-SVMs are also illustrated.