A new training method for support vector machines: Clustering k-NN support vector machines

Abstract

For training of support vector machines (SVMs) efficiently, a new training algorithm, clustering k-NN (k-nearest neighbor) support vector machines (CKSVMs) based on a Gaussian function regulated locally is proposed. In order to reflect degree of training data point as a support vector the Gaussian function is used with k-nearest neighbor (k-NN) method and Euclidean Distance measure. To add local control property to the training algorithm, a simple clustering scheme is implemented before Gaussian functions are constructed for each cluster. In addition, probabilistic SVM outputs are used for extension from binary classification to multi-class classification in pairwise approach. This training algorithm is applied to three commonly used classification problems. Experimental results show that the CKSVM has more classification accuracy than standard multi-class LS-SVM, FLS-SVM and LS-SVM with k-NN method which is proposed in our previous study. In addition to this, the training algorithm highly improved efficiency of the SVM classifier via simple algorithm.