Selection methods for extended least squares support vector machines

Abstract

PurposeThe purpose of this paper is to present extended least squares support vector machines (LS‐SVM) where data selection methods are used to get sparse LS‐SVM solution, and to overview and compare the most important data selection approaches.Design/methodology/approachThe selection methods are compared based on their theoretical background and using extensive simulations.FindingsThe paper shows that partial reduction is an efficient way of getting a reduced complexity sparse LS‐SVM solution, while partial reduction exploits full knowledge contained in the whole training data set. It also shows that the reduction technique based on reduced row echelon form (RREF) of the kernel matrix is superior when compared to other data selection approaches.Research limitations/implicationsData selection for getting a sparse LS‐SVM solution can be done in the different representations of the training data: in the input space, in the intermediate feature space, and in the kernel space. Selection in the kernel space can be obtained by finding an approximate basis of the kernel matrix.Practical implicationsThe RREF‐based method is a data selection approach with a favorable property: there is a trade‐off tolerance parameter that can be used for balancing complexity and accuracy.Originality/valueThe paper gives contributions to the construction of high‐performance and moderate complexity LS‐SVMs.