Regularized Least Squares Twin SVM for Multiclass Classification

Abstract

Support vector machines (SVMs) have been successfully used in classification and regression problems. However, SVM suffers from high computational complexity which limits its applicability. Twin SVM (TWSVM) reduces the complexity of SVM, however, it still suffers due to the optimization of quadratic programming problems (QPPs). To make TWSVM model more efficient, least squares twin SVM (LSTSVM) solves a pair of linear equations for generating the optimal hyperplanes. LSTSVM is useful for solving multiclass classification problems due to less computational cost and good generalization performance. Multiclass classification problems require high computational cost and thus need efficient algorithms to reduce the training time. A new regularization based method for multiclass classification problems for different multiclass classification methods, namely “One-versus-All (OVA)”, “One-versus One (OVO)”, “All-versus-One (AVO)” and “Direct Acyclic Graph (DAG)” is proposed in this work. It is named as multiclass regularized least squares twin support vector machine (MRLSTSVM). The standard LSTSVM algorithm gives emphasis on reducing the empirical risk only, however, the proposed MRLSTSVM implements structural risk minimization (SRM) principle to reduce overfitting. Our regularization based approach leads to positive definite matrices in the formulation of MRLSTSVM. For each classifier, the computational complexity is analyzed and discussed their advantages and disadvantages. The performance analysis is tested by conducting experiments on a wide range of benchmark UCI datasets. In comparison to other baseline multiclass classifiers in terms of accuracy, the proposed approach MRLSTSVM (OVO) shows better generalization performance.