A Novel Simulated Annealing-Based Learning Algorithm for Training Support Vector Machines

Abstract

A theoretical advantage of large margin classifiers such as support vector machines (SVM) concerns the empirical and structural risk minimization which balances the complexity of the model against its success at fitting the training data. Metaheuristics have been used to work with SVMs in order to select features, tune hypeparameters or even achieve a reduced-set of support vectors. In spite of such tasks being interesting, metaheuristics such as simulated annealing (SA) do not play an important role in the process of solving the quadratic optimization problem, which arises from support vector machines. To do so, well-known methods such as sequential minimal optimization, kernel adatron or even classical mathematical methods have been used with this goal. In this paper, we propose to use simulated annealing in order to solve such a quadratic optimization problem. Our proposal is interesting when compared with those aforementioned methods, since it is simple and achieved similar (or even higher) accuracy and high sparseness in the solution.