Batch Support Vector Training Based on Exact Incremental Training

Abstract

In this paper we discuss training support vector machines (SVMs) repetitively solving a set of linear equations, which is an extension of accurate incremental training proposed by Cauwenberghs and Poggio. First, we select two data in different classes and determine the optimal separating hyperplane. Then, we divide the training data set into several chunk data sets and set the two data to the active set, which includes current and previous support vectors. For the combined set of the active set and a chunk data set, we detect the datum that maximally violates the Karush-Kuhn-Tacker (KKT) conditions, and modify the optimal hyperplane by solving a set of equations that constrain the margins of unbounded support vectors and the optimality of the bias term. We iterate this procedure until there are no violating data in any combined set. By the computer experiment using several benchmark data sets, we show that the training speed of the proposed method is comparable with the primal-dual interior-point method combined with the decomposition technique, usually with a smaller number of support vectors.