Incremental batch learning with support vector machines

Abstract

Support vector machines (SVMs) have become a popular tool for learning with large amounts of high dimensional data. But it may sometimes be preferable to learn incrementally from previous SVM results, as SVMs which involve the solution of a quadratic programming problem suffer from the problem of large memory requirement and CPU time when trained in a batch mode on large data sets. The SVMs may be used in online learning setting. We proposed an approach for incremental batch learning with support vector machines for classification and regression, and define the normal solution of the incremental batch learning with SVMs as the solution minimizing a given positive-definite quadratic form in the coordinates of the difference vector between the normal vectors at the (k-1)-th and k-th incremental batch step and discuss the relation to standard SVM. It is shown that the concept that was learned at the last step does not change if the new data satisfy the separable condition. An empirical evidence is given to prove that this approach can effectively deal with changes in the target concept that are results of the incremental learning setting according to three evaluation criteria: stability, improvement and recoverability.