Dual active set method for support vector machines under multi-constraint activation

Abstract

Active set methods are competitive alternatives of SVM optimizers to working set or decomposition techniques for moderate scale problems. While there are many works on applying the primal active set methods to SVM, few study the dual active set methods. Comparing with the primal active set method, the dual active set method is more efficient and numerically stable due to using an unconstrained minimum as a feasible start point and having choices of which constraints to add to the active set. However, since the conventional dual active set method can only add one violated constraint in one step, this paper proposes a novel dual active set method, which allows to add multiple constraints and accelerates the convergence. Moreover, the new method does not require to compute the matrix inversion when applied to SVM, and essentially reduces the training time. Experiment results on several benchmark data sets validate that the proposed method is indeed more efficient than the primal and the conventional dual active set methods.