Active Support Vector Learning with Statistical Queries

Abstract

Abstract The article describes an active learning strategy to solve the large quadratic programming (QP) problem of support vector machine (SVM) design in data mining applications. The learning strategy is motivated by the statistical query model. While most existing methods of active SVM learning query for points based on their proximity to the current separating hyperplane, the proposed method queries for a set of points according to a distribution as determined by the current separating hyperplane and a newly defined concept of an adaptive confidence factor. This enables the algorithm to have more robust and efficient learning capabilities. The confidence factor is estimated from local information using the k nearest neighbor principle. Effectiveness of the method is demonstrated on real life data sets both in terms of generalization performance and training time.