Query by Committee, Linear Separation and Random Walks

Abstract

Recent works have shown the advantage of using Active Learning methods, such as the Query by Committee (QBC) algorithm, to various learning problems. This class of Algorithms requires an oracle with the ability to randomly select a consistent hypothesis according to some predefined distribution. When trying to implement such an oracle, for the linear separators family of hypotheses, various problems should be solved. The major problem is time-complexity, where the straight-forward Monte Carlo method takes exponential time. In this paper we address some of those problems and show how to convert them to the problems of sampling from convex bodies or approximating the volume of such bodies. We show that recent algorithms for approximating the volume of convex bodies and approximately uniformly sampling from convex bodies using random walks, can be used to solve this problem, and yield an efficient implementation for the QBC algorithm. This solution suggests a connection between random walks and certain properties known in machine learning such as ∈-net and support vector machines. Working out this connection is left for future work.