A Geometric Approach to Leveraging Weak Learners

Abstract

AdaBoost is a popular and effective leveraging procedure for improving the hypotheses generated by weak learning algorithms. AdaBoost and many other leveraging algorithms can be viewed as performing a constrained gradient descent over a potential function. At each iteration the distribution over the sample given to the weak learner is the direction of steepest descent. We introduce a new leveraging algorithm based on a natural potential function. For this potential function, the direction of steepest descent can have negative components. Therefore we provide two transformations for obtaining suitable distributions from these directions of steepest descent. The resulting algorithms have bounds that are incomparable to AdaBoost’s, and their empirical performance is similar to AdaBoost’s.KeywordsPotential FunctionGradient DescentSteep DescentGeometric ApproachGeneralization ErrorThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.