{\cal U}Boost: Boosting with the Universum

Abstract

It has been shown that the Universum data, which do not belong to either class of the classification problem of interest, may contain useful prior domain knowledge for training a classifier [1], [2]. In this work, we design a novel boosting algorithm that takes advantage of the available Universum data, hence the name UBoost. UBoost is a boosting implementation of Vapnik's alternative capacity concept to the large margin approach. In addition to the standard regularization term, UBoost also controls the learned model's capacity by maximizing the number of observed contradictions. Our experiments demonstrate that UBoost can deliver improved classification accuracy over standard boosting algorithms that use labeled data alone.