Update vs. upgrade: Modeling with indeterminate multi-class active learning

Abstract

This paper brings up a very important issue for active learning in practice. Traditional active learning mechanism is based on the assumption that the number of classes happens to be known in advance, and thus selective sampling is confined to the determinate model. However, as is the case for many applications, the model class is usually indeterminate and there is every chance that the hypothesis itself is inappropriate. To address this problem, we propose a novel indeterminate multi-class active learning algorithm, which comprehensively evaluates the instance based on both the value in refining the existing model and the potential in triggering model rectification. In this way, balance is effectively achieved between model update and model upgrade. Advantage of the proposed algorithm is demonstrated by experiments of classification tasks on both synthetic and real-world dataset.