Actively learning from mistakes in class imbalance problems

Abstract

Active learning is innately very suitable for addressing class imbalance problems, as the actively selected training set is significantly less unbalanced than the real distribution. An improved active learning algorithm named CMIES (correcting mistakes in early stages) was proposed. It is based on the belief that a learner can learn more from correcting a mistake than ascertaining something uncertain. In early stages of active learning, the confidence of the classifier is very low. Even instance with the highest probability to be positive according to the current classifier is quite likely to be actually negative, especially with unbalanced data distribution. Correcting such a mistake is much more beneficial than learning from the most uncertain instance, resulting in extreme shrinkage of the version space and thus great improvement of the classification performance.