Learning Ensembles in the Presence of Imbalanced Classes

Abstract

Class imbalance occurs when data classes are not equally represented. Generally, it occurs when some classes represent rare events, while the other classes represent the counterpart of these events. Rare events, especially those that may have a negative impact, often require informed decision-making in a timely manner. However, class imbalance is known to induce a learning bias towards majority classes which implies a poor detection of minority classes. Thus, we propose a new ensemble method to handle class imbalance explicitly at training time. In contrast to existing ensemble methods for class imbalance that use either data driven or randomized approaches for their constructions, our method exploits both directions. On the one hand, ensemble members are built from randomized subsets of training data. On the other hand, we construct different scenarios of class imbalance for the unknown test data. An ensemble is built for each resulting scenario by combining random sampling with the estimation of the relative importance of specific loss functions. Final predictions are generated by a weighted average of each ensemble prediction. As opposed to existing methods, our approach does not try to fix imbalanced data sets. Instead, we show how imbalanced data sets can make classification easier, due to a limited range of true class frequencies. Our procedure promotes diversity among the ensemble members and is not sensitive to specific parameter settings. An experimental demonstration shows, that our new method outperforms or is on par with state-of-the-art ensembles and class imbalance techniques.