Ensembling perturbation-based oversamplers for imbalanced datasets

Abstract

Oversampling is effective to handle imbalanced data and is often combined with ensemble learning to boost its performance, for examples the synthetic minority oversampling technique (SMOTE) and its ensemble variants SMOTEBagging and SMOTEBoost. However, SMOTE-based methods often introduce unnecessary noises when classes are not well separated, which may impinge upon classifier training and hinder the classification performance. In addition, the aggressive generation of SMOTE-based methods would further complicate the learning process. Therefore, we propose a simple yet effective approach to generate diverse instances to balance the class distribution in a more conservative way in this work. Major differences among the proposed method and existing methods, which also highlight the originality of this work, are listed as follows. Firstly, unlike existing methods which utilize nearest neighbors-based methods to determine regions of interest, we analyze and compute the impact of class imbalance quantitatively for each minority instance in order to guide the data generation process. This ensures instances can be synthesized in a safe and informative region to avoid noise generation. Secondly, new instances are generated via perturbing a random subsets of input features of given seed instances. We show that this process generates instances that share the same asymptotic mean and covariance as given seed instances, thus classifiers with high generalization capability can be achieved. The generation process is repeated to train multiple base classifiers which are then fused via a majority voting to further improve its performance. Nonparametric Wilcoxon test and Friedman test confirm that the proposed method significantly outperforms most reference methods over thirty-five imbalanced datasets in terms of five metrics. The important finding of this work is that we show that ensembling simple perturbation-based oversamplers can yield better performance than many advanced ensemble methods for imbalanced datasets.