New margin-based subsampling iterative technique in modified random forests for classification

Abstract

Diversity within base classifiers has been recognized as an important characteristic of an ensemble classifier. Data and feature sampling are two popular methods of increasing such diversity. This is exemplified by Random Forests (RFs), known as a very effective classifier. However real-world data remain challenging due to several issues, such as multi-class imbalance, data redundancy, and class noise. Ensemble margin theory is a proven effective way to improve the performance of classification models. It can be used to detect the most important instances and thus help ensemble classifiers to avoid the negative effects of the class noise and class imbalance. To obtain accurate classification results, this paper proposes the Ensemble-Margin Based Random Forests (EMRFs) method, which combines RFs and a new subsampling iterative technique making use of computed ensemble margin values. As for comparative analysis, the learning techniques considered are: SVM, AdaBoost, RFs and the Subsample based Random Forests (SubRFs). The SubRFs uses Out-Of-Bag (OOB) estimation to optimize the training size. The effectiveness of EMRFs is demonstrated on both balanced and imbalanced datasets.