Ensemble Feature Ranking Methods for Data Intensive Computing Applications

Abstract

This paper presents a novel approach to feature ranking based on the notion of ensemble learning. The combination of multiple learners in the ensemble learning approach is supposedly superior than one single learner. Extending this concept to feature ranking, this paper investigates the ensemble feature ranking approach where multiple feature selection techniques are combined to give one ranking, which can be superior to that of the individual ranking techniques. The ensembles are assessed in terms of their robustness to class noise. This study shows that the robustness of an ensemble depends on the type of techniques in the ensemble. The threshold-based feature selection techniques perform better than the standard filters as components in terms of robustness to class noise. The poor stability of ensembles of standard filters is particularly evident. Also noticeable is the decrease in stability as standard filters are added to a stable ensemble.