Ensemble feature selection using election methods and ranker clustering

Abstract

Feature selection (FS) has become a significant part of the data processing pipeline. Recently, ensemble FS has emerged as a new methodology that promises to improve FS robustness and performance. In this paper, we propose several ensemble FS methods built on voting aggregation schemes such as plurality vote, single transferable vote, Borda count, and novel weighted Borda count. Additionally, we present the new concept of clustering FS methods prior to building ensembles using a mean-shift clustering algorithm. The proposed methods are examined using three accuracy measures: the ability to correctly identify relevant features, FS stability, and influence on classification. The ensembles and clustered ensembles based on a weighted Borda count show very balanced performance, achieving quality results in all investigated measures and outperforming the other methods examined.