A hybrid feature selection scheme for high-dimensional data

Abstract

There is a growing interest in developing feature subset selection schemes for high-dimensional datasets by filter, wrapper, embedded, and hybrid manners. In this paper, we propose a new hybrid (filter-wrapper) feature selection approach. At first, in the filter step, we rank input features according to their relevance with the class label. Afterwards, we apply different clustering methods for the classification of the selected features. We perform an inner and outer cluster ranking based on the primary feature ranking in the next step. Then, different search strategies are performed on the best cluster of features in the wrapper phase. Moreover, we add some of them to the feature set based on the classifiers (nearest neighbor, decision tree, support vector machine, and random forests) feedback. Then, the algorithm goes to the next cluster, and this process is continued till all clusters are met. Finally, we compare the results of the proposed method to the state-of-the-art schemes. Comparison results imply the superiority of the proposed method to the counterparts on eight high-dimensional datasets in terms of accuracy and computational complexity.