Hybrid feature selection using component co-occurrence based feature relevance measurement

Abstract

Feature selection, which is used to choose a subset of relevant features has attracted considerable attention in recent years. Typical feature selections include: traditional filters, mutual information based methods, clustering based methods and hybrid methods. As many feature selections cannot achieve the best features effectively and efficiently, a new hybrid feature selection method is proposed in this paper. First, the drawbacks of some existing feature relevance measurements are analyzed and a component co-occurrence based feature relevance measurement is proposed. Then, the implementation of the proposed feature selection is given: (1) the samples are preprocessed and two feature subsets are obtained by using two different optimal filters. (2) A feature weight based union operation is proposed to merge the obtained feature subsets. (3) As the hierarchical agglomerative clustering algorithm can produce clusters of high qualities without requiring the cluster number, it is applied to obtain the final feature subset by using a predetermined threshold. In the experiments, two typical classifiers: support vector machine and K-nearest neighbor are used on eight datasets (Lung-cancer, Breast-cancer-wisconsin, Arrhythmia, Arcene, CNAE-9, Madelon, Spambase and KDD-cup-1999), and the 10-cross validation is carried out when the F1 measurement is used. Experimental results show that the performance of the proposed feature relevance measurement is superior to those of traditional methods. In addition, the proposed feature selection outperforms many existing typical methods on classification accuracy and execution speed, illustrating its effectiveness in achieving the best features.