MIC_FS : A novel model for feature selection by mutual information guided by clustering

Abstract

Feature selection is an important problem for pattern classification systems. There are many methods for feature selection available, in which the feature selection method based on mutual information proposed by authors of Ref.[13] is one of the more effective approaches. However, it is often difficult to compute the mutual information for the continuous data whether using discretization strategy or directly employing density estimation method(e.g., Parzen windows). So, in this paper, we propose a novel model for feature selection by mutual information guided by clustering(MIC_FS). According to MIC_FS, a novel algorithm for feature selection(AMICFS) is introduced. In newly developed algorithm AMICFS, the mutual information between two features can be directly induced by the unsupervised fuzzy c-means clustering, and meanwhile the significance of features and the relevancy between features are simultaneously considered, hence a more effectively ranked feature list can be efficiently obtained in most cases. The experiments on 6 real-life benchmark datasets show that AMICFS is better or comparable as compared to Fisher Score.