A Fuzzy Mutual Information-based Feature Selection Method for Classification

Abstract

In this paper, we present a feature selection method called Fuzzy Mutual Information-based Feature Selection with Non-Dominated solution (FMIFS-ND) using a fuzzy mutual information measure which selects features based on feature-class fuzzy mutual information and feature-feature fuzzy mutual information. To evaluate classification accuracy of the proposed method, a modification of the k-nearest neighbor (KNN) classifier is also presented in this paper to classify instances based on the distance or similarity between individual features. The performance of both methods is evaluated on multiple UCI datasets by using four classifiers. We compare the accuracy of our feature selection method with existing feature selection methods and validate accuracy of the proposed classifier with decision trees, random forests, naive Bayes, KNN and support vector machines (SVM). Experimental results show that the feature selection method gives high classification accuracy in most high dimensional datasets as well as the accuracy of proposed classifiers outperforms the traditional KNN classifier.