Feature selection and threshold method based on fuzzy joint mutual information

Abstract

Improving classification performance is one of the main challenges in a variety of real-world applications. Unfortunately, classification models are sensitive to undesirable features of data such as redundant and irrelevant features. Feature selection (FS) is a powerful solution to address the negative effect of these features. Among various methods, Feature selection based on mutual information (MI) is an effective method to select the significant features and deny the undesirable ones. Although most of the existing methods can estimate the feature relevancy efficiently, they may find difficulty to estimate the feature redundancy well. This is due to the individual estimation between the candidate feature and the features of pre-selected subset. To address this limitation, this paper introduces a novel feature selection method, called Fuzzy Joint Mutual Information (FJMI). The proposed method also overcomes common limitations as dealing with continuous feature without information loss and returns the best feature subset automatically without a user-defined threshold. To evaluate the effectiveness of the proposed method, FJMI is compared with six conventional and state-of-the-art feature selection methods. The experimental results on 16 benchmark datasets, with moderate size, show a promising improvement by the proposed method in terms of classification performance, feature selection stability, and number of selected features.