Flexible-Fuzzy Mutual Information based Feature Selection on High Dimensional Data

Abstract

Due to advancement and development of the technologies, the growth of data becomes larger and needs high dimensional space. These high dimensional datasets are still challenging task in the domain of machine learning for the researchers. Feature selection (FS) is one the crucial technique which widely used for reducing the dimension of the data from the high dimensional space. Mutual Information (MI) is an important filter-based method used in trend for selecting most informative features. MI evaluates the mutual information shared between feature-class and feature-feature. In this paper, for selecting relevant features from high dimensional data a framework has been proposed which includes two feature selection methods. In first method, Fuzzy-Mutual Information based FS is used for calculating the mutual information among feature-class and feature-feature. Based on the calculated values most relevant features are selected. In Second method, Flexible Mutual Information based Feature Selection (FMIFS) is employed for selecting the predominant subset based on the mutual information. Both methods have been implemented to reduce features in two high dimensional datasets such as SRBCT and Leukaemia. For validating the proposed technique 10-fold cross validation is performed with Naive Bayes and SVM classifier. The experimental results were compared with standard feature selection methods.