Mutual Information Based Feature Selection Research Articles

Colon cancer prediction with genetics profiles using evolutionary techniques

Microarray data provides information on gene expression levels of thousands of genes in a cell in a single experiment. DNA microarray is a powerful tool in the diagnosis of cancer. Numerous efforts have been made to use gene expression profiles to improve precision of tumor classification. In this study comparison between class prediction accuracy of two different classifiers, Genetic Programming and Genetically Evolved Decision Trees, was carried out using the best 10 and best 20 genes ranked by the t-statistic and mutual information. Genetic Programming proved out to be the better classifier for this dataset based on area under the receiver operating characteristic curve (AUC) and total accuracy using mutual information based feature selection. We conclude that Genetic Programming together with mutual information based feature selection is the most efficient alternative to the existing colon cancer prediction techniques.

Comparing two genetic overproduce-and-choose strategies for fuzzy rule-based multiclassification systems generated by bagging and mutual information-based feature selection

In [14] we proposed a scheme to generate fuzzy rule-based multiclassification systems by means of bagging, mutual information-based feature selection, and a multicriteria genetic algorithm for static component classifier selection guided by the ensemble training error. In the current contribution we extend the latter component by making use of the bagging approach's capability to evaluate the accuracy of the classifier ensemble using the out-of-bag estimates. An exhaustive study is developed on the potential of the two multicriteria genetic algorithms respectively considering the classical training error and the out-of-bag error fitness functions to design a final multiclassifier with an appropriate accuracy-complexity trade-off. Several parameter settings for the global approach are tested when applied to nine popular UCI datasets with different dimensionality.

A study of mutual information based feature selection for case based reasoning in software cost estimation

Software cost estimation is one of the most crucial activities in software development process. In the past decades, many methods have been proposed for cost estimation. Case based reasoning (CBR) is one of these techniques. Feature selection is an important preprocessing stage of case based reasoning. Most existing feature selection methods of case based reasoning are ‘wrappers’ which can usually yield high fitting accuracy at the cost of high computational complexity and low explanation of the selected features. In our study, the mutual information based feature selection (MICBR) is proposed. This approach hybrids both ‘wrapper’ and ‘filter’ mechanism which is another kind of feature selector with much lower complexity than wrappers, and the features selected by filters are likely to be generalized to other conditions. The MICBR is then compared with popular feature selectors and the published works. The results show that the MICBR is an effective feature selector for case based reasoning by overcoming some of the limitations and computational complexities of other feature selection techniques in the field.

Mutual information-based feature selection and partition design in fuzzy rule-based classifiers from vague data

Algorithms for preprocessing databases with incomplete and imprecise data are seldom studied. For the most part, we lack numerical tools to quantify the mutual information between fuzzy random variables. Therefore, these algorithms (discretization, instance selection, feature selection, etc.) have to use crisp estimations of the interdependency between continuous variables, whose application to vague datasets is arguable. In particular, when we select features for being used in fuzzy rule-based classifiers, we often use a mutual information-based ranking of the relevance of inputs. But, either with crisp or fuzzy data, fuzzy rule-based systems route the input through a fuzzification interface. The fuzzification process may alter this ranking, as the partition of the input data does not need to be optimal. In our opinion, to discover the most important variables for a fuzzy rule-based system, we want to compute the mutual information between the fuzzified variables, and we should not assume that the ranking between the crisp variables is the best one. In this paper we address these problems, and propose an extended definition of the mutual information between two fuzzified continuous variables. We also introduce a numerical algorithm for estimating the mutual information from a sample of vague data. We will show that this estimation can be included in a feature selection algorithm, and also that, in combination with a genetic optimization, the same definition can be used to obtain the most informative fuzzy partition for the data. Both applications will be exemplified with the help of some benchmark problems.

The application of mutual information-based feature selection and fuzzy LS-SVM-based classifier in motion classification

This paper presents an effective mutual information-based feature selection approach for EMG-based motion classification task. The wavelet packet transform (WPT) is exploited to decompose the four-class motion EMG signals to the successive and non-overlapped sub-bands. The energy characteristic of each sub-band is adopted to construct the initial full feature set. For reducing the computation complexity, mutual information (MI) theory is utilized to get the reduction feature set without compromising classification accuracy. Compared with the extensively used feature reduction methods such as principal component analysis (PCA), sequential forward selection (SFS) and backward elimination (BE) etc., the comparison experiments demonstrate its superiority in terms of time-consuming and classification accuracy. The proposed strategy of feature extraction and reduction is a kind of filter-based algorithms which is independent of the classifier design. Considering the classification performance will vary with the different classifiers, we make the comparison between the fuzzy least squares support vector machines (LS-SVMs) and the conventional widely used neural network classifier. In the further study, our experiments prove that the combination of MI-based feature selection and SVM techniques outperforms other commonly used combination, for example, the PCA and NN. The experiment results show that the diverse motions can be identified with high accuracy by the combination of MI-based feature selection and SVM techniques. Compared with the combination of PCA-based feature selection and the classical Neural Network classifier, superior performance of the proposed classification scheme illustrates the potential of the SVM techniques combined with WPT and MI in EMG motion classification.

Mutual Information-Based Feature Selection in Studying Perturbation of Dendritic Structure Caused by TSC2 Inactivation

In this study, the effect of protein Tuberous sclerosis 2 (TSC2) on the dendritic spine density and length was demonstrated by using TSC2-RNAinactivation. In addition, the role of rapamycin, an antagonist of the molecular target of rapamycin, in the morphological changes of spine caused by TSC2 silencing was investigated. The features were extracted from highresolution three-dimensional image stacks collected by two-photon laser scanning microscopy of green fluorescing pyramidal cells expressing TSC2-RNA interference (RNAi), or TSC2-RNAi and rapamycin treatment in rat hippocampal slice cultures. We proposed to apply the lognormal distribution method for feature extraction. The extracted features of three cases under investigation, namely, (1) green-fluorescent protein GFP vs TSC2-RNAi, (2) GFP vs TSC2-RNAi and rapamycin, and (3) TSC2-RNAi vs TSC2-RNAi and rapamycin, were analyzed by mutual information-based feature selection and evaluated by three classifiers, K-nearest neighbor, Perceptron, and two-layer neural networks. The results showed that both the spine density and length have significant morphological changes after TSC2-RNAi treatment. However, rapamycin treatment could reverse the effect of TSC2-RNAi on spine length but not on spine density. These results are consistent with the results reported in the scientific literature. Finally, we explored the application of pattern recognition method in a small sample with richer feature properties, namely bootstrap mutual information estimation and a mutual information- based feature selection method.

Applying a Mutual Information Theory Based Feature Selection Method to a Classifier

OBJECTIVE: The purpose of this study was to explore the usability of a feature selection method based on the mutual information theory to increase predictive performance of a classifier in data mining. METHODS: The HIV Cost and Services Utilization Study(HCSUS) dataset was used to apply the feature selection method to a classifier. Its contribution to increasing the predictive performance of the classifier was evaluated by comparing the Naive Bayes(NB) and the Logistic Regression(LG) models using different variables. The infrequent office visit representing limited health service utilization was selected as an outcome variable. HUGIN Researcher(TM) 6.3 was used to train and test the NB models and SAS(R) 8.0 was used for the LG modeling. RESULTS: Higher AUC in the NB model was obtained using the variables selected by the mutual information based feature selection method(AUC=.639, CI=.611, .660); lower AUC using the variables defined by a previous study(AUC=.599, CI=.570, .620). There was no difference between the LG models with different variables. CONCLUSION: This study demonstrated the mutual information method may be useful in identifying relevant predictors as the feature selection method, which can contribute to an increase in the predictive performance of a classifier.

Estimating Optimal Feature Subsets Using Efficient Estimation of High-Dimensional Mutual Information

A novel feature selection method using the concept of mutual information (MI) is proposed in this paper. In all MI based feature selection methods, effective and efficient estimation of high-dimensional MI is crucial. In this paper, a pruned Parzen window estimator and the quadratic mutual information (QMI) are combined to address this problem. The results show that the proposed approach can estimate the MI in an effective and efficient way. With this contribution, a novel feature selection method is developed to identify the salient features one by one. Also, the appropriate feature subsets for classification can be reliably estimated. The proposed methodology is thoroughly tested in four different classification applications in which the number of features ranged from less than 10 to over 15,000. The presented results are very promising and corroborate the contribution of the proposed feature selection methodology.

Effective feature selection scheme using mutual information

This article proposes a novel mutual information-based feature selection scheme. In this scheme, the mutual information is estimated directly in an effective way even when one is handling a relative small data set. At the same time, the computation efficiency of the mutual information estimation is improved by proposing a supervised data compression algorithm. With these contributions, the proposed feature selection scheme is able to effectively identify the salience features. The proposed methodology is compared with the related study through applying to different classification problems in which the number of features ranged from less than 10 to over 12,600. The presented results are very promising and corroborate the contributions of the proposed methodology.

Supervised mutual-information based feature selection for motor unit action potential classification

A new supervised mutual information-based feature selection method is presented. Using real motor unit action potential (MUAP) data from 10 EMG signals, the performances of 32 time-sample feature sets, feature subsets selected using first- and second-order mutual information and features obtained using linear discriminant analysis (LDA) and principal component analysis (PCA) were evaluated using a minimum Euclidean distance (MED) classifier. The evaluation showed that by using only 20 first-order features or only 15 second-order features mean error rates and error rate variations equivalent to using all 32 samples or LDA or PCA could be obtained. The computational cost of first-order feature selection was considerably less than LDA, PCA and second-order feature selection. The performance of first-order features was further evaluated using a more robust classifier. Unlike the MED classifier, the robust classifier only assigned a candidate MUAP if the assignment was sufficiently certain. For the robust classifier the average error rates using 20 features were similar to using the full feature set, yet higher assignment rates were obtained. Results from both evaluations suggest that the sets of first-order features were an efficient representation of lower dimension, which provided high accuracy classification with reduced computational requirements.

Patterns of vocalization and gaze in mother-infant play: Reliability within and among dyads

The common spatial pattern (CSP) algorithm is effective in decoding the spatial patterns of the corresponding neuronal activities from electroencephalogram (EEG) signal patterns in brain–computer interfaces (BCIs). However, its effectiveness depends on the subject-specific time segment relative to the visual cue and on the temporal frequency band that is often selected manually or heuristically. This paper presents a novel statistical method to automatically select the optimal subject-specific time segment and temporal frequency band based on the mutual information between the spatial–temporal patterns from the EEG signals and the corresponding neuronal activities. The proposed method comprises four progressive stages: multi-time segment and temporal frequency band-pass filtering, CSP spatial filtering, mutual information-based feature selection and naïve Bayesian classification. The proposed mutual information-based selection of optimal spatial–temporal patterns and its one-versus-rest multi-class extension were evaluated on single-trial EEG from the BCI Competition IV Datasets IIb and IIa respectively. The results showed that the proposed method yielded relatively better session-to-session classification results compared against the best submission.