A new approach for detecting the best feature set

Abstract

The efficiency and accuracy of training neural-net classifiers is typically improved by eliminating of features that are redundant and irrelevant. The objective is to reduce the size of the input feature set and at the same time retain as much as possible of the class discriminatory information. Such an input features set after reducing features from the original input features, which is called best feature space, would offer a reduction of both cost and complexity of feature collection as well as improve the efficiency and accuracy of the resultant classifier. In this paper, we develop and evaluate two composite feature selection algorithms: mutual information feature space forward selection (MIFSFS) and mutual information feature space backward selection (MTFSBS). These two algorithms use mutual information (MI), for both continuous-valued and discrete-valued features, between the feature space and the class in order to find the best feature space from the original input features. The most important output from the new algorithms is that we not only can identity the irrelevant features, but also identify the redundant features, which cannot be identified by the common feature selection algorithms (for example artificial neural networks, ANN). Empirical studies of both realistic new and previously published classification problems indicate that the proposed algorithms are robust, stable and efficient. One of real application is drug discovery. Knowledge discovery from gene expression data is a highly research topical area for the drug discovery. Finding a number of genes out of the thousands caused a certain types of diseases is a significant contribution for the preventing and fighting diseases. Here we use the myeloma disease as an example to demonstrate how to identify the genes caused the myeloma disease.