Feature selection based on measurement of ability to classify subproblems

Abstract

Feature selection is important and necessary especially for processing large scale data. Existing feature selection methods generally compute a discriminant value with respect to class variable for a feature to indicate its classification ability. Such a scalar value can hardly reveal the multi-faceted classification abilities of a feature for the different subproblems in a classification task. In this paper, an effective way is proposed for feature selection based on measurement of ability to classify subproblems and discrimination structure complementarity of features. The classification abilities of a feature for different subproblems are calculated respectively. Hence for the feature, a discrimination structure vector representing its classification abilities for all subproblems can be obtained. In feature selection, the features, which can individually classify as many subproblems as possible, are firstly evaluated and selected. Subsequently, their complementary features are selectively chosen, which can complementarily classify the subproblems that the selected features cannot classify. Two algorithms are designed for progressively selecting features, by firstly eliminating irrelevant features and then abandoning redundant features based on discrimination structure complementarity. The proposed algorithms are compared with some related methods for feature selection on some open gene expression datasets and UCI datasets. Experimental results demonstrate the effectiveness of the proposed method.