A Feature Metric Algorithm Combining the Wasserstein Distance and Mutual Information

Abstract

Mutual information based feature selection algorithms measure feature relevance by comparing mutual information between features and class labels, however those features do not necessarily lead to good feature selection and classification accuracy. For each feature dimension, the Wasserstein distance can be used to measure the difference of distribution of categories, which provides a more powerful feature relevance metric method than mutual information. Feature redundancy can still be measured by mutual information. Thus, an optimization objective function combing the measures of Wasserstein distance and mutual information is proposed, which can obtain a smaller feature set with strong feature relevance and less feature redundancy. The effectiveness of the proposed metric method is verified by conducting tests on the UCI datasets. Compared with other common feature selection method such as MRMR, CIFE, MIFS, and MIM, our method reduces the number of selected features by almost 50% to 80% but gets higher accuracy.