A mutual information and information entropy pair based feature selection method in text classification

Abstract

Text classification is an important research field of data mining topics. This article brings a mutual information and information entropy pair based feature selection method (MIIEP_FS) based on the theory of information entropy and information entropy pair concept. This method measure the classification effect using feature by mutual information method and show the difference extent between the features being selected and the ones selected by information entropy. The experimental results show that the MIIEP_FS method proposed is more effective than MI and CHI methods. Macro F1 degrees of different kinds of machine learning algorithms: Naive Bayes and KNN method are higher by MIIEP_FS method, sometimes even more than the ones of support vector machines.