A new document representation based on global policy for supervised term weighting schemes in text categorization.

Abstract

There are two main factors involved in documents classification, document representation method and classification algorithm. In this study, we focus on document representation method and demonstrate that the choice of representation methods has impacts on quality of classification results. We propose a document representation strategy for supervised text classification named document representation based on global policy (DRGP), which can obtain an appropriate document representation according to the distribution of terms. The main idea of DRGP is to construct the optimization function through the importance of terms to different categories. In the experiments, we investigate the effects of DRGP on the 20 Newsgroups, Reuters21578 datasets, and using the SVM as classifier. The results show that the DRGP outperforms other text representation strategy schemes, such as Document Max, Document Two Max and global policy.