Data-driven global-ranking local feature selection methods for text categorization

Abstract

Bag-of-words is the most used representation method in text categorization. It represents each document as a feature vector where each vector position represents a word. Since all words in the database are considered features, the feature vector can reach tens of thousands of features. Therefore, text categorization relies on feature selection to eliminate meaningless data and to reduce the execution time. In this paper, we propose two filtering methods for feature selection in text categorization, namely: Maximum f Features per Document (MFD), and Maximum f Features per Document – Reduced (MFDR). Both algorithms determine the number of selected features f in a data-driven way using a global-ranking Feature Evaluation Function (FEF). The MFD method analyzes all documents to ensure that each document in the training set is represented in the final feature vector. Whereas MFDR analyzes only the documents with high FEF valued features to select less features therefore avoiding unnecessary ones. The experimental study evaluated the effectiveness of the proposed methods on four text categorization databases (20 Newsgroup, Reuters, WebKB and TDT2) and three FEFs using the Naïve Bayes classifier. The proposed methods present better or equivalent results when compared with the ALOFT method, in all cases, and Variable Ranking, in more than 93% of the cases.