Improving Text Classification Performance by Heuristics-Guided Exploration of Derived Features

Abstract

One important problem facing text classifiers is the vast amount of features, many of which may not be relevant, that one can use in the classification process. Sleeping-Experts is one of those classifiers which can effectively deal with large number of irrelevant attributes. It is an online multiplicative weight updating algorithm similar to the Winnow algorithm. In its original design, it provided context-sensitive text classification by including ` sparse phrases' in the feature set. Although Sleeping-Experts has the capability to handle a large number of features, the combinatorial explosion of derived features like `sparse phrases' still leads to substantial ineffectiveness and inefficiency when they are exhaustively examined. In this paper we proposed a heuristics-guided approach to the exploration of derived features in relation to the Sleeping-Experts algorithm. Our experiment results show the use of some simple heuristics can improve both the efficiency and effectiveness of text classification based on such model.