A robust model for intelligent text classification

Abstract

Methods for taking into account linguistic content into text retrieval are receiving growing attention. Text categorization is an interesting area for evaluating and quantifying the impact of linguistic information. Work on text retrieval through the Internet suggests that embedding linguistic information at a suitable level within traditional quantitative approaches is the crucial issue able to bring the experimental stage to operational results. This kind of representational problem is studied in this paper where traditional methods for statistical text categorization are augmented via a systematic use of linguistic information. The addition of NLP capabilities also suggested a different application of existing methods in revised forms. This paper presents an extension of the Rocchio formula as a feature weighting and selection model used as a basis for multilingual information extraction. It allows an effective exploitation of the available linguistic information that better emphasizes the latter with significant data compression and accuracy. The results is an original statistical classifier fed with linguistic features and characterized by the novel feature selection and weighting model. It outperforms existing systems while keeping most of their interesting properties. Extensive tests of the model suggest its application as a viable and robust tool for large scale text classification and filtering, as well as a basic module for more complex scenarios.