Chinese text classification for small sample set

Abstract

Text classification is one of the most important topics in the fields of Internet information management and natural language processing. Machine learning based text classification methods are currently most popular ones with better performance than rule based ones. But they always need lots of training samples, which not only brings heavy work for previous manual classification, but also puts forward a higher request for storage and computing resources during the computer post-processing. Naïve Bayes algorithm is one of the most effective methods for text classification with the same problem. Only in the large training sample set can it get a more accurate result. This paper mainly studies Naïve Bayes classification algorithm for Chinese text based on Poisson distribution model and feature selection. The experimental results have shown that this method keeps high classification accuracy even in a small sample set.