Research on Automatic Recognition of New Words on Weibo

Abstract

To effectively capture emerging vocabulary on Weibo, this article proposes a new Weibo new word recognition strategy that combines Weibo data and support vector machine. Firstly, select positive and negative example sentences from Weibo corpus and trained corpus with part of speech tagging. Then, the lexical features in these sentences are transformed into vectors, and then trained using support vector machines to obtain classification support vectors for Weibo new words. Finally, input the vectorized features into the already trained support vector machine classifier to identify new Weibo words. Based on the experimental results, the system found the optimal feature combination.