A Biterm-based Dirichlet Process Topic Model for Short Texts

Abstract

Topic models are prevalent in many fields (e.g. context analysis), which are applied to discovering the latent topics. In document modeling, conventional topic models (e.g. latent Dirichlet allocation and its variants) do well for normal documents. However, the severe data sparsity problem makes the topic modeling in short texts difficult and unreliable. To tackle this problem, an effective approach (biterm topic model) has been proposed recently which learns topics by directly modeling the generation of word co-occurrence patterns at corpus-level rather than at document-level. But it requires human intervention for determining the number of topics. In this paper, we propose a Dirichlet process based on word cooccurrence to make topic mining from short texts more automatically. Meanwhile, we design a Markov chain Monte Carlo sampling scheme for posterior inference in our model which is an extension of the sampling algorithm based on Chinese restaurant process. Finally, we conduct experiments on real data. The results show that our method outperforms the baseline on quality of topic and perplexity and it is more flexible. KeywordsDirichlet Process; Clustering; Biterm; Short Texts; Topic Mining;