Abstract
In recent years, short texts have become a kind of prevalent text on the internet. Due to the short length of each text, conventional topic models for short texts suffer from the sparsity of word co-occurrence information. Researchers have proposed different kinds of customized topic models for short texts by providing additional word co-occurrence information. However, these models cannot incorporate sufficient semantic word co-occurrence information and may bring additional noisy information. To address these issues, we propose a self-aggregated topic model incorporating document embeddings. Aggregating short texts into long documents according to document embeddings can provide sufficient word co-occurrence information and avoid incorporating non-semantic word co-occurrence information. However, document embeddings of short texts contain a lot of noisy information resulting from the sparsity of word co-occurrence information. So we discard noisy information by changing the document embeddings into global and local semantic information. The global semantic information is the similarity probability distribution on the entire dataset and the local semantic information is the distances of similar short texts. Then we adopt a nested Chinese restaurant process to incorporate these two kinds of information. Finally, we compare our model to several state-of-the-art models on four real-world short texts corpus. The experiment results show that our model achieves better performances in terms of topic coherence and classification accuracy.
Highlights
With the growth of social media and applications of mobile phones, short texts have been a kind of prevalent and important information on the internet
The joint probability distribution of our model is p(l∗, l, z, w, α, β, γ, δ), where l∗ is the long document variable generated from the first step of the nested Chinese restaurant process, l is the long document variable generated from the second step of the nested Chinese restaurant process, z is the variable of the topic, w is the variable of the word, α is the dispersion prior of short texts that sample l∗, β is the dispersion prior of short texts that sample l, γ is the prior of multi-nominal distribution between z and l, and δ is the prior of multi-nominal distribution between z and w
We set parameters of DESTM as α = 0 and η = −1. The model with these settings means aggregating short texts according to the complete document embeddings and of cause including all noisy information
Summary
With the growth of social media and applications of mobile phones, short texts have been a kind of prevalent and important information on the internet. For documents with regular size, conventional topic models like LDA [2] and HDP [3] perform well These methods can automatically generate topics according to word co-occurrence information. Other methods incorporate word embeddings generated from an auxiliary corpus with documents of regular length [10,11,12,13,14]. Long documents incorporate sufficient word co-occurrence information and make local word co-occurrence information no longer sparse These models seem more reasonable than other strategies and auxiliary information is not needed. Long documents generated by our model can effectively avoid incorporating non-semantic word co-occurrence information. Document embedding information provides similarities of short texts and can avoid incorporating non-semantic word co-occurrence information.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
