Abstract
Topic modeling is a widely studied foundational and interesting problem in the text mining domains. Conventional topic models based on word co-occurrences infer the hidden semantic structure from a corpus of documents. However, due to the limited length of short text, data sparsity impedes the inference process of conventional topic models and causes unsatisfactory results on short texts. In fact, each short text usually contains a limited number of topics, and understanding semantic content of short text needs to the relevant background knowledge. Inspired by the observed information, we propose a regularized non-negative matrix factorization topic model for short texts, named TRNMF. The proposed model leverages pre-trained distributional vector representation of words to overcome the data sparsity problem of short texts. Meanwhile, the method employs the clustering mechanism under document-to-topic distributions during the topic inference by using Gibbs Sampling Dirichlet Multinomial Mixture model. TRNMF integrates successfully both word co-occurrence regularization and sentence similarity regularization into topic modeling for short texts. Through extensive experiments on constructed real-world short text corpus, experimental results show that TRNMF can achieve better results than the state-of-the-art methods in term of topic coherence measure and text classification task.
Highlights
Recent years have witnessed the increased development and popularity of various kinds of Web applications such as online social networks, recommender systems and Q&A systems
RELATED WORK we review two lines of relevant research work: 1) topic modeling for short text, 2) topic modeling for short text via vector embeddings
Liang et al [39] propose a global and local word embedding-based topic model (GLTM) for short texts, where the global word embeddings is learned from large external corpus and the local word embeddings is obtained by employing the continuous skip-gram model with negative sampling
Summary
Recent years have witnessed the increased development and popularity of various kinds of Web applications such as online social networks, recommender systems and Q&A systems. It is a simple and method-independent scheme that leverages external knowledge base to alleviate the data sparseness of short texts and discover the latent semantic information over short texts Existing works along this line largely depend on either external thesauri (e.g., WordNet) or lexical knowledge derived from documents in a specific domain (e.g., Wikipedia). The proposed TRNMF extends the non-negative matrix factorization model by introducing topic regularization from large text corpus in the term of topic-word distribution and document regularization by employing clustering mechanism to cluster short texts in. The model leverages the global word-word co-occurrence information learned from large text corpus to alleviate the data sparsity problem, and uses clustering method to improve topic inference quality.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
