Generation of topic evolution graphs from short text streams

Abstract

Topic evolution mining on short texts is an important research topic in natural language processing. Existing methods have been focused either on the topic evolution of normal documents or on the evolution of topics along a timeline. In this paper, we aim to generate topic evolutionary graphs from short texts, which not only capture the main topic timeline, but also reveal the correlations between related subtopics. Firstly, we propose an Encoder-only Transformer Language Model (ETLM) to quantify the relationship between words. Then we propose a novel topic model, referred as weighted Conditional random field regularized Correlated Topic Model (CCTM), which leverages semantic correlations to discover meaningful topics and topic correlations. Finally, topic evolutionary graphs are generated by an Online version of CCTM (OCCTM) to capture the evolutionary patterns of main topics and related subtopics. Experimental results on real-world datasets demonstrate our method outperforms baselines on quality of topics and presents motivated patterns for topic evolution mining.