Topic Shift Detection in Online Discussions using Structural Context

Abstract

Topic shift occurs frequently in online discussions, and automatically detecting topic shift can help to better capture the main clues and obtain relevant answers from large number of comments. Traditional topic-shift detection methods calculate text similarity and have limited success because they ignore semantic relatedness. In this paper, we propose a new topic shift detection model that uses conversational structure to enrich the context information and word embedding to build the semantic associations for each comment - post pair. Experiments show that the proposed model leads to better performance in terms of precision, recall, and F1 score.