Topic based hierarchical summarisation of Twitter

Abstract

Twitter has become a rich source of information nowadays. The data generated however is so large in volume that it is not possible to manually go through each and every tweet to understand the context of data. One of the ways to get insight into the bulk of data at hand is to know the topics contained in it. As in the context of Twitter, we define topics to be long-lasting subjects around which the conversations of people revolve, such as sports, music and politics amongst others. However, the topics identified may be large in number and might be cumbersome for human interpretation. Considering these views, in this paper we address the information overload problem of Twitter data and propose a topic based hierarchical summarisation framework for the same. In contrast to imposing restrictions on topic models to depict the hierarchical structure, we propose an algorithm which constructs a topic hierarchy out of any given number of topics. We showcase the effectiveness of the proposed algorithm for the Twitter dataset prepared for Egyptair MS181 flight incident.