Hierarchical Topic Modeling of Twitter Data for Online Analytical Processing

Abstract

Social platforms, such as Twitter, reveal much about the tastes of the public. Many studies focus on the content analysis of social platforms, which assists in product promotion and sentiment investigation. On the other hand, online analytical processing (OLAP) has been proven to be very effective for analyzing multidimensional structured data. The key purpose of applying OLAP to text messages, (e.g., tweets), called text OLAP, is to mine and construct the hierarchical dimension based on the unstructured text content. In contrast to the plain texts which text OLAP usually handles, the social media content includes a wealth of social relationship information which can be employed to extract a more effective dimensional hierarchy. In this paper, we propose a topic model called twitter hierarchical latent Dirichlet allocation (thLDA). Based on hierarchical latent Dirichlet allocation, thLDA aims to automatically mine the hierarchical dimension of tweets’ topics, which can be further employed for text OLAP on the tweets. Furthermore, thLDA uses word2vec to analyze the semantic relationships of words in tweets to obtain a more effective dimension. We conduct extensive experiments on huge quantities of Twitter data and evaluate the effectiveness of thLDA. The experimental results demonstrate that it outperforms other current topic models in mining and constructing the hierarchical dimension of tweeters’ topics.