Unsupervised Event Detection Using Self-learning-based Max-margin Clustering: Analysis on Streaming Tweets

Abstract

ABSTRACT We propose an unsupervised approach for tweet clustering from large-scale Twitter repository in this paper. The amount of acquired data from streaming media like Twitter is vast in nature. They contain readily available information regarding important events taking place during the time span. Hence, it is indeed difficult to deploy supervised learning strategies for analyzing the tweets for meaningful information extraction. On top of that, the tweets are unstructured in nature given the diversities of the end-users who put the tweets. Given that, an unsupervised tweet-processing technique can be of immense help for different inference tasks including event extraction, sentiment analysis, to name a few. Based on the aforementioned bottlenecks of the majority of the existing techniques, we propose a novel unsupervised event detection strategy from streaming tweets. In this regard, we propose a self-learning max-margin clustering which deploys the notion of SVM in an unsupervised setup. We evaluate proposed system and compare it with the popular techniques from the literature using 6.5 million streaming tweets, collected in June 2017. In our experiments, self-learning-based max-margin clustering outperforms the techniques of literature in terms of precision, Silhouette score, and Calinski–Harabasz score.