Social media signal detection using tweets volume, hashtag, and sentiment analysis

Abstract

Social Media is a well-known platform for users to create, share and check the new information. The world becomes a global village because of the utilization of internet and social media. The data present on Twitter contains information of great importance. There is a strong need to extract valuable information from this huge amount of data. A key research challenge in this area is to analyze and process this huge data and detect the signals or spikes. Existing work includes sentiment analysis for Twitter, hashtag analysis, and event detection but spikes/signal detection from Twitter remains an open research area. From this line of research, we propose a signal detection approach using sentiment analysis from Twitter data (tweets volume, top hashtag and sentiment analysis). In this paper, we propose three algorithms for signal detection in tweets volume, tweets sentiment and top hashtag. The algorithms are the- Average moving threshold algorithm, Gaussian algorithm, and hybrid algorithm. The hybrid algorithm is a combination of the average moving threshold algorithm and Gaussian algorithm. The proposed algorithms are tested over real-time data extracted from Twitter and two large publically available datasets- Saudi Aramco dataset and BP America dataset. Experimental results show that hybrid algorithm outperforms the Gaussian and average moving threshold algorithm and achieve a precision of 89% on real-time tweets data, 88% on Saudi Aramco dataset and 81% on BP America dataset with the recall of 100%.