A new big data approach for topic classification and sentiment analysis of Twitter data

Abstract

Twitter is a major micro-blogging service, with millions of active users. These users use Twitter to post status messages called tweets and share their opinions using hash tags on various events. Hence, Twitter is considered a major real time streaming source and one of an effective and accurate indicator of opinions. The amount of data generated by Twitter is huge and it is difficult to scan entire data manually. This paper proposes a Hybrid Lexicon-Naive Bayesian Classifier (HL-NBC) method for sentimental analysis. In addition to that, Sentiment analysis engine is preceded by topic classification, which classifies tweets into different categories and filters irrelevant tweets. The proposed method is compared with Lexicon, Naive Bayesian classifier for uni-gram and bi-gram features. Out of the different approaches, the proposed HL-NBC method does sentiment classification in an improved way and gives accuracy of 82%, which is comparatively better than other methods. Also, the sentiment analysis is performed in a shorter time compared to traditional methods and achieves 93% improvement in processing time for larger datasets.