Comparative Study of Classification Algorithms to Analyze and Predict a Twitter Sentiment in Apache Spark

Abstract

Nowadays, the major sources of information exchange are Twitter, Facebook, WordPress, etc. The tweets can be considered as the source of the public opinion on an event, a product, or a topic. Consequently, it contains large volumes of natural data. Enormous dataset contains a huge volume and variety of information. Therefore, it cannot be prepared utilizing normal conventional tools. It can be processed by building up distributed environment or by contracting cloud based isolated infrastructure. Therefore, better approaches and instruments are required to bring the respect of the information. Apache spark is actually appropriate for performing machine learning on large-scale information. To discover out how rapidly Spark processes of huge information, we make an approach that utilize Machine Learning library (MLlib) classification algorithms in Apache Spark. We implement Logistic Regression, Multilayer Perceptron (MLP), Random Forest and Support Vector Machine (SVM) and compare among them. The models are for analysis and predicting the sentiment based on a corona tweets. A sentiment analysis based on tweets is a challenging issue. The classification algorithm is assessed by precision, recall, f-measure, accuracy and time consumed. The results show that logistic regression algorithm has higher speed in doing enormous information processing than other chosen algorithms. Moreover, according to the obtained results, apache spark has exceptionally great speed in handling big data. The classification algorithm is assessed by precision, recall, f-measure, accuracy and time consumed.