Comparative Study to Perform Tweets/NEWS Classification Using Big Data Analytics for Predicting Stock Market Price Movement

Abstract

AbstractIn finding prediction in a range of domains, sentiment classification of the Twitter and NEWS data has been applied successfully. However, it is still difficult and continuing research to use sentiment classification to forecast stock market factors. Sentiment analysis is complex and essential task that must be done, and natural language processing and machine learning all are concerned. The purpose of this research is to compare accuracy of the three feature extraction techniques (TF-IDF, CountVectorizer, Word2Vec) and three machine learning classifiers (logistic regression, random forest, and Naive Bayes classifier) in order to provide stock-related tweets and news data with positive, negative, and neutral sentiment. The experimented results show that random forest algorithm provides highest accuracy and logistic regression algorithm with the less accuracy.KeywordsSpark NLP pipelinePreprocessingTokenizerFeature extractionMachine learning classifierSentiment classification