Novel Sentiment Analysis from Twitter for Stock Change Prediction

Abstract

Literature in behavioral economics and socioeconomics tells us that the public’s sentiment expression affects individual decision-making and hence the market collective decision-making. In this paper, we investigate whether public sentiment drives stock market performance. To be specific, we look at whether there is an association between changes in the Dow Jones Industrial Average (DJIA) and sentiment expression by using a large-scale comprehensive dataset of emotional state swings obtained from Twitter. We analyze relevant textual content on daily Twitter feeds using two sentiment quantification tools: FinBert, which is a categorical indicator that captures positive, neutral, and negative sentiment, and XLNet, which quantifies public sentiment from three types of moods (Positive, Neutral and Negative). Based on the time series dataset of the sentiment indicators, the relationship between public sentiment and DJIA index value is studied through Granger causal analysis and self-organizing fuzzy neural network. In addition, the changes in DJIA closing prices are predicted. Our results show that the accuracy of DJIA predictions can be significantly improved by including information on public sentiment. We have achieved state-of-the-art accuracy when predicting the daily up and down movement of the Dow Jones Industrial Average closing prices.