Stock Market Prediction using Daily News Headlines

Abstract

In this work, we develop a model for forecasting the stock market using News Headlines. The main step of the work is to pre-process daily news, as well as Opinion Finding method to generate features for the prediction problem. The target variable is a binary variable which takes value of 1, when the Dow Jones Industrial Average (DJIA) increases, and zero otherwise. A deep learning model with Long Short-Term Memory (LSTM) architecture to improve the prediction of the changes in DJIA. As a baseline, a time series data of DJIA is used without daily news. Then, we develop a model of time series prediction along with N-Gram of daily news. We discuss how N-gram model can help to improve the baseline. In the main pre-processing task, we develop a sentiment analysis (using polarity lexicons). This model assigns weights as sentiment polarity to all the news headlines. We also find the scores for degree of subjectivity and objectivity in news headlines, and we use them as additional features in our model. In experiments, an LSTM model is developed. A grid search is used to find the best architecture along with the effect of generated features from text pre-processing and Opinion Finding on time series forecasting.