The impact of word sense disambiguation on stock price prediction

Abstract

State-of-the-art decision support systems for stock price prediction incorporate pattern-based event detection in text into their predictions. These systems typically fail to account for word meaning, even though word sense disambiguation is crucial for text understanding. Therefore, we propose an advanced natural language processing pipeline for event-based stock price prediction, that allows for word sense disambiguation to be incorporated in the event detection process. We identify events in natural language news messages and subsequently weight these events for their historical impact on stock prices. We assess the merit of word sense disambiguation in event-based stock price prediction in two evaluation scenarios for NASDAQ-100 companies, based on historical stock prices and news articles retrieved from Dow Jones Newswires over a 2-year period. We evaluate the precision of generated buy and sell signals based on our predicted stock price movements, as well as the excess returns generated by a trading strategy that acts upon these signals. Event-based stock price predictions seem most reliable about 2 days into the future. The number of detected events tends to reduce with over 30% when graph-based word sense disambiguation using a degree centrality measure is applied in the event detection process, thus reducing the noise introduced into the stock price movement predictions by high-impact ambiguous events. As a result, modest improvements in the precision of buy and sell signals generated based on these predictions tend to lead to vast improvements of on average about 70% in the associated excess returns.