Machine-learning stock market volatility: Predictability, drivers, and economic value

Abstract

We investigate whether machine learning (ML) techniques, using a large set of financial and macroeconomic variables, help to predict S&P 500 realized volatility and deliver economic value. We evaluate regularization methods (Ridge, Lasso, and Elastic Net), tree-based methods (Random Forest and Gradient boosting), and Neural Networks. We find that ML algorithms outperform the benchmark model (HAR) at a short horizon (1 month), but not over longer periods (6 and 12 months). Regularization methods and Neural Networks emerge as the most competitive ML methods. We find that the quality of predictors is crucial, with financial and macroeconomic uncertainty proxies playing the most significant role. From an economic perspective, however, predictive ML models do not yield substantial gains compared to the benchmark.