Monte Carlo forecast evaluation with persistent data

Abstract

Persistent processes, including local-to-unity and random walks, are commonly considered as forecasting models of interest. However, the associated forecast errors follow non-standard distributions that complicate forecast evaluation tests. We propose a finite sample simulation-based solution to this problem. The method requires a flexible parametric null model that can be simulated as long as a finite dimension nuisance parameter can be specified. The size control of our method is robust to non-standard limiting distributions, such as degenerate asymptotic distribution problems that arise from nested and unit root models. Our simulation studies demonstrate that many of the existing forecast evaluation methods, including various bootstraps, over-reject for highly persistent data. In contrast, our method is level correct and has good power. We extend our approach to the inversion of forecast evaluation statistics in order to construct exact confidence sets for the benchmark model. Confidence sets provide much more information than tests, particularly in the case of the persistence-adjusted relevance of predictive regressors (Rossi, 2005).