A Scoring Rule for Factor and Autoregressive Models Under Misspecification

Roberto Casarin,Fausto Corradin,Francesco Ravazzolo,Domenico Sartore

doi:10.2139/ssrn.3219696

Abstract

Factor models (FM) are now widely used for forecasting with large set of time series. Another class of models, which can be easily estimated and used in a large dimensional setting, is multivariate autoregressive models (MAR), where independent autoregressive processes are assumed for the series in the panel. We compare the forecasting abilities of FM and MAR models when assuming both models are misspecified and the data generating process is a vector autoregressive model. We establish which conditions need to be satisfied for a FM to overperform MAR in terms of mean square forecasting error. The condition indicates in presence of misspecification that FM is not always overperforming MAR and that the FM predictive performance depends crucially on the parameter values of the data generating process. Building on the theoretical relationship between FM and MAR predictive performances, we provide a scoring rule which can be evaluated on the data to either select the model, or combine the models in forecasting exercises. Some numerical illustrations are provided both on simulated data and on well-known large economic datasets. The empirical results show that the frequency of the true positive signals is larger when FM and MAR forecasting performances differ substantially and it decreases as the horizon increases.

Highlights

The recent fast growth in big data allows researchers to model and predict variables of interest more accurately and suggests that there are large potential gains from using a big set of variables instead of a single univariate time series models in many inference applications
In this paper we focus on two simple models that are widely used in forecasting: factor models, with reduced number of factors (FM), and multivariate autoregressive models (MAR), where no interaction is assumed between the series in the panel
Our new model-specific scoring rule for Factor models (FM) and MAR models indicates that the forecasting performances of the models depend crucially on the parameters setting of data generating process

Summary

Introduction

The recent fast growth in (real-time) big data allows researchers to model and predict variables of interest more accurately and suggests that there are large potential gains from using a big set of variables instead of a single univariate time series models in many inference applications. Our new model-specific scoring rule for FM and MAR models indicates that the forecasting performances of the models depend crucially on the parameters setting of data generating process. A′kΓX,1Ak = Γk, and A′kΓX Ak = Γk We assume both FM and MAR models are misspecified and provide in the following theorem a scoring rule which is a function of the FM and MAR parameters. We show how to use the inequality to score the FM and MAR models or to combine their forecasts (see Billio et al (2013)).

Conclusion

Proof of the result in Remark 1

Proof of the result in Theorem 1

Proof of the result in Theorem 2

Findings

Proof of the result in Theorem 3