On evaluating the performance of different forecasters

Abstract

ABSTRACTIn this article, we have evaluated the performance of different forecasters and tested association between their performances for different pairs of variables. We have used three data sets of track records of professional U.S. economic forecasters participating in the Blue Chip consensus forecasting service (the data sets contain the root mean square errors (RMSE) of different forecasters for different years). To evaluate the performance of forecasters we have covered three well-known tests, namely the usual F test (cf. Fisher (1923)), Kruskal Wallis test (cf. Kruskal and Wallis (1952)), and Extension of Median test (cf. Daniel (1990)). To test the association between the forecaster's performances for different pairs of variables, we have considered Gini mean correlation coefficient rg1 (cf. Yitzhaki, S., and Olkin, I. (1991) and Yitzhaki (2003)), Modified rank correlation coefficient (cf. Zimmerman (1994)) and three modifications of Spearman rank correlation coefficient. We have observed that different forecasters do not necessarily offer same average performance. Moreover, an evidence of association between two criteria does not always lead us reaching at the same decision. The outcomes of the study may help the practitioners in selecting the best forecaster(s) for policymaking purposes.