Results of a Price‐Forecasting Competition: Comment

Abstract

son-Merton test as given in table 2 of Dorfman and McIntosh are in error. In almost all cases, the reported confidence levels exceed the actual levels. The errors in reported confidence levels also have affected the values of the loss function and, hence, the ranking of the models by the loss function criterion. In this comment, we provide the correct confidence levels of the Henriksson-Merton test for each of the three models and recompute the loss function using the correct confidence levels. We also present results from the Henriksson-Merton test in a regression framework suggested by Cumby and Modest. The Henriksson-Merton test is a nonparametric test which focuses on direction rather than magnitude of changes. Calculation of this test provides information about the number of times each model correctly and incorrectly predicts both upward and downward directional changes in the variables of interest. For a series of N out-of-sample forecasts, define N, as number of observations where prices actually rose; N2, number of observations where prices actually did not rise; N N + N2, total number of out-of-sample observations; n1, number of successful predictions given that price rose; n2, number of unsuccessful predictions given that price did not rise; and n = n + n2, number of times a price rise was forecast. The Henriksson-Merton test for forecasting ability examines whether the observed number of successful