EFFICIENT TESTING OF FORECASTS

Abstract

Each day a weather forecaster predicts a probability for each type of weather for the next day. After n days, all the predicted probabilities and the real weather data are sent to a test which decides whether to accept the forecaster as having prior knowledge about the distribution of nature. Consider tests that accept with high probability forecasters who know the distribution of nature. Sandroni shows that any such test can be passed with high probability by a forecaster who has no prior knowledge about the distribution of nature, provided that the duration n is revealed to the forecaster in advance [14]. However, Fortnow and Vohra show that Sandroni's result requires forecasters with high computational complexity [6]. Consider the family [Formula: see text] of forecasters who select a deterministic Turing-machine forecaster according to an arbitrary distribution and then use that machine for all future forecasts. We show that Sandroni's result requires forecasters even more powerful than those in [Formula: see text]. We also show that Sandroni's result does not apply when the duration n is not revealed to the forecaster in advance.