Screening probability forecasts: contrasts between choosing and combining

Abstract

In many forecasting situations, forecasts can be produced by several different methods. The ultimate objective of considering multiple methods may be to select a single method (the choosing scenario) or to aggregate the multiple forecasts into a single forecast (the combining scenario). Procedures for screening candidate forecasts— sufficiency in the choosing scenario and extraneousness in the combining scenario—are described here. Screening can identify forecasting methods that are dominated in the sense that their forecasts are clearly inferior to those of other methods or do not add any information to the combination of forecasts. These evaluation procedures are illustrated and contrasted by considering prototypical examples and an application involving precipitation probability forecasts. The value of screening is that it can reduce the set of candidate forecasting methods to a manageable number, which can then be evaluated in greater detail.