Forecast Families: A New Method to Systematically Evaluate the Benefits of Improving the Skill of an Existing Forecast

Abstract

A growing number of studies have investigated how forecast skill, i.e., predictive power, translates into forecast value, i.e., usefulness, for improving forecast-informed decisions. The relationship between skill and value is widely understood to be complex and case-specific, yet few methods enable its systematic exploration using realistic forecast errors. This paper addresses this gap by proposing a single-parameter linear scaling method to generate families of synthetic forecasts with the desired skill improvements on an existing hindcast (a retrospective forecast of already-observed events). The method is applicable to any quantity for which a deterministic or ensemble hindcast is available, and generates a set of forecasts with different skill but strictly proportional errors. This like-for-like comparison preserves the autocorrelation and cross-correlations of errors, and opens the door for thorough, yet easily interpretable, explorations of the relationship between skill and value of a realistic forecast. We apply this new method to seasonal precipitation hindcasts (produced by the fifth generation of the Seasonal forecasting System of the European Centre for Medium-range Weather Forecasts, ECMWF-SEAS5) in order to explore their value for improving the management of a water supply system in the UK. The application showed that although value generally increases with skill, the skill–value relationship is not necessarily linear, and it strongly depends on operational preferences and hydrological conditions (wet or dry years). It also suggests that the forecast families methodology can help water managers and forecast developers identify when a skill increase would be most valuable. This has the potential to foster productive two-way conversations between forecast producers and users.