A long‐term assessment of precipitation forecast skill using the Fractions Skill Score

Abstract

AbstractThe Fractions Skill Score (FSS) is a spatial verification metric routinely computed in the operational verification suite. It enables the comparison of forecasts of different resolutions against a common spatial truth (radar rainfall analyses) in such a way that high‐resolution forecasts are not penalized for representativeness errors that arise from the ‘double penalty’ problem. Officially Met Office model precipitation forecast accuracy is monitored using the Equitable Threat Score (ETS) at gauge locations. These precipitation scores form part of a basket of measures assessing six surface parameters known as the UK index, which forms the basis for making decisions regarding model upgrades (especially over the UK). It is used to monitor the impact of continuous model improvements. This framework and the methodology underlying it, is less appropriate for high‐resolution forecasts for reasons as described above. For precipitation forecasts in particular, a new framework for long‐term monitoring is necessary and the FSS provides such a potential framework.This paper provides an objective critique of FSS results to date. It has been shown that the ‘convection‐permitting’ (4 km) Unified Model (MetUM) forecasts are better than the 12 km MetUM (significant at the 5% level). The scale at which the models have sufficient practical skill is typically 10 km better for the high‐resolution forecasts, and are better at forecasting afternoon convection exceeding 4 mm (6 h)−1. The use of frequency (percentile) thresholds is recommended because of the implicit bias removal this approach provides, as any rain in a forecast period is treated as ‘the event of interest’. © 2011 British Crown Copyright, the Met Office. Published by John Wiley & Sons Ltd.