Predictive performance of ensemble hydroclimatic forecasts: Verification metrics, diagnostic plots and forecast attributes

Abstract

AbstractPredictive performance is one of the most important issues for practical applications of ensemble hydroclimatic forecasts. While different forecasting studies tend to use different combinations of verification metrics and diagnostic plots, this paper presents a literature survey of popular verification metrics, diagnostic plots and their corresponding forecast attributes. Twenty metrics to quantify predictive performance of ensemble, deterministic and categorical forecasts are detailed; six types of diagnostic plots to visualize the relationship between ensemble forecasts and observations are presented; and the correspondence relationships of verification metrics and diagnostic plots with ten forecast attributes are illustrated. Numerical experiments are devised for raw ensemble precipitation forecasts generated by the Climate Forecast System version 2 (CFSv2) to illustrate the verification metrics, diagnostic plots and forecast attributes. The results suggest that verification metrics that pay attention to some similar attributes can be related to one another. Meanwhile, the metrics are not necessarily in full accordance as they tend to reflect different attributes. Time series plots visualize forecasts over a period of interest and graphically illustrate forecast bias, reliability and skill. Quantile range plots provide an overall diagnosis for the association of ensemble forecasts with observations. Overall, verification metrics can be illustrated by diagnostic plots for individual grid cells and also be screened by spatial plots at regional and global scales. The survey presented in this article can serve as a roadmap for the selection of verification metrics, diagnostic plots and forecast attributes for verifying ensemble hydroclimatic forecasts.This article is categorized under: Science of Water > Hydrological Processes