On the Relationship Between Reliability Diagrams and the “Signal‐To‐Noise Paradox”

Abstract

AbstractThe “signal‐to‐noise paradox” for seasonal forecasts of the winter North Atlantic Oscillation (NAO) is often described as an “underconfident” forecast and measured using the ratio‐of‐predictable components (RPCs) metric. However, comparison of RPC with other measures of forecast confidence, such as spread‐error ratios, can give conflicting impressions, challenging this informal description. We show, using a linear statistical model, that the “paradox” is equivalent to a situation where the reliability diagram of any percentile forecast has a slope exceeding 1. The relationship with spread‐error ratios is shown to be far less direct. We furthermore compute reliability diagrams of winter NAO forecasts using seasonal hindcasts from the European Centre for Medium‐range Weather Forecasts and the UK Meteorological Office. While these broadly exhibit slopes exceeding 1, there is evidence of asymmetry between upper and lower terciles, indicating a potential violation of linearity/Gaussianity. The limitations and benefits of reliability diagrams as a diagnostic tool are discussed.