Twin‐analysis verification: A new verification approach to alleviate pitfalls of own‐analysis verification

Abstract

AbstractIn operational numerical weather prediction (NWP), forecast verification against analysis from the same experiment is part of the standard evaluation practice. This “own‐analysis” verification is beneficial in providing complete and uniform spatial coverage but is known to suffer from overestimated skill scores when applied to short‐range forecasts due to the inevitable positive correlation between the forecast and analysis errors. To alleviate this problem, a new approach termed “twin‐analysis” verification is proposed. In this approach, the forecasts are verified against “twin analyses” that are produced by running an independent cycle using the same NWP system as the one used to produce the forecasts, but initializing the cycle using an independent first guess at the very beginning. In this way the error correlation between the forecasts and analyses is reduced, if not completely removed, while preserving the statistical properties of the analyses, allowing for a clearer interpretation of verification. This note reports the results of comparison between “twin‐analysis” and “own‐analysis” verification scores obtained for the Japan Meteorological Agency's global NWP system. It is reported that the two scores disagree up to 48 hr or even longer depending on the regions and elements, which highlights the reliability limit of “own‐analysis” verification. The proposed method does not completely remove the error correlation issue and also requires high computational costs; a practical use of the proposed approach in an operational environment is also discussed.