Abstract
A necessary condition for optimal use in decision making of probability forecasts in general, and ensemble forecasts in particular, is that they should be calibrated. Ensemble calibration implies that each observation is statistically indistinguishable from the members of its forecast ensemble. The dimensionality of multivariate forecast settings, where the observation is a vector of related quantities rather than a scalar, affords opportunities for a rich variety of types of ensemble miscalibration, and so requires more sophisticated metrics for detection and diagnosis. This article investigates the performance in an artificial data setting of five multivariate ensemble calibration metrics that have been proposed in the literature. It is found that none of these dominates the others for detecting covariance miscalibration across a variety of miscalibration types, and that each exhibits weak performance in some instances, implying that several of the metrics should be used simultaneously when multivariate ensemble miscalibration is to be investigated. Multivariate biases were detected more strongly and diagnostically using univariate rank histograms for the individual forecast vector components.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
More From: Quarterly Journal of the Royal Meteorological Society
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.