A New Measure of Ensemble Performance: Perturbation versus Error Correlation Analysis (PECA)

Abstract

Most existing ensemble forecast verification statistics are influenced by the quality of not only the ensemble generation scheme, but also the forecast model and the analysis scheme. In this study, a new tool called perturbation versus error correlation analysis (PECA) is introduced that lessens the influence of the initial errors that affect the quality of the analysis. PECA evaluates the ensemble perturbations, instead of the forecasts themselves, by measuring their ability to explain forecast error variance. As such, PECA offers a more appropriate tool for the comparison of ensembles generated by using different analysis schemes. Ensemble perturbations from both the National Centers for Environmental Prediction (NCEP) and the European Centre for Medium-Range Weather Forecasts (ECMWF) were evaluated and found to perform similarly. The error variance explained by either ensemble increases with the number of members and the lead time. The dynamically conditioned NCEP and ECMWF perturbations outperform both randomly chosen perturbations and differences between lagged forecasts [used in the “NMC” (for National Meteorological Center, the former name of NCEP) method for defining forecast error covariance matrices]. Therefore ensemble forecasts potentially could be used to construct flow-dependent short-range forecast error covariance matrices for use in data assimilation schemes. It is well understood that in a perfectly reliable ensemble the spread of ensemble members around the ensemble mean forecast equals the root-mean-square (rms) error of the mean. Adequate rms spread, however, does not guarantee sufficient variability among the ensemble forecast patterns. A comparison between PECA values and pattern anomaly correlation (PAC) values among the ensemble members reveals that the perturbations in the NCEP ensemble exhibit too much similarity, especially on the smaller scales. Hence a regional orthogonalization of the perturbations may improve ensemble performance.