The verification of objective analyses: Diagnostics of analysis system performance

Abstract

We use the spatial coherence of the fit of analyses to observation as the basis for new diagnostics of the performance of any linear objective analysis system. The diagnostics provide answers to such questions as: Up to now these questions could only be answered indirectly, by evaluation of the forecasts resulting from the analyses, or by study of the changes made by the initialisation. The lack of simple objective methods to address these questions has sometimes resulted in controversy about the interpretation of data impact studies. Suppose, in a practical analysis system, that the observation errors are uniform and uncorrelated, and the observations are homogeneously distributed. We show (for any practical linear analysis system) that if the Observation-minus-Analysis (OmA) auto-correlations are positive when extrapolated to zero separation, then the analysis has certainly not extracted all the information from the observations, and does not fit the data to within the observational error. If however the extrapolated OmA correlations are negative, then the analysis system does fit the data to within observational accuracy. If, in addition, the weights given to the observations in the analysis of the observed values are known, one can derive useful estimates of the analysis error at the observation points. This last result leads to an estimate of the effective data density. Along with the new verifications of mass and wind analyses, methods are developed to estimate the effectiveness with which data on mass-wind balance have been used. The results for practical analysis systems are used to interpret empirical determinations of the OmA auto-correlations in the ECMWF analysis system, for data with uncorrelated errors. We shall say that a practical analysis system is efficient if the data are fitted to within observational error, and is inefficient otherwise. The methods demonstrate that operational mid-tropospheric wind analyses over North America are quite efficient, but that the windshear and thickness analyses near the tropopause over North America are inefficient. The analysis of mass wind balance over North America is efficient for any single level, but the analysis of thermal wind balance for thin layers near the tropopause is inefficient, most notably on short horizontal scales. The analysis of the wind shear near the tropical tropopause is somewhat better than over North America, probably because of the use of sharper vertical structure functions.