Relationships between potential, attainable, and actual skill in a decadal prediction experiment

Abstract

A sequence of ensemble forecasts permits the estimation of the “predictable” and “unpredictable” components of the forecasts and hence the predictability of the system. Predictability is a feature of a physical and/or mathematical system which characterizes its “ability to be predicted” as measured, for instance, by the rate at which initially close states separate. Modern climate models may be used to predict climate evolution on seasonal to decadal timescales and they may also be used to estimate predictability. The predictability in this case measures the model’s ability to predict its own evolution and, for a well behaved climate model, this may be a useful measure of the predictability of the climate system itself. Forecast skill, where forecasts are compared to observations, indicates a forecast system’s “ability to predict” the evolution of the climate system. Common measures of skill include correlation, mean square error and mean square skill score. When the model is used to predict its own evolution, rather than the evolution of the actual climate system, the same measures of skill may be used but are then predictability measures and termed measures of “potential skill”. The expectation is that the model’s skill in predicting its own evolution will be greater than its ability to predict the actual evolution of the system in the face of observational uncertainty and model deficiencies. The further expectation is that the differences between potential and actual skill can give an indication of where and to what extent there is the potential for improvement in forecast skill. Predictability studies may be undertaken with available models but it may not be immediately obvious which statistical conditions are necessary if potential skill is to be a reasonable estimate of attainable skill. The purpose of this paper is to investigate and illustrate the spatial and temporal behaviour of the predictable and unpredictable components of surface air temperature in a decadal prediction experiment and to discuss some of the implications for predictability studies and for analyzing model behaviour. Formal relationships between potential, actual, and attainable skill are developed which, however, provide only modest constraints on model results if they are to be used to infer predictability. The results of various scalings of the forecasts on potential and actual skill and their relationship are also considered. Differences in the actual and potential forecast skill between models and/or model versions can be partitioned into components associated with known statistics as a guide to forecast improvement.