Single-model Ensembles Research Articles

Towards the characterization of streamflow simulation uncertainty through multimodel ensembles

Distributed hydrologic modeling holds significant promise for improved estimates of streamflow with high spatial resolution. However, uncertainty in model structure and parameters, which are distributed in space, and in operational weather radar rainfall estimates, which comprise the main input to the models, contributes to significant uncertainty in distributed model streamflow simulations over a wide range of space and time scales. Using the simulations produced for the Distributed Model Intercomparison Project (DMIP), this paper develops and applies sample-path methods to characterize streamflow simulation uncertainty by diverse distributed hydrologic models. The emphasis in this paper is on the model parameter and structure uncertainty given radar rainfall forcing. Multimodel ensembles are analyzed for six application catchments in the Central US to characterize model structure uncertainty within the sample of models (both calibrated and uncalibrated) participating in DMIP. Ensembles from single distributed and lumped models are also used for one of the catchments to provide a basis to characterize the impact of parametric uncertainty versus model structure uncertainty in flow simulation statistics. Two main science questions are addressed: (a) what is the value of multimodel streamflow ensembles in terms of the probabilistic characterization of simulation uncertainty? And (b) how do probabilistic skill measures of multimodel versus single-model ensembles compare? Discussed also are implications for the operational use of streamflow ensembles generated by distributed hydrologic models. The results support the serious consideration of ensemble simulations and predictions created by diverse models in real time flow prediction.

DEVELOPMENT OF A EUROPEAN MULTIMODEL ENSEMBLE SYSTEM FOR SEASONAL-TO-INTERANNUAL PREDICTION (DEMETER)

A multi-model ensemble-based system for seasonal-to-interannual prediction has been developed in a joint European project known as DEMETER (Development of a European Multimodel Ensemble Prediction System for Seasonal to Interannual Prediction). The DEMETER system comprises seven global atmosphere–ocean coupled models, each running from an ensemble of initial conditions. Comprehensive hindcast evaluation demonstrates the enhanced reliability and skill of the multimodel ensemble over a more conventional single-model ensemble approach. In addition, innovative examples of the application of seasonal ensemble forecasts in malaria and crop yield prediction are discussed. The strategy followed in DEMETER deals with important problems such as communication across disciplines, downscaling of climate simulations, and use of probabilistic forecast information in the applications sector, illustrating the economic value of seasonal-to-interannual prediction for society as a whole.

Retrospective ENSO Forecasts: Sensitivity to Atmospheric Model and Ocean Resolution

Results are described from a series of 40 retrospective forecasts of tropical Pacific SST, starting 1 January and 1 July 1980–99, performed with several coupled ocean–atmosphere general circulation models sharing the same ocean model—the Modular Ocean Model version 3 (MOM3) OGCM—and the same initial conditions. The atmospheric components of the coupled models were the Center for Ocean–Land–Atmosphere Studies (COLA), ECHAM, and Community Climate Model version 3 (CCM3) models at T42 horizontal resolution, and no empirical corrections were applied to the coupling. Additionally, the retrospective forecasts using the COLA and ECHAM atmospheric models were carried out with two resolutions of the OGCM. The high-resolution version of the OGCM had 1° horizontal resolution (1/3° meridional resolution near the equator) and 40 levels in the vertical, while the lower-resolution version had 1.5° horizontal resolution (1/2° meridional resolution near the equator) and 25 levels. The initial states were taken from an ocean data assimilation performed by the Geophysical Fluid Dynamics Laboratory (GFDL) using the high-resolution OGCM. Initial conditions for the lower-resolution retrospective forecasts were obtained by interpolation from the GFDL ocean data assimilation. The systematic errors of the mean evolution in the coupled models depend strongly on the atmospheric model, with the COLA versions having a warm bias in tropical Pacific SST, the CCM3 version a cold bias, and the ECHAM versions a smaller cold bias. Each of the models exhibits similar levels of skill, although some statistically significant differences are identified. The models have better retrospective forecast performance from the 1 July initial conditions, suggesting a spring prediction barrier. A consensus retrospective forecast produced by taking the ensemble average of the retrospective forecasts from all of the models is generally superior to any of the individual retrospective forecasts. One reason that averaging across models appears to be successful is that the averaging reduces the effects of systematic errors in the structure of the ENSO variability of the different models. The effect of reducing noise by averaging ensembles of forecasts made with the same model is compared to the effects from multimodel ensembling for a subset of the cases; however, the sample size is not large enough to clearly distinguish between the multimodel consensus and the single-model ensembles. There are obvious problems with the retrospective forecasts that can be connected to the various systematic errors of the coupled models in simulation mode, and which are ultimately due to model error (errors in the physical parameterizations and numerical truncation). These errors lead to initial shock and a “spring variability barrier” that degrade the retrospective forecasts.

The skill of multi-model seasonal forecasts of the wintertime North Atlantic Oscillation

The skill assessment of a set of wintertime North Atlantic Oscillation (NAO) seasonal predictions in a multi-model ensemble framework has been carried out. The multi-model approach consists in merging the ensemble hindcasts of four atmospheric general circulation models forced with observed sea surface temperatures to create a multi-model ensemble. Deterministic (ensemble-mean based) and probabilistic (categorical) NAO hindcasts have been considered. Two different sets of NAO indices have been used to create the hindcasts. A first set is defined as the projection of model anomalies onto the NAO spatial pattern obtained from atmospheric analyses. The second set obtains the NAO indices by standardizing the leading principal component of each single-model ensemble. Positive skill is found with both sets of indices, especially in the case of the multi-model ensemble. In addition, the NAO definition based upon the single-model leading principal component shows a higher skill than the hindcasts obtained using the projection method. Using the former definition, the multi-model ensemble shows statistically significant (at 5% level) positive skill in a variety of probabilistic scoring measures. This is interpreted as a consequence of the projection method being less suitable because of the presence of errors in the spatial NAO patterns of the models. The positive skill of the seasonal NAO found here seems to be due not to the persistence of the long-term (decadal) variability specified in the initial conditions, but rather to a good simulation of the year-to-year variability. Nevertheless, most of the NAO seasonal predictability seems to be due to the correct prediction of particular cases such as the winter of 1989. The higher skill of the multi-model has been explained on the basis of a more reliable description of large-scale tropospheric wave features by the multi-model ensemble, illustrating the potential of multi-model experiments to better identify mechanisms that explain seasonal variability in the atmosphere.

Multi‐model multi‐analysis ensembles in quasi‐operational medium‐range forecasting

AbstractEnsemble prediction systems (EPS) for medium‐range forecasting attempt to account for uncertainty in numerical weather prediction (NWP) by sampling the distribution function of future atmospheric states. Forecast uncertainty derives from uncertainty in both the analysed initial conditions (analysis errors) and in the forecast evolution (model errors). Current operational systems are primarily based on sampling the analysis errors through initial‐condition perturbations with, at best, only limited sampling of model errors. One approach to sampling model errors and also to widening the sampling of analysis errors, is to include more than one NWP model, and more than one operational analysis to which perturbations are added, in the ensemble system. Previous work has demonstrated from a small number of case‐studies that this multi‐model multi‐analysis ensemble (MMAE) approach can perform significantly better than a single‐model system such as the Ensemble Prediction System (EPS) run by the ECMWF (European Centre for Medium‐Range Weather Forecasts). In this study a MMAE was created by combining the ECMWF ensemble with an ensemble using the Met Office model and analysis, and was run daily for a year to assess the benefits over a larger, quasi‐operational sample of forecasts. The results are compared with the operational ECMWF EPS which includes the latest upgrades, including stochastic physics which makes some allowance for uncertainty due to model errors. Results show that both for probabilistic forecasts (assessed by Brier skill scores and relative operating characteristics) and for deterministic forecasts based on the ensemble mean (assessed by root‐mean square errors) the MMAE has increased forecast skill relative to the EPS. These improvements are obtained with no overall increase in ensemble size. Ensemble spread is also greater in the MMAE, and the increased skill is believed to be due to the additional model producing solutions which are synoptically more different than those produced by a single model ensemble. Benefits of the MMAE vary both in time and with geographical region, depending on which individual ensemble system performs better in particular synoptic situations. It is found that the MMAE almost always performs as well as the best individual ensemble, and on occasions better than either of them. © Crown copyright, 2002.