Reply to “Comments on ‘A Simple, Coherent Framework for Partitioning Uncertainty in Climate Predictions’”

Abstract

We thank Dr. Northrop for his thoughtful comments (Northrop 2013) on our article (Yip et al. 2011, hereafter YFSH11). His analysis, using all 97 200 possible subsets of data comprising two runs per model and scenario, illustrates clearly that the results are sensitive to the particular dataset that is chosen and thus provides a more complete summary of the sources of variation in phase 3 of the Coupled Model Intercomparison Project (CMIP3) ensemble than we presented in YFSH11. Our principal aim, of course, was to describe and illustrate the analysis of variance (ANOVA) methodology for decomposing the total uncertainty in a climate ensemble into model uncertainty, scenario uncertainty, internal variability, and model–scenario interaction. We did not attempt a comprehensive analysis of the CMIP3 ensemble, but we are glad that Dr. Northrop has extended our analysis to do so. To check the robustness of the conclusions in YFSH11 to the choice of data subset, we repeated our original analysis for each of six different subsets corresponding to all possible choices of two runs from the four runs of the Parallel Climate Model (PCM) under scenario B1. The main conclusions from YFSH11 still stand, even though they were obtained from analyzing only one of the 97 200 datasets presented by Dr. Northrop: 1) scenario uncertainty dominates all other components after 2050, 2) internal variability is constant over time but decreases sharply as a fraction of the total uncertainty, 3) scenario uncertainty dominates the model–scenario interaction uncertainty over the entire century, and 4) model– scenario interaction makes an important contribution to the total uncertainty. To make a comprehensive assessment of the variation in an ensemble, it is ideal to include all possible model runs. However, ensembles with unbalanced designs, where unequal numbers of the different model–scenario combinations are available, complicate the analysis. The ANOVA framework can easily be extended to unbalanced designs, for example, by using linear regression on model and scenario factors (Sansom et al. 2013). The results from such frameworks depend on the design of the experiments and so this needs to be considered more carefully when designing multimodel ensembles (e.g., avoiding only one run per model in future scenarios, which is prevalent in CMIP5).