Intercomparison of prediction skills of ensemble methods using monthly mean temperature simulated by CMIP5 models

Abstract

This study focuses on an objective comparison of eight ensemble methods using the same data, training period, training method, and validation period. The eight ensemble methods are: BMA (Bayesian Model Averaging), HMR (Homogeneous Multiple Regression), EMOS (Ensemble Model Output Statistics), HMR+ with positive coefficients, EMOS+ with positive coefficients, PEA_ROC (Performance-based Ensemble Averaging using ROot mean square error and temporal Correlation coefficient), WEA_Tay (Weighted Ensemble Averaging based on Taylor’s skill score), and MME (Multi-Model Ensemble). Forty-five years (1961-2005) of data from 14 CMIP5 models and APHRODITE (Asian Precipitation- Highly-Resolved Observational Data Integration Towards Evaluation of Water Resources) data were used to compare the performance of the eight ensemble methods. Although some models underestimated the variability of monthly mean temperature (MMT), most of the models effectively simulated the spatial distribution of MMT. Regardless of training periods and the number of ensemble members, the prediction skills of BMA and the four multiple linear regressions (MLR) were superior to the other ensemble methods (PEA_ROC, WEA_Tay, MME) in terms of deterministic prediction. In terms of probabilistic prediction, the four MLRs showed better prediction skills than BMA. However, the differences among the four MLRs and BMA were not significant. This resulted from the similarity of BMA weights and regression coefficients. Furthermore, prediction skills of the four MLRs were very similar. Overall, the four MLRs showed the best prediction skills among the eight ensemble methods. However, more comprehensive work is needed to select the best ensemble method among the numerous ensemble methods.