Criteria to evaluate the validity of multi‐model ensemble methods

Abstract

Multi‐model ensemble (MME) methods have been developed to improve upon the simulations of individual general circulation models. Their performances can be evaluated using metrics such as correlation and root‐mean‐square error, between the simulations and observations. However, most metrics change with the length of the calibration period, meaning the skill of MME methods in simulating future climate change is poorly known. In the present work, an order preservation criterion and a boundary preservation criterion are proposed to guarantee the reliability of MME simulations in the future. The order preservation criterion makes every model contribute a positive value to the MME simulations, while the boundary preservation criterion restricts the range of variation in the MME simulations. Four commonly used MME methods are evaluated based on these two criteria. The results show that the multiple linear regression method and singular value decomposition method are unsuitable MME methods in most situations. However, the arithmetic ensemble mean and Bayesian model averaging can be used to combine model simulations. The two criteria proposed in this study provide a simple way to evaluate the validity of MME methods.