Evaluating the skills of the CMIP5 global climate models using multicriteria decision-making analysis in Ethiopia

Abstract

This study evaluates the skills of 30 CMIP5 GCMs and the Multimodel Ensemble (MME) in reproducing the characteristics of observed precipitation (Pr), minimum (Tmin), and maximum (Tmax) temperature over the Middle Awash sub-basin (MASB) in Ethiopia. The MME of the climate variables was generated using the simple arithmetic mean method. The entire analysis was performed on the raw historical GCM simulations (before bias correction) and observed data for the periods 1981–2005 based on monthly and annual time series data over the annual and seasonal temporal resolutions. This study considered two approaches. The first one was an evaluation of GCMs employing five statistical performance metrics (SPMs), i.e., mean, CV, PBIAS, RSR, and r. The second approach involves the application of multicriteria decision-making (MCDM) analysis, adopting three SPMs (PBIAS, RSR, and r). The relative weights of the three metrics were determined by the entropy method. Besides, the weighted average and compromise programming techniques were employed to rank and select the best-performing GCMs. The findings from the first approach using five SPMs demonstrate that, for a given variable of interest, a GCM that performs well for one SPM may fail to produce the same for another SPM on the same temporal scale. Likewise, for the same SPM at different resolutions, a GCM may perform well for a one-time scale but poorly for another. These suggested that the results of GCM skills relied mainly on the SPM, time scale, and data formats chosen for analysis. Hence, it is critical to comprehensively evaluate the skill of GCMs using multiple performance metrics over a range of spatial and temporal settings and data formats. In addition, results of the MCDM analysis proved that the ensemble of GCMs, which provide adequate performance in simulating the salient features of Pr, Tmin, and Tmax concomitantly across the MASB, encompass CMCC-CMS, BCC-CSM1.1(m), CMCC-CM, BNU-ESM, CanESM2, and MPI-ESM-MR. However, it was observed that different GCMs performed much differently in characterizing various variables over a range of temporal scales and data formats. The MME also proved its superior potential in duplicating the climate of the study area over several individual GCMs. The overall findings attested that instead of aggregating the ranks from the three variables into one, it is recommended to treat each variable independently while developing a subset of best-performing GCMs for ensembling since each GCM responds differently to each variable under a set of conditions. Finally, the approaches and findings from this study will be valuable input for subsequent climate and hydrologic studies in the study area and beyond.