Use Of Multi-model Ensembles Research Articles

Standardized precipitation evapotranspiration index (SPEI) for Canada: assessment of probability distributions

Candidate probability distributions for the Standardized Precipitation Evapotranspiration Index (SPEI) for Canada were examined using the Canadian Gridded (CANGRD) temperature and precipitation dataset and CMIP5 projections. The probability distribution is a core component to the calculation of standardized values. For SPEI, a continuous probability distribution is fitted to the water balance time series so that the resulting transformed index follows a standard normal distribution. Selection of an appropriate distribution is therefore important as an inappropriate distribution may lead to biased values and subsequently affect the interpretation of SPEI results. Candidate distributions considered for SPEI included the generalized logistic (GLO), generalized extreme value (GEV), Pearson Type III (PE3) and normal (NOR) distributions. A range of goodness of fit tests were used to assess how well the distributions fit SPEI. Differences in CANGRD SPEI time series between three pairs of distributions, GLO-GEV, GLO-PE3, and GEV-PE3, showed that there were larger differences in SPEI values between GLO and the other two distributions than the difference between GEV and PE3. SPEI results fitted with GLO resulted in lower threshold exceedances of extreme SPEI (-/+2) than GEV and GLO. A comparison between CMIP5 SPEI values fitted with GLO (SPEI-GLO) and PE3 (SPEI-PE3) showed that there were seasonal and spatial variations in results, although the use of multi-model ensembles reduced these differences. As was the case for CMIP5 SPEI-GLO projections, projected annual changes in CMIP5 SPEI-PE3 show an increase in drying at the surface in central/southern Canada. Overall, the study recommends using PE3 or GEV for SPEI analysis for Canada.

Evaluation of crop model prediction and uncertainty using Bayesian parameter estimation and Bayesian model averaging

A recent trend in crop modeling has been the use of multi-model ensembles (MMEs) for impact assessment, especially as it relates to climate change. Studies have shown that, compared to individual models, the mean or median of a MME is a better predictor that is more accurate in making predictions and capable of providing model uncertainty information. In previous studies that used MMEs, each individual model was assigned an equal weight by simply averaging the predictions over all the models. Here we adopted a different method of creating MMEs, namely using Bayesian model averaging (BMA), which assigns different weights to individual models according to their performance of reproducing historical data. The main purpose of this study is to illustrate how BMA can be applied to crop models, and to compare its performance with simple model averaging. In addition, we also illustrate a full chain of analysis for crop model ensembles, which includes parameter estimation, ensemble creation and evaluation, uncertainty quantification and evaluation. The approaches were implemented with simulations of rice phenology using a small, three-member model ensemble. The results showed that the ensemble model e-BMA had nearly the same prediction accuracy as the best single model and that it predicts quite a bit better than the simply averaging ensemble named e-equal here. Squared bias was found to be the largest contributor to overall prediction uncertainty, both for the individual models and the ensemble models, and thus should be the first priority for model improvement. The estimated prediction uncertainties were larger than the variance of the observations. In the future, BMA should be tested more widely.

Multi-model ensemble simulated non-point source pollution based on Bayesian model averaging method and model uncertainty analysis.

Watershed models are cost-effective and powerful tools for evaluating and controlling non-point source pollution (NPSP), while the reliability of watershed models in a management context depends largely on inherent uncertainties in model predictions. The objective of this study is to present the use of multi-model ensemble applied to streamflow, total nitrogen (TN), and total phosphorus (TP) simulation and quantify the uncertainty resulting from model structure. In this study, three watershed models, which have different structures in simulating NPSP, were selected to conduct watershed monthly streamflow, TN load, and TP load ensemble simulation and 90% credible intervals based on Bayesian model averaging (BMA) method. The result using the observed data of the Yixunhe watershed revealed that the coefficient of determination and Nash-Sutcliffe coefficient of the BMA model simulate streamflow, TN load, and TP load were better than that of the single model. The higher the efficiency of a single model is, the greater the weight during the BMA ensemble simulation is. The 90% credible interval of BMA has a high coverage of measured values in this study. This indicates that the BMA method can not only provide simulation with better precision through ensemble simulation but also provide quantitative evaluation of the model structure through interval, which could offer rich information of the NPSP simulation and management.

Analyses of rainfall extremes in East Africa based on observations from rain gauges and climate change simulations by CORDEX RCMs

This study derived twelve Extreme Rainfall Indices (ERIs) such as the Maximum Dry Spell (MDS) and Maximum Wet Spell (MWS) from daily rainfall observed over the period 1961–1990 at nine locations across East Africa. Capacity of six CO-ordinated Regional Climate Downscaling EXperiment (CORDEX) Africa Regional Climate Models (RCMs) driven by twenty six Climate Model Intercomparison Project phase 5 (CMIP5) General Circulation Models (GCMs) to reproduce the observed ERIs with respect to long-term mean and trends was evaluated. Four RCMs and their five driving GCMs were further analyzed with respect to ERIs. Ensemble means of the RCMs' biases in simulating trends in several ERIs were of magnitudes above 50%. On average, biases in reproducing long-term mean were smaller than those for trends in ERIs. The difference between the performances of RCMs and GCMs depended on the selected RCM–GCM pair. The ensemble means of the RCMs reproduced observed ERIs better than the individual RCMs corroborating that the use of multi-model ensembles can boost credibility of climate change simulations and projections. The RCMs performed better than their driving GCMs in reproducing MDS. The biases of both the RCMs and GCMs were smaller in reproducing the MWS than MDS. Nonetheless, in reproducing observed MWS, the ensemble mean of RCMs' biases was slightly larger than that of the driving GCMs indicating possible adding up of the uncertainties from the GCMs and RCMs. Suggested RCMs' improvements regarding aerosol impacts on rainfall include adding missing constituents (like nitrate), and refining the crudely represented components. RCMs also require high resolution description (in both space and time) of land use types, land surface covers and characteristics as well as landscape heterogeneity. The GCMs to be used as the initial and lateral boundary conditions for the RCMs require improvement in their representation of key dynamical and thermodynamical feedbacks in the Tropical Indian Ocean.

Towards a comprehensive characterization of evidence in synthesis assessments: the climate change impacts on the Brazilian water resources

The Intergovernmental Panel on Climate Change (IPCC) has put a lot of efforts to describe uncertainties and to judge the confidence level of its major conclusions. Despite a guidance to communicate uncertainty, the assignment of confidence is not sufficiently clear and, thus, hard to be reproduced by the extern community. By conducting a synthesis assessment about the impacts of climate change on the Brazilian water resources, we identified an opportunity to illustrate the characterization of evidence as adopted in IPCC reports. We propose a method to describe the evidence from model outputs wherein the quality and amount of studies, as well as the consistency among their conclusions, are subject of a transparent rating procedure. In summary, the more comprehensive the study in sampling uncertainties, the higher its quality. Likewise, the amount and consistency among conclusions is assigned in a systematic way. The method is applied for synthesizing a collection of 42 peer-reviewed articles. It reveals important aspects about the evidence of the potential impacts of climate change in the Brazilian water resources, such as changes into a drier hydrological regime. However, the use of multi-model ensemble, the evaluation of models, and the observational data is limited. The proposed method enables consistent communication of the degree of evidence in a transparent, traceable, and comprehensive fashion. The method can be used as a tool to support experts on their judgment. The approach is reproducible and can guide synthesis work not only in Brazil but anywhere else.

Integrating climate models into hydrological modelling: What’s going on in Brazil?

ABSTRACT Studies integrating climate modelling output into hydrological modelling have grown substantially in the last two decades worldwide; however, there has not been a systematic review about those applications in the Brazilian territory. The aim of this study is to identify how the scientific community has been dealing with the topic in Brazil. The study is based on a systematic review of available peer-reviewed literature. We identify regions and socioeconomic sectors of interest and propose a method to evaluate the methodological consistency of the studies with the current state-of-the-art. The review shows that the topic has grown substantially in this decade, reaching 63 documents until 2018. The sectors under highest concern are the hydropower and the drinking water supply. The Paraná and Atlântico Nordeste Oriental hydrographic regions received great attention; whereas the Atlântico Sudeste did not. In terms of methodology, the use of multi-model ensemble leaves room for improvement. The results suggest a lack of human resources and access to computational infrastructure to handle climate data. Given the current challenges that Brazilian science is facing, we suggest the synchronization of efforts among research institutions. This systematic review provides information to help guiding decision makers to improve the topic in Brazil.

Multi-model ensembles for assessment of flood losses and associated uncertainty

Abstract. Flood loss modelling is a crucial part of risk assessments. However, it is subject to large uncertainty that is often neglected. Most models available in the literature are deterministic, providing only single point estimates of flood loss, and large disparities tend to exist among them. Adopting any one such model in a risk assessment context is likely to lead to inaccurate loss estimates and sub-optimal decision-making. In this paper, we propose the use of multi-model ensembles to address these issues. This approach, which has been applied successfully in other scientific fields, is based on the combination of different model outputs with the aim of improving the skill and usefulness of predictions. We first propose a model rating framework to support ensemble construction, based on a probability tree of model properties, which establishes relative degrees of belief between candidate models. Using 20 flood loss models in two test cases, we then construct numerous multi-model ensembles, based both on the rating framework and on a stochastic method, differing in terms of participating members, ensemble size and model weights. We evaluate the performance of ensemble means, as well as their probabilistic skill and reliability. Our results demonstrate that well-designed multi-model ensembles represent a pragmatic approach to consistently obtain more accurate flood loss estimates and reliable probability distributions of model uncertainty.

Estimating uncertainty in crop model predictions: Current situation and future prospects

In this introductory paper to the special issue on crop model prediction uncertainty, we present and compare the methodological choices in the studies included in this issue, and highlight some remaining challenges. As a common framework for all studies, we define prediction uncertainty as the distribution of prediction error, which can be written as the sum of a bias plus a predictor uncertainty term that represents the random variation due to uncertainty in model structure, model parameters or model inputs. Several themes recur in many of the studies: Use of multi-model ensembles (MMEs) to quantify model structural uncertainty; Emphasis on uncertainty in those inputs related to prediction of regional results or climate change impact assessment; Simultaneous consideration of multiple sources of uncertainty; Emphasis on exploring the variability of uncertainty over space or time; Use of sensitivity analysis techniques to disaggregate the separate contributions to prediction uncertainty. Relatively new approaches include the estimation of both the bias and predictor uncertainty terms of prediction error, the construction of MMEs specifically designed to explore the uncertainty in model structure, the use of emulators for sensitivity analysis and the exploration of ways to reduce prediction uncertainty other than through model improvement. Major remaining challenges are standardization of approaches to quantifying uncertainty in model structure, parameters and inputs, going beyond studies of specific sources of uncertainty to estimation of overall prediction uncertainty, comparing and combining validation and uncertainty studies, and evaluation of uncertainty estimates. Looking forward, we suggest that assessment of prediction uncertainty should be a standard part of any modelling project. The studies here will contribute toward that goal.

Reliability and usability of tourism climate indices

Tourism climate indices (TCI) are commonly used to describe the climate conditions suitable for tourism activities, from the planning, investment or daily operations perspectives. A substantial amount of research has been carried out, in particular with respect to new indices formulae adapted to specific tourism products, and parameters and their weighting, taking into account surveys on the stated preferences of tourists, especially in terms of comfort. This paper illustrates another field of research, which seeks to better understand the different sources of uncertainty associated with indices. Indeed, slight differences in formula thresholds, variations in computation methods, and also the use of multimodel ensembles create nuances that affect the ways in which indices projections are usually presented. Firstly, we assess the impact of differences in preference surveys on the definition of indices thresholds, in particular for thermal comfort. Secondly, we compare computation methods for France, showing the need to better specify detailed data sources and their use to ensure the comparability of results. Thirdly, using multimodel ensembles for the Mediterranean basin, we assess the uncertainty inherent in long-term projections, which are used in modelling the economic impact of climate change. This paper argues in favour of a more cautious use of tourism comfort indices, with more consideration given to the robustness of data (validation, debiasing, uncertainty assessment, etc.) and users’ needs, from the climate services perspective.

Use of crop simulation modelling to aid ideotype design of future cereal cultivars.

A major challenge of the 21st century is to achieve food supply security under a changing climate and roughly a doubling in food demand by 2050 compared to present, the majority of which needs to be met by the cereals wheat, rice, maize, and barley. Future harvests are expected to be especially threatened through increased frequency and severity of extreme events, such as heat waves and drought, that pose particular challenges to plant breeders and crop scientists. Process-based crop models developed for simulating interactions between genotype, environment, and management are widely applied to assess impacts of environmental change on crop yield potentials, phenology, water use, etc. During the last decades, crop simulation has become important for supporting plant breeding, in particular in designing ideotypes, i.e. 'model plants', for different crops and cultivation environments. In this review we (i) examine the main limitations of crop simulation modelling for supporting ideotype breeding, (ii) describe developments in cultivar traits in response to climate variations, and (iii) present examples of how crop simulation has supported evaluation and design of cereal cultivars for future conditions. An early success story for rice demonstrates the potential of crop simulation modelling for ideotype breeding. Combining conventional crop simulation with new breeding methods and genetic modelling holds promise to accelerate delivery of future cereal cultivars for different environments. Robustness of model-aided ideotype design can further be enhanced through continued improvements of simulation models to better capture effects of extremes and the use of multi-model ensembles.

Evaluation and application of Bayesian multi-model estimation in temperature simulations

Use of multi-model ensembles from global climate models to simulate the current and future climate change has flourished as a research topic during recent decades. This paper assesses the performance of multi-model ensembles in simulating global land temperature from 1960 to 1999, using Nash-Sutcliffe model efficiency and Taylor diagrams. The future trends of temperature for different scales and emission scenarios are projected based on the posterior model probabilities estimated by Bayesian methods. The results show that ensemble prediction can improve the accuracy of simulations of the spatiotemporal distribution of global temperature. The performance of Bayesian model averaging (BMA) at simulating the annual temperature dynamic is significantly better than single climate models and their simple model averaging (SMA). However, BMA simulation can demonstrate the temperature trend on the decadal scale, but its annual assessment of accuracy is relatively weak. The ensemble prediction presents dissimilarly accurate descriptions in different regions, and the best performance appears in Australia. The results also indicate that future temperatures in northern Asia rise with the greatest speed in some scenarios, and Australia is the most sensitive region for the effects of greenhouse gas emissions. In addition to the uncertainty of ensemble prediction, the impacts of climate change on agriculture production and water resources are discussed as an extension of this research.

Uncertainty in simulating wheat yields under climate change

Projections of climate change impacts on crop yields are inherently uncertain(1). Uncertainty is often quantified when projecting future greenhouse gas emissions and their influence on climate(2). However, multi-model uncertainty analysis of crop responses to climate change is rare because systematic and objective comparisons among process-based crop simulation models(1,3) are difficult(4). Here we present the largest standardized model intercomparison for climate change impacts so far. We found that individual crop models are able to simulate measured wheat grain yields accurately under a range of environments, particularly if the input information is sufficient. However, simulated climate change impacts vary across models owing to differences in model structures and parameter values. A greater proportion of the uncertainty in climate change impact projections was due to variations among crop models than to variations among downscaled general circulation models. Uncertainties in simulated impacts increased with CO2 concentrations and associated warming. These impact uncertainties can be reduced by improving temperature and CO2 relationships in models and better quantified through use of multi-model ensembles. Less uncertainty in describing how climate change may affect agricultural productivity will aid adaptation strategy development and policymaking.

On the interpretation of constrained climate model ensembles

An ensemble of models can be interpreted in two ways. The first treats each model as an approximation of the true system with some random error. Alternatively, the true system can be interpreted as a sample drawn from a distribution of models, such that model and truth are statistically indistinguishable. Both interpretations are ubiquitous and have different consequences for the uncertainty of model projections, but are rarely defended. Here we argue that the two seemingly conflicting views are in fact complementary, and the interpretation of the ensemble may evolve seamlessly from the former to the latter. We show some ‘truth plus error’ like properties exist for historical and present day climate simulations in the CMIP archive, and that they can be explained by the ensemble design and tuning to observations, although both models and tuning are imperfect. For future projections, structural differences in model response arise which are independent of the present day state and thus the ‘indistinguishable’ interpretation is increasingly favored. Our inability to define performance metrics that identify ‘good’ and ‘bad’ models can be explained by the models having largely exploited the available observations. The remaining model error is largely structural and the observations are often uninformative to further reduce model biases or reduce the range of projections covered by the ensemble. The discussion here is motivated by the use of multi model ensembles in climate projections, but the arguments are generic to any situation where multiple different models constrained by observations are used to describe the same system.

Monsoons: prediction, variability and impact

It is estimated that more than 60% of the world's population depends heavily on monsoon rainfall for their subsistence. While a poor monsoon season can trigger food crisis and have a negative impact on a country's economy, an exceptional monsoon can have disastrous consequences on people's lives and property due to floods and landslides triggered by torrential rains. Even small variations in rainfall relative to the average expected monsoon rainfall can have a dramatic effect. Crop yields and water resources are controlled by the cycles of monsoon circulation and it is of no surprise that analysis of monsoon variability and monsoon prediction on monthly to seasonal time scales are extremely important scientific activities. Accurate forecasts of daily precipitation during the monsoon season are also essential in order to issue effective warnings of severe weather events. This special issue assembles a series of articles on the state of the art scientific understanding of prediction, variability and the impact of monsoons. The first set of papers (Agnihotri and Mohapatra, Sardar et al., and P. Kumar et al.) looks at daily and monthly forecasts of monsoon rainfall, as well as forecasts of upper tropospheric humidity fields. The sensitivity to different convection schemes is assessed comparing monthly rainfall forecasts to satellite derived observations. The understanding of forecast uncertainty linked to different parameterization of convection is crucial for a better representation of physical processes in numerical weather prediction models and thus better predictions. The use of multi-model ensembles in order to account for forecast uncertainty has become increasingly popular in current scientific practice and this special issue includes four papers on the topic (A. Kumar et al., Goswami et al., Singh et al. and Samala et al.). The performance of a simple ensemble mean and a weighted average, whereby weights are calculated using neural network concepts, are discussed. Moreover, the improvements of post-processed model output using a Canonical Correlation Analysis (CCA) is compared to direct model output. The multi-model ensemble is also used to examine weaknesses in forecasts of circulation anomalies during weak Indian monsoon years. The next set of papers (Ha et al., Mujumdar et al. and Rajeevan et al.) includes a review of the multi-scale variability of the East Asian monsoon, where possible links between climate change and monsoon variability are explored. The variability of the Indian monsoon is looked at in relation to La Niña. Moreover, the variability of the Northeast monsoon over India is discussed together with its predictability. The final collection of papers describes droughts resulting from changes in the monsoon regimes in South America and India (Coelho et al. and M. Naresh Kumar et al.). In the case of South America, the large-scale oceanic and atmospheric conditions are discussed as possible mechanisms associated with drought conditions. The analysis of time series of the Standardized Precipitation Index for India has shown that the drought frequency has increased in recent years. There is no doubt that further scientific research on monsoon processes and variability, and their predictability and impacts, is still urgently required. The complexity of physical processes and atmospheric forcings renders the task of understanding monsoons challenging for the scientific community, and this special issue is a contribution towards this effort.

Seasonal streamflow forecast: a GCM multi-model downscaling approach

This work investigates the predictability of seasonal to inter-annual streamflow over several river basins in Norway through the use of multi-model ensembles. As general circulation models (GCMs) do not explicitly simulate streamflow, a statistical link is made between GCM-forecast fields generated in December and average streamflow in the melting season May–June. By using the Climate Predictability Tool (CPT) three models were constructed and from these a multi-model was built. The multi-model forecast is tested against climatology to determine the quality of the forecast. Results from the forecasts show that the multi-model performs better than the individual models and that this method shows improved forecast skills if compared to previous studies conducted in the same basins. The highest forecast skills are found for basins located in the southwest of Norway. The physical interpretation for this is that stations on the windward side of the Scandinavian mountains are exposed to the prevailing winds from the Atlantic Ocean, a principal source of predictive information from the atmosphere on this timescale.

Use of multi-model ensembles from global climate models for assessment of climate change impacts

Multi-model ensembles of climate predictions constructed by running several global climate models for a common set of experiments are available for impact assessment of climate change. Multi-model ensembles emphasize the uncertainty in climate predictions resulting from structural differences in the global climate models as well as uncertainty due to variations in initial conditions or model parameterisations. This paper describes a methodology of using multi-model ensembles from global climate models for impact assessments which require local-scale climate scenarios. The approach is based on the use of a weather generator capable of generating the localscale daily climate scenarios used as an input by many process-based impact models. A new version of the LARS-WG weather generator, described in the paper, incorporates climate predictions from 15 climate models from the multi-model ensemble used in the IPCC Fourth Assessment Report (AR4). The use of the AR4 multi-model ensemble allows assessment of the range of uncertainty in the impacts of climate change resulting from the uncertainty in predications of climate. As an example, the impact of climate change on the probability of heat stress during flowering of wheat, which can result in significant yield losses, was assessed using local-scale climate scenarios in conjunction with a wheat simulation model at 4 European locations. The exploitation of much larger perturbed physics ensembles is also discussed.