Global gridded crop model evaluation: benchmarking, skills, deficiencies and implications

Christoph Müller,Thomas A M Pugh,Joshua Elliott,Allard De Wit,Delphine Deryng,Hong Yang,Wenfeng Liu,Ashwan Reddy,Gen Sakurai,Xuhui Wang,Carol X Song,Philippe Ciais,Steven Hoek,James Chryssanthacopoulos,Almut Arneth,Nikolay Khabarov,Erwin Schmid,Stefan Olin,Michael Glotter,Alex C Ruane,Christian Folberth,Cynthia Rosenzweig,Toshichika Iizumi,D K Ray ,R C Izaurralde ,Rastislav Skalský ,Juraj Balkovič ,P Lawrence ,Curtis D Jones

doi:10.5194/gmd-10-1403-2017

Abstract

Abstract. Crop models are increasingly used to simulate crop yields at the global scale, but so far there is no general framework on how to assess model performance. Here we evaluate the simulation results of 14 global gridded crop modeling groups that have contributed historic crop yield simulations for maize, wheat, rice and soybean to the Global Gridded Crop Model Intercomparison (GGCMI) of the Agricultural Model Intercomparison and Improvement Project (AgMIP). Simulation results are compared to reference data at global, national and grid cell scales and we evaluate model performance with respect to time series correlation, spatial correlation and mean bias. We find that global gridded crop models (GGCMs) show mixed skill in reproducing time series correlations or spatial patterns at the different spatial scales. Generally, maize, wheat and soybean simulations of many GGCMs are capable of reproducing larger parts of observed temporal variability (time series correlation coefficients (r) of up to 0.888 for maize, 0.673 for wheat and 0.643 for soybean at the global scale) but rice yield variability cannot be well reproduced by most models. Yield variability can be well reproduced for most major producing countries by many GGCMs and for all countries by at least some. A comparison with gridded yield data and a statistical analysis of the effects of weather variability on yield variability shows that the ensemble of GGCMs can explain more of the yield variability than an ensemble of regression models for maize and soybean, but not for wheat and rice. We identify future research needs in global gridded crop modeling and for all individual crop modeling groups. In the absence of a purely observation-based benchmark for model evaluation, we propose that the best performing crop model per crop and region establishes the benchmark for all others, and modelers are encouraged to investigate how crop model performance can be increased. We make our evaluation system accessible to all crop modelers so that other modeling groups can also test their model performance against the reference data and the GGCMI benchmark.

Highlights

Agriculture is fundamental to human life and our ability to understand how agricultural production responds to changes in environmental conditions and land management has long been a central question in science (Russell, 1966; Spiertz, 2014)
Maize, wheat and soybean simulations of many global gridded crop models (GGCMs) are capable of reproducing larger parts of observed temporal variability (time series correlation coefficients (r) of up to 0.888 for maize, 0.673 for wheat and 0.643 for soybean at the global scale) but rice yield variability cannot be well reproduced by most models
In the analysis we largely focus on time series correlation of simulated and reference crop yields, given that the main application of gridded crop models at the global scale is related to studies on climate change impacts, where we expect models to respond reasonably to changes in atmospheric conditions

Summary

Introduction

Agriculture is fundamental to human life and our ability to understand how agricultural production responds to changes in environmental conditions and land management has long been a central question in science (Russell, 1966; Spiertz, 2014). In the face of continued population growth, economic development and the emergence of global-scale phenomena that affect agricultural productivity (most prominently climate change) crop models are applied at the global scale (Rosenzweig and Parry, 1994). Given the importance of climate change and the central interest in agriculture, global-scale crop model applications have been increasingly used to address a wide range of questions, beyond pure crop yield simulations (e.g., Bondeau et al, 2007; Del Grosso et al, 2009; Deryng et al, 2014; Osborne et al, 2013; Pongratz et al, 2012; Rosenzweig et al, 2014; Stehfest et al, 2007; Wheeler and von Braun, 2013). If models are applied at the global scale, they need to be assessed at the scale of interpretation, which ranges from gridded to national or regional aggregates (Elliott et al, 2014a; Fader et al, 2010; Müller and Robertson, 2014; Nelson et al, 2014a, b; Osborne et al, 2013)

Objectives

Methods

Results

Discussion

Conclusion