A network-based toolkit for evaluation and intercomparison of weather prediction and climate modeling

Abstract

Model evaluation is a critical component in the development and applications of environmental modeling systems. Conventional metrics such as Pearson product-moment correlation coefficient (r), root-mean-square error (RMSE), and mean absolute error (MAE), albeit process-based and limited to point-to-point statistical comparison, have been widely used in model evaluations. In this study, we propose a network-based toolkit for evaluation of model performance and multi-model comparisons with applications to weather prediction and climate modeling. The model outputs are topologically quantified through a range of network metrics to provide a holistic measure of system dynamics. We first use this toolkit to evaluate the performance of air temperature simulated by the Weather Research and Forecasting model with station measurements over the contiguous United States (CONUS). Results of network analysis suggest a good match between simulation and measurement, as indicated by conventional metrics (r, RMSE, and MAE) as well. The sensitivity of these network metrics is then analyzed based on CONUS station measurements with additive random errors using Monte Carlo simulations. Network metrics show more sensitive and highly nonlinear responses to increasing random errors as compared to conventional ones. Moreover, we use the new toolkit for intercomparison of the downscaled historical air temperature outputs from four global climate models. The similarity in both metrics and spatial structure highlights the capability of network analysis for capturing system dynamics in models alike. The network theory is therefore promising for evaluation and intercomparison of various environmental modeling systems with complex dynamics.