Evaluating the ability of numerical models to capture important shifts in environmental time series: A fuzzy change point approach

M.J Hollaway,P.A Henrys,R Killick,A Leeson,J Watkins

doi:10.1016/j.envsoft.2021.104993

Abstract

Numerical models are essential tools for understanding the complex and dynamic nature of the natural environment. The ability to evaluate how well these models represent reality is critical in their use and future development. This study presents a combination of changepoint analysis and fuzzy logic to assess the ability of numerical models to capture local scale temporal events seen in observations. The fuzzy union based metric factors in uncertainty of the changepoint location to calculate individual similarity scores between the numerical model and reality for each changepoint in the observed record. The application of the method is demonstrated through a case study on a high resolution model dataset which was able to pick up observed changepoints in temperature records over Greenland to varying degrees of success. The case study is presented using the DataLabs framework, a cloud-based collaborative platform which simplifies access to complex statistical methods for environmental science applications.

Highlights

The natural environment is a complex system that evolves through time in response to drivers such as climate change, economic change and social change (IPCC, 2018; Schroter et al, 2005)
This study presents a new approach to numerical model evaluation that utilises changepoint analysis to assess the ability of a model derived time series to capture different modes of temporal variability seen in the observed record
A new approach to numerical model evaluation has been developed by utilising a combination of changepoint analysis (using the PELT al gorithm developed by Killick et al (2012)) and fuzzy logic to assess the ability of climate models to capture key events seen in the observed record

Summary

Introduction

The natural environment is a complex system that evolves through time in response to drivers such as climate change, economic change and social change (IPCC, 2018; Schroter et al, 2005). Recent advances in high powered computing have resulted in models that are capable of running at finer spatial and temporal resolutions and/or include more processes, and better represent the dynamic natural environment (Collins et al, 2011; Gutjahr et al, 2019; Hu et al, 2018; Savage et al, 2013; Swart et al, 2019). These developments are impor tant as many environmental processes are local in nature and exhibit high spatial variability, e.g. air pollution episodes, localised heavy rainfall, or ice sheet melt. With this enhanced capability comes increased scru tiny of uncertainty in the model structure, parameters and outputs (Beven, 2006) and how these uncertainties are communicated to model users, developers and decision makers

Methods

Results

Discussion

Conclusion