Neutral models of landscape change as benchmarks in the assessment of model performance

Abstract

This paper introduces a methodological framework for performance assessment of spatial dynamic models by means of map comparison. The objective is to discern to what extent model performance, expressed by a variety of metrics, can be attributed to endogenously modeled processes or to exogenous model inputs. For this purpose, neutral models of landscape change are introduced that are subject to the same boundary conditions and constraints as the probed model, but otherwise are random except for a reluctance to change. The neutral models serve as benchmark and the difference in performance with the model under investigation can be attributed to the endogenous qualities of the model. Furthermore, the framework makes performance measures over multiple criteria and scales mutually comparable, thus providing insight in strengths and weaknesses of the model. The framework is applied for the performance assessment of a Constrained Cellular Automata land use model for La Réunion (Fr.). Map comparison metrics of land use presence and structure are evaluated at multiple scales. For criteria of land use presence the land use model outperforms the neutral models only at coarse scales, but for criteria of land use structure it performs better on all scales.