Validating land change models based on configuration disagreement

Abstract

Land change models are increasingly being employed to predict future landscapes and influence policy and decision-making. To ensure the highest model accuracy, validation methods have become commonplace following a land change simulation. The most common validation method employed uses quantity and allocation disagreement. However, these current measures may not account for differences in the configurations of land change, placing them in potential conflict with the principals of heterogeneity and spatial patterning of landscape ecology. We develop a new metric, termed configuration disagreement, designed to focus on the size, shape, and complexity of land change simulations. Using this metric, we demonstrate the value of including errors of configuration disagreement – in addition to quantity and allocation error – in the assessment of land change models. Four computational experiments of land change that vary only in spatial pattern are developed using the FUTURES land change model. For each experiment, configuration disagreement and the traditional validation metrics are computed simultaneously. Results indicate that models validated only with consideration of quantity and allocation error may misrepresent, or not fully account for, spatial patterns of landscape change. The research objective will ultimately guide which component, or components, of model disagreement are most critical for consideration. Yet, our work reveals why it may be more helpful to validate simulations in terms of configuration accuracy. Specifically, if a study requires accurately modeling the spatial patterns and arrangements of land cover. Configuration disagreement could add critical information with respect to a model's simulated changes in size, shape, and spatial arrangements, and possibly enhance ecologically meaningful land change science.