Spatial Validation of Agent-Based Models

Abstract

This paper adapts an existing techno–social agent-based model (ABM) in order to develop a new framework for spatially validating ABMs. The ABM simulates citizen opposition to locally unwanted land uses, using historical data from an energy infrastructure siting process in Southern California. Spatial theory, as well as the model’s design, suggest that adequate validation requires multiple tests rather than relying solely on a single test-statistic. A pattern-oriented modeling approach was employed that first mapped real and simulated citizen comments across the US Census tract. The suite of spatial tests included Global Moran’s I, complemented with bivariate correlations, as well as the local indicators of spatial association (LISA) test. The global tests showed the model explained up to 65% of the variation in the historical data for US Census tract-level citizen comments on a locally unwanted land use. These global tests were also found helpful to inform the model’s calibration for the current application. The LISA results were even stronger, showing that the model predicted citizen comment clustering correctly in five of six Census tracts. It slightly over predicted comments further away from the land use. The LISA results and pattern-oriented modeling validation techniques identified theoretical factors to improve the modeling specification in future applications. The combined suite of validation techniques helped improve confidence in the model’s predictions.