Assessing the accuracy of multi-temporal built-up land layers across rural-urban trajectories in the United States

Abstract

Global data on settlements, built-up land and population distributions are becoming increasingly available and represent important inputs to a better understanding of key demographic processes such as urbanization and interactions between human and natural systems over time. One persistent drawback that prevents user communities from effectively and objectively using these data products more broadly, is the absence of thorough and transparent validation studies. This study develops a validation framework for accuracy assessment of multi-temporal built-up land layers using integrated public parcel and building records as validation data. The framework is based on measures derived from confusion matrices and incorporates a sensitivity analysis for potential spatial offsets between validation and test data as well as tests for the effects of varying criteria of the abstract term built-up land on accuracy measures. Furthermore, the framework allows for accuracy assessments by strata of built-up density, which provides important insights on the relationship between classification accuracy and development intensity to better instruct and educate user communities on quality aspects that might be relevant to different purposes. We use data from the newly-released Global Human Settlement Layer (GHSL), for four epochs since 1975 and at fine spatial resolution (38m), in the United States for a demonstration of the framework. The results show very encouraging accuracy measures that vary across study areas, generally improve over time but show very distinct patterns across the rural-urban trajectories. Areas of higher development intensity are very accurately classified and highly reliable. Rural areas show low degrees of accuracy, which could be affected by misalignment between the reference data and the data under test in areas where built-up land is scattered and rare. However, a regression analysis, which examines how well GHSL can estimate built-up land using spatially aggregated analytical units, indicates that classification error is mainly of thematic nature. Thus, caution should be taken in using the data product in rural regions. The results can be useful in further improving classification procedures to create measures of the built environment. The validation framework can be extended to data-poor regions of the world using map data and Volunteered Geographic Information.