Towards user-adaptive remote sensing: Knowledge-driven automatic classification of Sentinel-2 time series

Abstract

Land cover mapping over large areas is essential to address a wide spectrum of socio-environmental challenges. For this reason, many global or regional land cover products are regularly released to the scientific community. Yet, the remote sensing community has not fully addressed the challenge to extract useful information from vast volumes of satellite data. Especially, major limitations concern the use of inadequate classification schemes and “black box” methods that may not match with end-users conceptualization of geographic features. In this paper, we introduce a knowledge-driven methodological approach to automatically process Sentinel-2 time series in order to produce pre-classifications that can be adapted by end-users to match their requirements. The approach relies on a conceptual framework inspired from ontologies of scientific observation and geographic information to describe the representation of geographic entities in remote sensing images. The implementation consists in a three-stage classification system including an initial stage, a dichotomous stage and a modular stage. At each stage, the system firstly relies on natural language semantic descriptions of time series of spectral signatures before assigning labels of land cover classes. The implementation was tested on 75 time series of Sentinel-2 images (i.e. 2069 images) in the Southern Brazilian Amazon to map natural vegetation and water bodies as required by a local end-user, i.e. a non-governmental organization. The results confirmed the potential of the method to accurately detect water bodies (F-score = 0.874 for bodies larger than 10 m) and map natural vegetation (max F-score = 0.875), yet emphasizing the spatial heterogeneity of accuracy results. In addition, it proved to be efficient to provide rapid estimates of degraded riparian forests at watershed level (R2 = 0.871). Finally, we discuss potential improvements both in the system's implementation, e.g. considering additional characteristics, and in the conceptual framework, e.g. moving from pixel- to object-based image analysis and evolving towards a hybrid system combining data- and knowledge-driven approaches.