Semantic annotation in earth observation based on active learning

Abstract

As the data acquisition capabilities of earth observation (EO) satellites have been improved significantly, a large amount of high-resolution images are downlinked continuously to ground stations. The data volume increases rapidly beyond the users’ capability to access the information content of the data. Thus, interactive systems that allow fast indexing of high-resolution images based on image content are urgently needed. In this paper, we present an interactive learning system for semantic annotation and content mining at patch level. It mainly comprises four components: primitive feature extraction including both spatial and temporal features, relevance feedback based on active learning, a human machine communication (HMC) interface and data visualisation. To overcome the shortage of training samples and to speed up the convergence, active learning is employed in this system. Two core components of active learning are the classifier training using already labelled image patches, and the sample selection strategy which selects the most informative samples for manual labelling. These two components work alternatively, significantly reducing the labelling effort and achieving fast indexing. In addition, our data visualisation is particularly designed for multi-temporal and multi-sensor image indexing, where efficient visualisation plays a critical role. The system is applicable to both optical and synthetic aperture radar images. It can index patches and it can also discover temporal patterns in satellite image time series. Three typical case studies are included to show its wide use in EO applications.