Incremental Learning With Open-Set Recognition for Remote Sensing Image Scene Classification

Abstract

Image scene classification aiming to assign specific semantic labels for each image is vital important for the applications of remote sensing (RS) data. In real world, since the observation environment is open and dynamic, RS images are collected sequentially and the numbers of images and classes grow rapidly over time. Most existing scene classification methods are offline learning algorithms which are inefficient and unscalable for this scenario. In this paper, an incremental learning with open set recognition framework is proposed for RS image scene classification in the open and dynamic environment, called ILOSR, which can identify the unknown classes from a stream of data and learn these new classes incrementally. Specifically, a controllable convex hull-based exemplar selection strategy is designed to address the catastrophic forgetting issue in incremental learning, which can reduce training time and memory footprint effectively. In addition, a new loss function based on prototype learning and uncertainty measurement is proposed for open set recognition to enhance the inter-class discrimination and intra-class compactness of the learned deep features. Experimental results on real remote sensing datasets demonstrate that the proposed method can not only outperform the state-of-the-art approaches on offline classification, incremental learning and OSR problem separately, but also achieve better and more stable performance in the experiments for incremental learning with OSR.