LIL: Lightweight Incremental Learning Approach Through Feature Transfer for Remote Sensing Image Scene Classification

Abstract

Existing deep learning models usually assume that all data obeys independent identically distribution, which is unreasonable in remote sensing. Due to the differences in camera parameters, spectral ranges, resolutions, and so on, the images acquired by remote sensing sensors may be greatly diverse, causing models to face catastrophic forgetting when they are trained on new data only. Thus, incremental learning is introduced. An ideal incremental learning model should be expanded as the number of tasks increases, so as to have enough ability to adapt to the changes in data. However, existing approaches normally expand heavy modules for each task, making the holistic models cumbersome. In this article, a lightweight incremental learning approach (LIL) is proposed for remote sensing image scene classification. We replace the role of the feature extractor with extracting features of a single task instead of task-sharing features of all tasks to lighten the backbone. In addition, we propose a light feature transfer module (FTM) to realize the alignment of data distributions between different tasks in the feature domain. Furthermore, dual-constraint loss with knowledge distillation and adversarial learning is introduced to promote the mapping and alignment of data distributions at both the feature level and the semantic level. In LIL, only a tiny FTM and a classifier are added to the model when the model learns a new task. Experimental results show that our approach with a small number of parameters outperforms state-of-the-art approaches for incremental learning on both a single dataset and a sequence of multiple datasets.