Continual learning for scene classification of high resolution remote sensing images

Abstract

Listen

Translate article

Deep learning has achieved great success in optical image classification tasks including high resolution remote sensing image classification and segmentation. However, recent deep learning models usually use static training methods, therefore, long time and huge data are needed during training. The traditional end-to-end deep learning models are difficult to achieve incremental learning, which is learning new knowledge without forgetting learned knowledge. The continual learning was recently used to overcome this catastrophic forgetting problem. It is mainly on continual updating and optimizing the learning model to learn knowledge incrementally. In this paper, we proposed a continual learning model for the classification of high resolution remote sensing image scenes. It is a memory consolidation method using fixed representation. Fixed representation learning divides the network into three parts to be trained separately to adapt for continual learning scenarios, and the memory consolidation method solves the catastrophic forgetting problem through distillation loss, optimal sample replay, and classification layer correction. We compared the proposed method with the existing continual learning methods elastic weight consolidation (EWC), and learning without forgetting (LWF) methods, the experimental results show that the proposed method is effective for remote sensing scenarios classification, and has good performance on suppressing catastrophic forgetting. We also comprehensively analyzed the proposed method in multiple angles, and it is seen that there is still room for optimizing the proposed method in terms of model architecture, loss value, and training method.