Remote Sensing Image Scene Classification Model Based on Dual Knowledge Distillation

Abstract

In the application of remote sensing image (RSI) scene classification, in order to solve the contradiction between the accuracy of Convolutional Neural Network (CNN) and the large amount of model parameters, a novel dual knowledge distillation (DKD) model combining dual attention (DA) and spatial structure (SS) is designed. First, new DA and SS modules are constructed and introduced into ResNet101 and light-weight CNN designed as teacher and student networks respectively. Then, in order to improve its local feature extraction and high-level semantic representation abilities for RSI by transmission the DA and SS knowledge in the teacher network to the student network, we design the corresponding DA and SS distillation losses. The comparative experimental results based on AID and NWPU-45 datasets show that when the training ratio is 20%, the accuracy of the student network after DKD is improved by 7.57% and 7.28% respectively, and in the case of fewer parameters, DKD has higher accuracy than most other methods.