Multi-scale fusion for few-shot remote sensing image classification

Abstract

ABSTRACT Few-shot image classification is currently valuable research in the field of remote sensing image applications. The difficulty in obtaining training data for a large number of labelled remote sensing images leads to difficulties in applying traditional deep learning-based remote sensing image classification methods. Existing few-shot remote sensing image classification methods usually use a large number of base datasets to train network models in the pre-training phase and perform few-shot classification tasks after model fine-tuning on new datasets in the meta-testing phase. However, an insufficient number of remote sensing images lead to inaccurate features being extracted by the model, and the large amount of new class data for fine-tuning is also not easily accessible. In addition, using a pre-trained model to extract features from the new dataset will create a”negative transfer” problem. In this paper, we aim to address the above challenges in two ways. First, we use a subset of ImageNet, a readily available few-shot natural image dataset, for model pre-training, and eliminate the fine-tuning operation to simulate a real-world application scenario where remote sensing data is extremely scarce. Second, due to the significant differences in scale and style between the ImageNet dataset and the remote sensing dataset, which lead to a serious negative transfer problem in the model, we design a multi-scale feature fusion module to comprehensively decide the labels of query samples considering all scales to compensate for the scale differences and alleviate the”negative transfer” problem. We conducted experiments on four benchmark remote sensing datasets and achieved satisfactory performance.