EFCOMFF-Net: A Multiscale Feature Fusion Architecture With Enhanced Feature Correlation for Remote Sensing Image Scene Classification

Abstract

Remote sensing images have the essential attribute of large-scale spatial variation and complex scene information, as well as the high similarity between various classes and the significant differences within same class, which are easy to cause misclassification. To solve this problem, an efficient systematic architecture named EFCOMFF-Net (Multi-scale Feature Fusion Network with Enhanced Feature Correlation) is proposed to reduce the gap among multi-scale features and fuse them to improve the representation ability of remote sensing images. Firstly, to strengthen the correlation of multi-scale features, a Feature Correlation Enhancement Module (FCEM) is specifically developed, which takes the features of different stages of the backbone network as input data to obtain multi-scale features with enhanced correlation. Considering the differences between the shallow features and the deep features, the EFCOMFF-Net-v1 related to shallow features and EFCOMFF-Net-v2 related to deep features with different structures are proposed. Secondly, the designed two versions of the deep learning network focus on the global contour information and need to encode more accurate spatial information. A Feature Aggregation Attention Module (FAAM) is designed and embedded into the network to encode the deep features by applying the spatial information aggregation features. Finally, considering that the simple integration strategy cannot reduce the gap between the shallow multi-scale features and the deep features, a Feature Refinement Module (FRM) is presented to optimize the network. ResNet50, DenseNet121, and ResNet152 are selected to conduct a considerable number of experiments on four datasets, which show the superiority of our method compared to recent methods.