M2FN: A Multilayer and Multiattention Fusion Network for Remote Sensing Image Scene Classification

Abstract

Deep convolutional neural networks (CNNs) have made great progress in remote sensing (RS) image scene classification. However, by visualizing the learned feature maps, we find that the popular CNN of ResNet can capture incomplete and inaccurate semantic information for classifying scene images with complex spatial distributions and varying object scales. In this letter, we propose a multilayer and multiattention fusion network (M2FN) to alleviate this issue. Specifically, we first introduce a multilayer adaptive feature fusion (MLAFF) module to model the information interaction between different layers and enhance the network’s multiscale representation ability. Then, we design a multidimensional attention (MA) module to weight the multilayer fused features by comprehensively considering their interdependencies between all possible dimensions. The proposed MA module extends the traditional spatial and channel attentions to a more comprehensive one. Experiments on two benchmark data sets demonstrate the superiority of M2FN for RS scene classification over many state-of-the-art methods.