Semantic interleaving global channel attention for multilabel remote sensing image classification

Abstract

ABSTRACT Multilabel remote sensing image classification (MLRSIC) has received increasing research interest. Taking the co-occurrence relationship of multiple labels as additional information helps to improve the overall performance. However, current methods only focus on using it to constrain the final feature which is output from a convolutional neural network (CNN). On the one hand, these methods need to exploit the potential of label correlation in feature representation fully. On the other hand, they increase the label noise sensitivity of the system, resulting in poor robustness. In this paper, a novel method called ‘Semantic Interleaving Global chaNnel Attention’ (SIGNA) is proposed for MLRSIC. First, the label co-occurrence graph is obtained according to the statistical information of the training set and fed into a graph neural network (GNN) to generate optimal semantic feature representations of each label. Next, the semantic features are interleaved with visual features which are extracted by CNNs to guide the overall features of the input image transform from the original feature space to the semantic feature space with embedded label relations. Then, global attention triggered by semantic interleaving is used to emphasize visual features in important channels. Finally, to make SIGNA easier to use and more optimized, multihead SIGNA-based feature adaptive weighting networks are proposed as plug-in blocks to plug into any layers of a CNN. For remote sensing images, better classification performance can be achieved by inserting the plug-in blocks into the shallow layers of CNNs. We conducted extensive experimental comparisons on three data sets: UCM, AID and DFC15. Experimental results demonstrate that the proposed SIGNA achieves superior classification performance compared to state-of-the-art (SOTA) methods. Notes that the codes of this paper will be open to the community for reproducibility research.