Multilabel Remote Sensing Image Annotation With Multiscale Attention and Label Correlation

Abstract

Deep-learning-based multilabel image annotation is receiving increasing attention in the field of remote sensing due to the great success of deep networks in single-label remote sensing image classification. Compared with those low-level features, the features extracted by the convolutional neural network (CNN) are more informative and can alleviate the problem of semantic gap. However, the CNN model tends to ignore the smaller objects when objects of different sizes exist in an image. In addition, how to efficiently leverage the correlation among multiple labels to enhance annotation performance remains an open issue. In this article, we propose an end-to-end deep learning framework for multilabel remote sensing image annotation. The framework is composed of a multiscale feature fusion module, a channel-spatial attention learning module, and a label correlation extraction module. The multiscale features from different layers of a CNN model are first fused and refined by using a channel-spatial attention mechanism. Then, the label correlation information is extracted from a label co-occurrence matrix and embedded into the multiscale attentive features to increase the discriminative ability of the resulting image features. The experiments on two benchmark datasets demonstrate the superiority of the proposed method in comparison with the state-of-the-art methods.