Multi-label Aerial Image Classification via Adjacency-Based Label and Feature Co-embedding

Abstract

Multi-label image classification is a fundamental task in aerial image processing, which automatically generates image annotations for better image content interpretation. Many existing methods realize multi-label classification through an image level, while they ignore the dependencies among labels and the cross-modal relations between labels and image features. In this paper, we propose a simple and intuitive multi-label classification method via adjacency-based label and feature co-embedding for aerial images. To be specific, we introduce an adjacency-based label embedding module to maintain the original label relationships in the semantic space. A label and feature co-embedding module is designed to enhance the text-image cross-modal interactions and to obtain the attention-based label-specific vectors, which effectively excavate the response relations between labels and images. Experiments on two benchmark aerial image multi-label datasets show that our approach achieves considerable performance compared with seven previous approaches. Besides, visualization analyses indicate the label embeddings learned by our model maintain a meaningful semantic topology, which explicitly exploit label-feature dependencies.