Multiple Semantic Embedding with Graph Convolutional Networks for Multi-Label Image Classification

Abstract

Multi-label image classification focuses on predicting a set of object labels presented in an image, which is a practical yet challenging task since labels are normally co-occurred in an image while mutual interactions among labels and the correspondence between a given image and the corresponding labels are rarely considered in the existing methods. To address above challenges, we propose a Multiple Semantic Embedding model with Graph Convolutional Networks (MSEGCN) for multi-label image classification by capturing important dependencies related labels. Specifically, the proposed MSEGCN leverages graph structure to guide label co-occurrence propagation among different categories to obtain appropriate label representations. Then, by formulating multi-label classification problem as a label ranking problem with the aid of end-to-end convolutional neural network framework, we focus on learning a transform matrix to seek the image-label relevance relations in embedding space. Furthermore, an adaptive weighting strategy is introduced to effectively improve the classification performance. Experimental studies across a wide range of benchmark datasets show that our method achieves highly competitive performances against other state-of-the-art approaches.