Zero-Shot Visual Recognition via Semantic Attention-Based Compare Network

Abstract

Zero-shot visual recognition aims to classify images whose classes are not have been seen in the training stage. Most of current approaches first project the attribute semantic feature and image feature into a common space, then the manually defined similarity measures or linear classifiers are used to recognize the unseen image. Different from the existing study, in this paper, we propose a novel Semantic Attention-based Compare Network (SACN) which is comprised of a visual feature extraction module, a feature fusion module and a similarity compare module. Our SACN has several advantages: (1) In visual feature extraction module, a convolutional neural network (CNN) with multi-losses is introduced to extract more distinguishing visual features. (2) In feature fusion module, an attribute semantic attention mechanism is proposed to associate visual feature and class semantic representation. (3) In similarity compare module, a distance compare network is presented to predict the similarities of the test image with all unseen classes. To evaluate the proposed model, we have conducted extensive experiments on two widely used benchmarks, and both qualitative and quantitative evaluation results have demonstrated the effectiveness of the proposed SACN. More importantly, the proposed method is the 2nd place solution to the AI Challenger 2018 (Global AI Contest).