Transductive semantic knowledge graph propagation for zero-shot learning

Abstract

The domain shift is a very common phenomenon in Zero-shot Learning (ZSL), because the data distribution between the source and target domain might not match well in real scenarios. This paper focuses on dealing with the domain drift problem in ZSL based on the semantic knowledge graph propagation. Our method consists of visual feature extraction (VFE) module, semantic feature extraction module (SFE) and feature mapping module (FM). In VFE module, the high-level visual features are extracted under Convolutional Neural Network (CNN) framework, where unseen samples participate in model training and stay away from seen ones under the premise of distinguishability. SFE module relies on the Graph Convolutional Network (GCN) framework and focuses on the transmission of semantic embeddings. A modified message aggregation and transformation strategy is proposed to effectively relieve the information smoothing phenomenon caused by the increase of GCN layers. In the recognition stage, AutoEncoder framework achieves a two-way visual and semantic interaction to further minimize the domain drift phenomenon. Simulation results on ImageNet dataset have demonstrated the performance of the proposed method.