Sketch-based 3D shape retrieval via teacher–student learning

Abstract

One of the main difficulties of sketch-based 3D shape retrieval is the significant cross-modal difference between 2D sketches and 3D shapes. Most previous works adopt one-stage methods to directly learn the aligned common embedding space of sketches and shapes by a shared classifier. However, the intra-class difference of the sketch is more significant than the shape, harming the feature learning of 3D shapes when the two modalities are considered under the shared classifier. This issue harms the discrimination of the learned common embedding space. This paper proposes a novel two-stage method to learn a common aligned embedding space via teacher–student learning to address the issue. Specifically, we first employ a classification network to learn the discriminative features of shapes. The learned shape features are considered a teacher to guide the feature learning of sketches. Moreover, we design a guidance loss to achieve the feature transfer with semantic alignment. The proposed method achieves an effective, aligned cross-modal embedding space. Experiments on three public benchmark datasets prove the superiority of the proposed method over state-of-the-art methods.