A deformable CNN-based triplet model for fine-grained sketch-based image retrieval

Abstract

With the popularity of electronic touch-screen and pressure sensing devices, fine-grained sketch based image retrieval (FG-SBIR) has become a research hotspot. In this paper, we stress the core problems of FG-SBIR: a. how to reduce the difference between the non-homogenous of heterogeneous media, and b. how to improve the distinguishability of sketch features. Specifically, a sketch generation model is first proposed to replace the conventional pre-processing of roughly extracting image edges, moreover, this model can alleviate the dilemma of sketch data scarcity. We then construct a novel FG-SBIR model which takes advantage of deformable convolutional neural network while taking into consideration of semantic attributes together. In addition, we build a fine-grained clothing sketch-image dataset, which has rich attribute annotations, for the first time. Extensive experiments exhibit that our proposed model achieves a better performance in improving the retrieval accuracy over the state-of-the-art baselines.