Domain Adaptation Problem in Sketch Based Image Retrieval

Abstract

In this article, we present two algorithms that discover the discriminative structures of sketches, given pairs of sketches and photos in sketch-based image retrieval (SBIR) scenarios. Unlike the existing approaches, we aim at the few-shot and domain adaptation (DA) problems, and set up a module with canonical correlation analysis (CCA) technique in our algorithms to improve retrieval performance. For single source domain settings, our first algorithm can effectively transfer a classifier trained on a known dataset to a new one. For multisource settings, our second algorithm sophisticatedly combines multisource domain data to yield a classifier on the target domain. To the best of our knowledge, these two works are the first research in SBIR field. Experiments on the Sketchy and TU-Berlin sketch benchmark datasets demonstrate the effectiveness of our algorithms and compelling performance. Compared with the state-of-the-art methods, our algorithms do not use text based semantic information, but achieve competitive results. Furthermore, experiments also turn out that feature representation by available trained deep networks has distinct advantages and combination with traditional machine learning methods brings substantial improvements against the state-of-the-art methods on image retrievals. The complete source code of the proposed algorithms will be released on gitHub: [ GitHub- ADAM0912/Domain-Adaptation-Problem-in-Sketch-Based-Image-Retrieval: The code repository for the articleDomain Adaptation Problem in Sketch Based Image Retrieval ] .