Learning multi-level domain invariant features for sketch re-identification

Abstract

Person re-identification (Re-ID) aims to match the photos of suspects in the gallery database by a query photo. Under Re-ID, the input is usually a query photo of the target person, however, in a realistic setting, such a photo is not always available. In this paper, we study the problem of Sketch Re-ID, which is not based on a person's photo but the professional sketch of the target person. We use a full-body sketch of a target person drawn by a professional to find the photo with the same ID in the gallery. This problem is very challenging because photos and sketches belong to two completely different domains. Specifically, a sketch is a highly abstract description of a person, which only contains some rough outline information. We address the Sketch Re-ID problem by proposing a novel framework to jointly model photos and sketches into a common embedding space. Our framework uses a triplet classification network as a base network. We propose to use a spatial attention module and combine high-level and mid-level output features of CNN to represent the input images. Moreover, we design a novel domain-invariant feature by using a gradient reverse layer (GRL) to solve the domain gap problem. We validated our approach on the Sketch Re-ID dataset, which contains 200 persons, each of whom has a sketch and two photos from different cameras associated. To evaluate the generalization of our method, we also performed experiments on some other public Sketch-based Image Retrieval (SBIR) datasets. The extensive experimental results show that our method can get higher performance than state-of-the-art models.