IsGAN: Identity-sensitive generative adversarial network for face photo-sketch synthesis

Abstract

Face photo-sketch synthesis aims to generate face sketches from real photos and vice versa. It can be abstracted as a constrained quantization problem. Although many efforts have been dedicated to this problem, it is still a challenging task to synthesize detail-preserving photos or sketches due to the significant differences between face sketch (drawn by people) and photo (taken by cameras) domains. In this paper, we propose a novel Identity-sensitive Generative Adversarial Network (IsGAN) to address it. Our key insight is to formalize face photo-sketch synthesis as a special case of image-to-image translation and propose to embed identity information through adversarial learning. In particular, an adversarial architecture is used to capture the differences between the two domains, and a new network loss, namely, identity recognition loss is introduced to preserve the detailed identifiable information, which is crucial for photo-sketch synthesis. In addition, to enforce structural consistency during generation, a cyclic-synthesized loss is applied between the generated image of one domain and cycled image of another. The experiments on the CUFS and CUFSF datasets suggest that our model achieves state-of-the-art performance in both qualitative and quantitative measures.