Face photo-sketch synthesis via full-scale identity supervision

Abstract

Face photo-sketch synthesis refers transforming a face image between photo domain and sketch domain. It plays a crucial role in law enforcement and digital entertainment. A great deal of efforts have been devoted on face photo-sketch synthesis. However, limited by the weak identity supervision, existing methods mostly yield indistinct details or great deformation, resulting in poor perceptual appearance or low recognition accuracy. In the past several years, face identification achieved great progress, which represents the face images much more precisely than before. Considering the face image translation is also a type of face image re-representation, we attempt to introduce face recognition models to improve the synthesis performance. First, we applied existing synthesis models to augment the training set. Then, we proposed a full-scale identity supervision method to reduce redundant information introduced by these pseudo samples and take the valid information to enhance the intra-class variations. The proposed framework consists of two sub-networks: cross-domain translation (CT) network and intra-domain adaptation (IA) network. The CT network translates the input image from source domain to latent image of target domain, which overcomes the great gap between two domains with less structural deformation. The IA network adapts the perceptual appearance of latent image to target image by adversarial learning. Experimental results on CUHK Face Sketch Database and CUHK Face Sketch FERET Database demonstrate the proposed method preserved best perceptual appearance and more distinct details with less deformation.