Integrating GAN with CNN for Face Sketch Synthesis

Abstract

Face sketch synthesis presents very useful applications in different domains such as online digital entertainment and the identification of suspects in criminal cases. Many existing face sketch synthesis methods straightly learn the relationship between photo images and sketch images, and it's relatively difficult for these methods to take the discrete specific details for the sketch synthesis process. The synthesized sketches generated by these traditional methods consistently manifest the coarse textures of face images, however the fine details of few critical facial segments are exclusively lost. By addressing these problems, we propose a novel framework for face sketch synthesis by applying deep learning features in this paper. Initially, we use the Generative Adversarial Network (GAN) model to get the more realistic synthesized sketch through learning the nonlinear mapping relationship between face photo-sketch images and use them as the coarse estimation images for the network. During this synthesis process, the generated output images are highly contaminated with noise. Then, we use a convolutional neural network (CNN) based model called ResNet as the secondary processing tool for generated synthesized images. As the result, it fairly reduces the noise and distortion of the synthesized images and we got fine estimation images as the residual images for the network. Given the coarse synthesized images and these residual images, a fully functional network model is designed to get the final synthesized face sketches with the fine and critical details in contrast to traditional techniques. Multiple experiments illustrate that our proposed method has remarkable results on the face sketch synthesis tasks as compared to the traditional methods.