Face Sketch Synthesis From a Single Photo–Sketch Pair

Abstract

Face sketch synthesis is crucial in many practical applications, such as digital entertainment and law enforcement. Previous methods relying on many photo–sketch pairs have made great progress. State-of-the-art face sketch synthesis algorithms adopt Bayesian inference (BI) (e.g., Markov random fields) to select local sketch patches around corresponding position from a set of training data. However, these methods have two limitations: 1) they depend on many training photo–sketch pairs and 2) they cannot tackle nonfacial factors (e.g., hairpins, glasses, backgrounds, and image size) if these factors are excluded in training data. In this paper, we propose a novel face sketch synthesis method that is capable of handling nonfacial factors only using a single photo–sketch pair from coarse to fine. Our method proposes a cascaded image synthesis (CIS) strategy and integrates sparse representation-based greedy search (SRGS) and BI for face sketch synthesis. We first apply SRGS to select candidate sketch patches from the whole training photo–sketch pairs sampled from the only photo–sketch pair. We then employ BI to estimate an initial sketch. Afterward, the input photo and the estimated initial sketch are taken as an additional photo–sketch pair for training. Finally, we adopt CIS with the given two photo–sketch pairs to further improve the quality of the initial sketch. The experimental results on several databases demonstrate that our algorithm outperforms state-of-the-art methods.