Markov random fields and facial landmarks for handling uncontrolled images of face sketch synthesis

Abstract

Face sketch synthesis has drawn great attention in many computer vision applications such as law enforcement and digital entertainment. The majority of existing face sketch synthesis techniques are exemplar-based techniques, where a set of training photo–sketch pairs are first divided into patches. For an input photo patch from the face to be synthesized, k similar photo patches are found from the training set. The corresponding sketch patch of the best match is then selected to be synthesized. In such techniques, a multiscale Markov random fields (MRF) model is utilized for synthesizing a sketch using candidate sketch patches; having observed that techniques tend to fail with face photos acquired in uncontrolled imaging conditions like pose and lighting variations. For example, some structures along the lower part of the face sketch contour get lost due to ignoring the global face shape information and illumination changes. In this paper, we propose a reliable face sketch synthesis method based on MRF model and facial landmarks, called MRF-FL that can maintain further structures with uncontrolled face photos. Besides matching the input photo with training photo, the input photo and training sketch are also matched based on the facial landmarks so as to enhance face sketch structures around the lower part of face sketch contour. Experimental results showed that the proposed MRF-FL achieves superior performance compared with recent face sketch synthesis methods on CUHK and AR face sketch databases.