A Superpixel-Wise Approach for Face Sketch Synthesis

Abstract

Face sketch synthesis from an input photo has drawn great attention in law enforcement and digital entertainment applications. Currently, the input photo is simply divided into overlapped rectangular patches and transformed to a sketch through weighted average of corresponding sketch patches. However, the regular patches lead to defects in the structure of synthesized sketch. In addition, existing methods need to cross the whole training dataset in order to collect the corresponding sketch patches, which limits their usability with the big training datasets. In this paper, a superpixel-wise approach based on the Superpixel technique incorporated into the Locality-constraint Linear Coding (LLC), termed as SuperLLC, is proposed to enhance the facial structure of synthesized sketch and simultaneously maintain fixed computational complexity regardless of the training dataset size. First, the input photo is segmented into overlapped superpixels to find their corresponding sketch superpixels from the training dataset. The LLC is then imposed to regularize proper weights to reconstruct the target sketch from these sketch superpixels, with averaging the overlapped areas between adjacent superpixels. But before all these, for each input photo, a sub-training set is selected based on facial landmarks distance between the input photo and the training photos set. This insures a steady synthesis process. Both subjective and objective experiments on public face sketch databases are finally carried out to reveal the superior performance of the proposed SuperLLC method compared to state-of-the-art methods.