Union is Strength: Improving face sketch synthesis by fusing outcomes of Fully-Convolutional-Networks and Random Sampling Locality Constraint

Abstract

Automated formulation of sketches from face photos has seen successive growth since the work of Wang and Tang in recent years. Each new methodology is, however, able to partially achieve its objective of sketch synthesis while using pairs of photos and viewed sketches as a training medium. The viewed sketches are also used as a testing medium to determine the success of those methodologies. Resulting sketches do not fully capture all features of the training photos and viewed sketches. Their similarity value to respective sketch is also around 30 – 50%. One technique may produce sketches with sharp edges, but they do not bear completeness of facial features. Another technique produces sketches with the completeness of facial elements, but they are not well-focused. Second limitation of existing techniques is attributable to face-recognition procedure which is used as a validation step for these methodologies. Face-recognition process with help of synthesized sketches delivers reliable results over datasets with a limited diversity of age, ethnicity, and light intensities. We propose a novel and cost-effective approach to fuse resulting sketches of two test techniques. The two techniques are merged to yield a better sketch containing well-defined features, sharp contours, and less noise. Secondly, fusion suppresses limitations of the component methodologies reaching the resulting sketch. To test this idea of combining sketch-synthesis methods, we experiment with the most basic techniques of image fusion including simple (arithmetic), PCA, and Wavelet based fusions. The proposed setup considered FCN (complete features but less sharpness) and Fast-RSLCR (sharp edges but missing contours) as candidate techniques. It is tested on two datasets namely CUFS and CUFSF. Second dataset incorporates variations of age, ethnicity, light intensities, and slightly deformed features between photos and viewed sketches. Our results indicate achievement of 60.29% SSIM score (enhancement by 3.84%) and 79.03% face-recognition score (enhancement by 5.62%) as compared to Fast-RSLCR.