Face Swapping: Realistic Image Synthesis Based on Facial Landmarks Alignment

Dongyue Chen,Qiusheng Chen,Jianjun Wu,Xiaosheng Yu,Tong Jia

doi:10.1155/2019/8902701

Abstract

We propose an image-based face swapping algorithm, which can be used to replace the face in the reference image with the same facial shape and features as the input face. First, a face alignment is made based on a group of detected facial landmarks, so that the aligned input face and the reference face are consistent in size and posture. Secondly, an image warping algorithm based on triangulation is presented to adjust the reference face and its background according to the aligned input faces. In order to achieve more accurate face swapping, a face parsing algorithm is introduced to realize the accurate detection of the face-ROIs, and then the face-ROI in the reference image is replaced with the input face-ROI. Finally, a Poisson image editing algorithm is adopted to realize the boundary processing and color correction between the replacement region and the original background, and then the final face swapping result is obtained. In the experiments, we compare our method with other face swapping algorithms and make a qualitative and quantitative analysis to evaluate the reality and the fidelity of the replaced face. The analysis results show that our method has some advantages in the overall performance of swapping effect.

Highlights

Face synthesis refers to the image processing technology of the automatic fusion of two or more different faces into one face, which is widely used in fields of video synthesis, privacy protection, picture enhancement, and entertainment applications
Our main contributions of the proposed algorithm include the following: (1) construct a pipeline of face swapping which integrates some learning-based modules into the traditional replacementbased approach, (2) improve the sense of reality and reliability of the synthesis face based on the precise detection of the facial landmarks, and (3) the face occlusion problem can be solved by introducing an accurate face parsing algorithm
A new face swapping algorithm based on facial landmarks detection is proposed, which can achieve fast, stable, and robust face replacement without the threedimensional model

Summary

Introduction

Face synthesis refers to the image processing technology of the automatic fusion of two or more different faces into one face, which is widely used in fields of video synthesis, privacy protection, picture enhancement, and entertainment applications. When we want to share some of the interesting things on social networks, we can use the face synthesis technique which can be regarded as a fusion of facial features and details to change our appearances appropriately without privacy leaks. As another type of face fusion, face swapping combines some parts of one person’s face with other parts of the other’s face to form a new face image. In the application of virtual hairstyle visualization, the client’s facial area can be fused with the hair areas of the model images to form new photos, so that customers can virtually browse their own figures with different hairstyles. Our main contributions of the proposed algorithm include the following: (1) construct a pipeline of face swapping which integrates some learning-based modules into the traditional replacementbased approach, (2) improve the sense of reality and reliability of the synthesis face based on the precise detection of the facial landmarks, and (3) the face occlusion problem can be solved by introducing an accurate face parsing algorithm

Related Work

Method

Experiment

Conclusion