Real Time Face Swap Based on Patch Warping

Abstract

Face swap is one of the popular machine vision problems recently. Important problems in real-time face swap are the need for high computation power and the decrease in success of applications in case of face pose variations. In this paper, an original face swap algorithm which can work in real time is proposed. After detecting two faces in the input image, 68 landmark points of each face are localized. Using these points, a facial model has been formed which separates the faces into 23 quadrilateral planar regions. Homographies are calculated between the quadrilaterals formed by the same points on both sides in order not to disrupt the holistic structure of the selected regions. Face swap is performed by warping the face patches with calculated homographies. With the proposed method, the calculation resources used in the algorithm have been used efficiently and identity information such as eye, mustache and eyebrow has been preserved. The symmetries of the visible regions are used for the invisible parts in the viewing angles that the camera cannot see the entire face. The success of our study has been tested with different camera resolutions and faces at different angles and qualitative results are given.