Automatic flexible face replacement with no auxiliary data

Abstract

In this paper, we propose a system for flexible face replacement in photos that does not require human intervention. The proposed replacement algorithm is able to efficiently paste one person׳s face into another׳s with different head poses and/or hairstyles. This work can enhance the quality of group photos that contain human faces, as they are seldom flawless. Previous methods require complex user interaction or auxiliary data such as a 3D head model library or a photo library, while our system does not. Given a photo, the faces are automatically detected and transformed into similar poses by fitting corresponding facial features. We then construct an energy function and optimize it for a better face contour between the transformed source face and the target one, followed by applying seamless cloning. Our system removes hair before face transfer and adds it back afterwards, maintaining the same hairstyles. We demonstrate that our system produces high-quality results without manual tuning of parameters.