Facial expression transfer by using 3d-aware expression flow

Abstract

We study the problem of transferring facial expressions from one face to another. Direct copying and blending face components using existing methods results in semantically unnatural composites, since expression is a global effect and a local face component in one expression is often incompatible with the shape and other components in another expression. To solve this problem we present the expression flow method, which is a 2D flow field that can warp the target face globally. We develop a shape fitting algorithm, which jointly constructs 3D face shapes to the input images with the same identity but different expressions. The expression flow is computed by projecting the difference between the two 3D shapes to 2D image plane. We apply our algorithms in several applications including face compositing, face morphing, video stitching, and facial expression exaggeration. Our system is able to generate faces with much higher fidelity than existing methods.