A Selective Expression Manipulation With Parametric 3D Facial Model

Abstract

This paper proposes a novel method to represent expressive 3D facial shapes called the Selective Expression Manipulation (SEM) by fitting the expression coefficients of delta-blendshapes, which is the standard parametric facial model widely used in industries. SEM focuses on preserving blendshape semantics to characterize the facial shapes since the facial shape obtained by minimizing the distance to sparse facial landmarks might fail to signify a facial expression from a human being’s perspective. Assuming each delta-blendshape corresponds to a facial movement with semantic meaning, SEM finds a series of facial motions required to compose the target facial expression. In addition, SEM sequentially determines a sufficient number of expressions and coefficients closely resembling the target facial movements by introducing similarities to quantify the directional correlation of facial motions between a target and a blendshape, excluding redundant expressions in terms of motions from the neutral shape. As a result, far fewer inter-correlated expressions that significantly increase the target correlation can be obtained. Furthermore, SEM exhibits substantial improvement in accuracy, correlation, semantics, and stability in experiments over previous facial fitting schemes and state-of-the-art methods. It is demonstrated that SEM enables accurate and realistic 3D facial shape generation by semantically manipulating expression delta-blendshapes.