Manifold-enhanced CycleGAN for facial expression synthesis

Abstract

ABSTRACT Facial expressions contain a great deal of information in inter-personal communication and social interaction and have a great impact on face recognition. This paper investigates expression synthesis methods based on deep learning and their performance in expression classification and face recognition. We propose a new synthesis method based on the active appearance model and CycleGAN. The eigendecomposition method is used to obtain the expression shape of a target subject through linear operations. An expression intensity coefficient is developed to control the dynamics of generated expressions. Subsequently, a generative model is introduced for the synthesis on textures. The identity loss is integrated into the traditional CycleGAN model, incorporated with the adversarial loss and the cycle consistency loss, to better preserve texture attributes and identity information of target subjects. Experiments and comparisons show that the proposed method offers marked improvements in verification of expressions and identities, as well as in generalization across databases.