A complementary facial representation extracting method based on deep learning

Abstract

Abstract The identification and expression are two orthogonal properties of faces. But, few studies considered the two properties together. In this paper, the two properties are modeled in a unified framework. A pair of 18-layered Convolutional Deconvolutional Networks (Conv-Deconv) is proposed to learn a bidirectional mapping between the emotional expressions and the neutral expressions. One network extracts the complementary facial representations (i.e. identification representations and emotional representations) from emotional faces. The other network reconstructs the original faces from the extracted representations. Two networks are mutually inverse functions. Based on the framework, the networks are extended for various tasks, including face generation, face interpolation, facial expression recognition, and face verification. A new facial expression dataset called Large-scale Synthesized Facial Expression Dataset (LSFED) is presented. The dataset contains 105,000 emotional faces of 15,000 subjects synthesized by computer graphics program. Its distorted version (LSFED-D) is also presented to increase the difficulty and mimic real-world conditions. Good experiment results are obtained after evaluating our method on the synthesized clean LSFED dataset, the synthesized distorted LSFED-D dataset, and the real-world RaFD dataset.