Multi-pose Facial Expression Recognition Based on Unpaired Images

Abstract

Giving machines the ability to perceive human emotions and enable them to recognize our emotional states is one of the important goals to realize human-computer interaction. In the past decades, facial expression recognition (FER) has always been a research hotspot in the field of computer vision. However, the existing facial expression datasets generally have the problems of insufficient data and unbalanced categories, leading to the phenomenon of over-fitting. To solve this problem, most methods employ the generative adversarial network (GAN) for data augmentation, and achieve good results in facial image generation. But these works focus only on facial identity or head poses, which are not robust for the transformation of facial expression recognition from the laboratory environment to unconstrained scenes. Therefore, we employ the disentangled representation learning to obtain facial feature representation, so as to reduce the impact of pose changes and identity biases on FER. Specifically, the generator uses the encoder-decoder structure to map each face image to two latent spaces: the pose space and the identity space. In each latent space, we disentangle the target attribute from other attributes, and then concatenate corresponding feature vectors to generate a new image with one person’s identity and another person’s pose. Experimental results on Multi-PIE and RAFD datasets show that the proposed method can obtain high quality generated images and effectively improve the recognition rate of facial expressions.