Multi-View Facial Expression Recognition based o n Multitask Learning and Generative Adversarial Network

Abstract

Facial expressions contain rich emotional information, which is an important method in human communication. At present, most of the researches on facial expression recognition is conducted on the frontal faces. However, in real life, the captu re device may capture facial expression data from various poses. Different from the existing techniques, in this paper, We propose a multitask deep learning method that uses the links between var ious poses and expressions to improve the accuracy of expression recognition. We use the adversarial network to supplement the e xpression information of the face for the head poses with a sever e lack of expression information. There are some advantages: Firstly, we use vgg16 to train the expression images of each deflecti on angle separately and find that the expression recognition acc uracy rate for small offset angle (−30 °, −15 °, 15°, 30 °) is larger t han 0 ° angle when the face angle is greater than 45 °, the accur acy of recognition decreases sharply. So we use multitask learnin g to jointly train these small offsets angle images and frontal ima ges, the multitask learning can learn the emotion-preserving repr esentations at various poses to predict the expression class label f rom the input face, and bring again to its recognition accuracy r ate. Secondly, for the poses of a severe lack of expression inform ation, we use TP-GAN to convert a large deflection pose image i nto a frontal face and supplement its expression information. Th e experimental results show that our proposed algorithm has a g ood recognition effect on facial expressions for all poses. Compa red with the most advanced expression recognition methods, this paper has also achieved the state-of-the-art recognition results.