WGAN-Based Robust Occluded Facial Expression Recognition

Yan Zhao,Yang Lu,Wenting Zhao,Shigang Wang

doi:10.1109/access.2019.2928125

Abstract

Research on facial expression recognition (FER) technology can promote the development of theoretical and practical applications for our daily life. Currently, most of the related works on this technology are focused on un-occluded FER. However, in real life, facial expression images often have partial occlusion; therefore, the accurate recognition of occluded facial expression images is a topic that should be explored. In this paper, we proposed a novel Wasserstein generative adversarial network-based method to perform occluded FER. After complementing the face occlusion image with complex facial expression information, the recognition is achieved by learning the facial expression features of the images. This method consists of a generator G and two discriminators D 1 and D 2 . The generator naturally complements occlusion in the expression image under the triple constraints of weighted reconstruction loss l wr , triplet loss l t , and adversarial loss l a . We optimize the discriminator D 1 to distinguish between real and fake by constructing an adversarial loss l a between the generated complementing images, original un-occluded images, and small-scale-occluded images based on the Wasserstein distance. Finally, the FER is completed by introducing classification loss l c into D 2 . To verify the effectiveness of the proposed method, an experimental analysis was performed on the AffectNet and RAF-DB datasets. The visual occlusion complementing results, comparison of recognition rates of facial expression images with and without de-occlusion processing, and T-distributed stochastic neighbor embedding visual analysis of facial expression features all prove the effectiveness of the proposed method. The experimental results show that the proposed method is better than the existing state-of-the-art methods.

Highlights

Facial expression is a form of non-verbal communication, which is the main means of expressing social information between human beings
AffectNet and real-world affective faces database (RAF-DB) are selected as the experimental datasets in this study, because these two datasets reflect the natural facial expressions in real life, and the number of images is abundant
Each image has about 40 independent labels, and the facial expression images in the database are different in age, gender, race, head posture, lighting conditions and other aspects, which compound the characteristics of the expression images in real life

Summary

Introduction

Facial expression is a form of non-verbal communication, which is the main means of expressing social information between human beings. The facial expression recognition (FER) is a technology for using computer to recognize changes in facial features of a human face to classify different facial expressions. FER is not affected by race, skin, age, and gender [1]–[3]. The expression recognition technology is a cross product of many disciplines, such as biology, psychology, and computer science. In recent years, this technology has gained significant attention of the researchers because of its high usage value and research significance. The associate editor coordinating the review of this manuscript and approving it for publication was Md. Asikuzzaman

Methods

Results

Conclusion