Understanding Deep Learning Techniques for Recognition of Human Emotions Using Facial Expressions: A Comprehensive Survey

Abstract

Emotion recognition plays a significant role in cognitive psychology research. However, measuring emotions is a challenging task. Thus, several approaches have been designed for facial expression recognition (FER). Although, the challenges increases further as the data transits from laboratory-controlled environment to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in-the-wild</i> circumstances. Nowadays, applications are overwhelmed by a profusion of deep learning (DL) techniques in the real-world problems. DL networks have steadily led to a better understanding of low-dimensional discriminative features from high-dimensional complex face patterns for automatic FER. The modern FER systems based on deep neural networks mainly suffer from two problems: overfitting due to the inadequate availability of training data and complications unassociated with the expressions, such as occlusion, posture, illumination, and identity bias. This study aims to provide a comprehensive survey of the significant DL-based methods that have made a notable contribution to the field of FER. Different components of the methods, such as pre-processing, feature extraction, and classification of facial expressions, are described systematically. Moreover, the discussed approaches are analyzed to compare their performance along with their advantages and limitations. Further, different databases relevant to FER are also explored in this study. Essentially, the main aim of this survey is twofold. The former is to discuss the current scenario of FER approaches and the latter is to present some thoughts on the future directions of facial emotion recognition by machines: what are the obstacles and prospects for FER researchers?.