Configural Representation of Facial Action Units for Spontaneous Facial Expression Recognition in the Wild

Abstract

In this paper, we propose an approach for spontaneous expression recognition in the wild using configural representation of facial action units. Since all configural features do not contribute to the formation of facial expressions, we consider configural features from only those facial regions where significant movement is observed. These chosen configural features are used to identify the relevant facial action units, which are combined to recognize facial expressions. Such combinational rules are also known as coding system. However, the existing coding systems incur significant overlap among facial action units across expressions, we propose to use a coding system based on subjective interpretation of the expressions to reduce the overlap between facial action units, which leads to better recognition performance while recognizing expressions. The proposed approach is evaluated for various facial expression recognition tasks on different datasets: (a) expression recognition in controlled environment on two benchmark datasets, CK+ and JAFFE, (b) spontaneous expression recognition on two wild datasets, SFEW and AFEW, (c) laughter localization on MAHNOB laughter dataset, and (d) recognizing posed and spontaneous smiles on UVA-NEMO smile dataset.