A visual self-attention network for facial expression recognition

Abstract

In real world, facial expression images are easily disturbed by different external environments, such as face poses, illumination intensities, and occlusion, and features extracted from whole facial expression images often have a lot of noise, which leads to low recognition accuracy of the model. Inspired by the facts that local images are less disturbed by the environment and human beings usually recognize facial expressions by integrating some local information about human faces, this paper proposes a model for recognition of facial expressions, based on the fusion of local image features, that aims to recognize facial expressions by comprehensively considering local information about human faces. To particularly integrate the robust characteristics of local images, this paper constructs a visual self-attention network by transformation of the Transformer model in the field of natural language processing, which can take into account the interaction between local images and make the image feature more discriminative. To demonstrate the performance of the model, this paper conducts experiments on three constrained small datasets, CK+, Oulu-CASIA and MMI, and three unconstrained large datasets, FER2013, RAF-DB and Expw. The results show that the proposed model can attain the recognition accuracy of the state-of-the-art methods on all datasets, thereby verifying the effectiveness of the proposed method.