Lightweight facial expression recognition method based on attention mechanism and key region fusion

Abstract

To improve the facial expression recognition accuracy in resource-constrained and real-time application equipment such as mobile and embedded devices, a lightweight method for facial expression recognition is proposed based on attention mechanism and key regions fusion. To reduce the computation complexity, a lightweight convolutional neural network, mini_Xception, is used as the basic expression recognition model for expression classification. The attention mechanism is introduced to enhance the learning of the important features of the whole face. Then a parameter is introduced to locate the key regions and construct key region models. Finally, to realize the complementarity of models and learn more comprehensive features, the whole facial expression recognition model is fused with the key region models. The proposed method can capture and utilize the important facial expression information in related regions displayed through class activation mapping visualization. The experimental results on JAFFE, CK+ datasets, and a real scene dataset verify the effectiveness of the proposed method.