Multi-feature fusion network for facial expression recognition in the wild

Abstract

Facial expression recognition (FER) has been one of the research focuses in recent years due to its significance in human-computer interactions. However, there are still challenges in the field of FER caused by the diversity and variation of facial expressions in real scenes, the singleness of feature type and the lack of enough discriminant features cannot effectively improve the recognition performance. To solve these problems, we propose a Multi-feature Fusion Network (MFNet) with dual-branch based on deep learning. Firstly, the MFNet uses the pyramid parallel multiscale residual network structure with progressive max-pooling of channel attention to extract multi-level facial features and enhance the discrimination of features; In the meantime, a shallow Gabor convolutional network is designed to enhance the adaptation of learned features to the orientation and scale changes and improve the ability to capture local details features; Finally, the maximum expression features obtained by the above two networks are fused to make more effective expression recognition. Experiments on three public large-scale wild FER datasets (RAF-DB, FERPlus, and AffectNet) show that our MFNet has a superior recognition performance than other recognition methods.