Deep learning-based dimensional emotion recognition combining the attention mechanism and global second-order feature representations

Abstract

For the emotion recognition tasks, the emotion-related features and the emotionally irrelevant ones are fully extracted by the deep neural network is particularly desirable. In this paper, a dimensional emotion recognition algorithm is proposed by combing the attention mechanism and global second-order feature representations. More specially, the residual attention network (RAN) is utilized to select the features related to the task, and then the global second-order pooling network is employed to model the correlation between these features to complete the modeling of the feature interdependence relationship, improving the deficiency of the RAN in this point so as to enhance the capacity of extracting the emotion-related features and the emotionally irrelevant ones. The experiments on the AffectNet dataset demonstrate the validity of the proposed model, which is comparable to and even superior to those from the support vector regression method, the convolutional neural network (CNN)-based method, and the recently-developed CNN-based variants.