A Facial Expression Recognition Method Based on a Multibranch Cross-Connection Convolutional Neural Network

Abstract

This paper proposes a new method based on a multiple branch cross-connected convolutional neural network (MBCC-CNN) for facial expression recognition. Compared with traditional machine learning methods, the proposed method can extract image features more effectively. In addition, in contrast to single-structure convolutional neural networks, the MBCC-CNN model is constructed based on the residual connection, Network in Network, and tree structure approaches together. It also adds a shortcut cross connection for the summation of the convolution output layer, which makes the data flow between networks more smooth, improves the feature extraction ability of each receptive field. The proposed method can fuse the features of each branches more effectively, which solves the problem of insufficient feature extraction of each branches and increases the recognition performance. The experimental results based on the Fer2013, CK+, FER+ and RAF data sets show that the recognition rates of the proposed MBCC-CNN method are 71.52%, 98.48%, 88.10% and 87.34%, respectively. Compared with some most recently work, the proposed method can provide better facial expression recognition performance and has good robustness. The python code can be download from <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/scp19801980/Facial-expression-recognition</uri> .