Facial expression recognition based on deep convolution long short-term memory networks of double-channel weighted mixture

Abstract

With the aging population and the increasing number of empty nest elderly, more and more researches focus on home service robots. Autonomous analysis of human emotions by robots is helpful to provide better services for human beings. Facial expression, as an important modality in emotional recognition, is helpful to improve emotional recognition. In order to explore a new method that can effectively improve the recognition rate of expression two facial expression recognition(FER) methods are proposed in this paper. They are double-channel weighted mixture deep convolution neural networks (WMDCNN) based on static images and deep cnn long short-term memory networks of double-channel weighted mixture(WMCNN-LSTM) based on image sequences. WMDCNN network can quickly recognize facial expressions and provide static image features for WMCNN-LSTM network. WMCNN-LSTM network utilizes the static image features to further acquire the temporal features of image sequence, which can realize the accurate recognition of facial expressions. The experimental results show that the average recognition rate of the WMDCNN network on the four datasets of CK+, JAFFE, Oulu-CASIA and MMI are 0.985, 0.923,0.86,0.78 respectively. The WMCNN-LSTM method has an average recognition rate of 0.975, 0.88, and 0.87 on the three datasets CK+, Oulu-CASIA and MMI respectively. By comparing with the existing FER method, our method further improves the recognition rate in the above four expression data sets.