Facial expression recognition with convolutional neural networks via a new face cropping and rotation strategy

Abstract

With the recent development and application of human–computer interaction systems, facial expression recognition (FER) has become a popular research area. The recognition of facial expression is a difficult problem for existing machine learning and deep learning models because that the images can vary in brightness, background, pose, etc. Deep learning methods also require the support of big data. It does not perform well when the database is small. Feature extraction is very important for FER, even a simple algorithm can be very effective if the extracted features are sufficient to be separable. However, deep learning methods automatically extract features so that some useless features can interfere with useful features. For these reasons, FER is still a challenging problem in computer vision. In this paper, with the aim of coping with few data and extracting only useful features from image, we propose new face cropping and rotation strategies and simplification of the convolutional neural network (CNN) to make data more abundant and only useful facial features can be extracted. Experiments to evaluate the proposed method were performed on the CK+ and JAFFE databases. High average recognition accuracies of 97.38% and 97.18% were obtained for 7-class experiments on the CK+ and JAFFE databases, respectively. A study of the impact of each proposed data processing method and CNN simplification is also presented. The proposed method is competitive with existing methods in terms of training time, testing time, and recognition accuracy.