An Efficient Approach to Face Emotion Recognition with Convolutional Neural Networks

Abstract

Solutions for emotion recognition are becoming more popular every year, especially with the growth of computer vision. In this paper, classification of emotions is conducted based on images processed with convolutional neural networks (CNNs). Several models are proposed, both custom and transfer learning types. Furthermore, combinations of them as ensembles, alongside various methods of dataset modification, are presented. In the beginning, the models were tested on the original FER2013 dataset. Then, dataset filtering and augmentation were introduced, and the models were retrained accordingly. Two methods of emotion classification were examined: a multi-class classification, and a binary classification. In the former approach, the model returns the probability for each class. In the latter, separate models for each single class are prepared, together with an adequate dataset based on FER2013. Each model recognizes a single emotion from the others. The obtained results and a comparison of the applied methods across different models is presented and discussed.