Novel 1D and 2D Convolutional Neural Networks for Facial and Speech Emotion Recognition

Abstract

AbstractAs humans, we express most naturally via speech. Speech emotion recognition systems are defined as a set of techniques for processing and classifying speech signals in order to detect the emotions that are inherent in them. For emotion recognition in audio files, a novel 1D convolutional neural network (Audio_EmotionModel) is designed, which contains 5 1D convolutional layers, 5 max pooling layers and 3 dropout layers. The Audio_EmotionModel is used for RAVDESS dataset for emotion recognition in audio datasets. For Emotion recognition in images (obtained by splitting videos into images), a novel 2D Convolutional neural network is designed (Image_EmotionModel), which contains 4 2D convolutional layers, 2 max pooling layers, 3 dropout layers and 2 batch normalization layers. The Image_EmotionModel is used for RAVDESS dataset for emotion recognition in video datasets (videos are converted into frames). The results clearly indicate, the two proposed models perform better than various state-of-art models. Human Recognition [29] achieved only 40.9% while recognizing the emotions, clearly the proposed Audio_EmotionModel outperformed the human recognition by nearly more than 50%.KeywordsEmotion recognitionSpeech recognitionConvolutional neural networkDeep learning