Abstract
Speech emotion recognition (SER) is a difficult and challenging task because of the affective variances between different speakers. The performances of SER are extremely reliant on the extracted features from speech signals. To establish an effective features extracting and classification model is still a challenging task. In this paper, we propose a new method for SER based on Deep Convolution Neural Network (DCNN) and Bidirectional Long Short-Term Memory with Attention (BLSTMwA) model (DCNN-BLSTMwA). We first preprocess the speech samples by data enhancement and datasets balancing. Secondly, we extract three-channel of log Mel-spectrograms (static, delta, and delta-delta) as DCNN input. Then the DCNN model pre-trained on ImageNet dataset is applied to generate the segment-level features. We stack these features of a sentence into utterance-level features. Next, we adopt BLSTM to learn the high-level emotional features for temporal summarization, followed by an attention layer which can focus on emotionally relevant features. Finally, the learned high-level emotional features are fed into the Deep Neural Network (DNN) to predict the final emotion. Experiments on EMO-DB and IEMOCAP database obtain the unweighted average recall (UAR) of 87.86 and 68.50%, respectively, which are better than most popular SER methods and demonstrate the effectiveness of our propose method.
Highlights
As the most natural and convenient medium in human communication, speech signals contain the linguistic information like semantic and language type, and contain rich nonlinguistic information, such as facial expression, speech emotion, and so on
Inspired by Zhang et al (2017) and Zhao et al (2018), in this paper, we propose a novel method based on Deep Convolution Neural Network (DCNN) and Bidirectional Long Short-Term Memory with attention model (DCNN-Bidirectional Long Short-Term Memory with Attention (BLSTMwA))
(2) We demonstrate that the three channels of log Melspectrograms (3-D log-Mels) as DCNN input is suitable for affective feature extraction which achieves better performance than Level Descriptors (LLDs)
Summary
As the most natural and convenient medium in human communication, speech signals contain the linguistic information like semantic and language type, and contain rich nonlinguistic information, such as facial expression, speech emotion, and so on. Zhang et al (2017) proposed a new method which directly to use three channels of log Mel-spectrograms as the pre-trained DCNN’s input They used pyramid matching algorithm (DTPM) to normalize the segment-level features with unequal length. In 2018, Zheng et al (2018) proposed a new SER model combine with convolutional neural network (CNN) and random forest (RF) They adopted CNN to extract the emotional features from spectrograms, and used RF for classification. Attention mechanism can increase relatively high weights to emotionrelated features, emphasizing the importance of these features, and reduce the influence of irrelevant features It can help the network automatically focus on the emotion relevant segments and obtain discriminative features with utterance-level for SER.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
