Speech Emotion Recognition Research Research Articles

A New Network Structure for Speech Emotion Recognition Research.

Deep learning promotes the breakthrough of emotion recognition in many fields, especially speech emotion recognition (SER). As an important part of speech emotion recognition, the most relevant acoustic feature extraction has always attracted the attention of existing researchers. Aiming at the problem that the emotional information contained in the current speech signals is distributed dispersedly and cannot comprehensively integrate local and global information, this paper presents a network model based on a gated recurrent unit (GRU) and multi-head attention. We evaluate our proposed emotion model on the IEMOCAP and Emo-DB corpora. The experimental results show that the network model based on Bi-GRU and multi-head attention is significantly better than the traditional network model at detecting multiple evaluation indicators. At the same time, we also apply the model to a speech sentiment analysis task. On the CH-SIMS and MOSI datasets, the model shows excellent generalization performance.

Speech emotion recognition based on multi-dimensional feature extraction and multi-scale feature fusion

In recent years, convolutional neural networks have been widely applied in the research of speech emotion recognition. However, most existing methods treat speech spectrogram features as images to learn the emotion features, which makes it difficult to capture rich emotion information at different scales. In this paper, a speech emotion recognition method based on multi-dimensional feature extraction and multi-scale feature fusion is proposed. Multi-dimensional features are extracted from Mel-frequency cepstral coefficients using parallel convolutional layers, emotion feature representations are learned at different scales using multi-scale residual convolutional layers, and a global feature fusion module is finally used to integrate local multi-scale emotion features to obtain key global emotion features. Experiments are conducted on the IEMOCAP, CREMA-D, and RAVDESS dataset, and the results show that our proposed network achieves a 3.5%-4% improvement in four commonly used evaluation metrics, including unweighted accuracy and weighted accuracy, compared with the state-of-the-art methods.

Progressive distribution adapted neural networks for cross-corpus speech emotion recognition.

In this paper, we investigate a challenging but interesting task in the research of speech emotion recognition (SER), i.e., cross-corpus SER. Unlike the conventional SER, the training (source) and testing (target) samples in cross-corpus SER come from different speech corpora, which results in a feature distribution mismatch between them. Hence, the performance of most existing SER methods may sharply decrease. To cope with this problem, we propose a simple yet effective deep transfer learning method called progressive distribution adapted neural networks (PDAN). PDAN employs convolutional neural networks (CNN) as the backbone and the speech spectrum as the inputs to achieve an end-to-end learning framework. More importantly, its basic idea for solving cross-corpus SER is very straightforward, i.e., enhancing the backbone's corpus invariant feature learning ability by incorporating a progressive distribution adapted regularization term into the original loss function to guide the network training. To evaluate the proposed PDAN, extensive cross-corpus SER experiments on speech emotion corpora including EmoDB, eNTERFACE, and CASIA are conducted. Experimental results showed that the proposed PDAN outperforms most well-performing deep and subspace transfer learning methods in dealing with the cross-corpus SER tasks.

A Research of Speech Emotion Recognition Based on CNN Network

A Research of Speech Emotion Recognition Based on CNN Network

Recognition of emotion from speech using evolutionary cepstral coefficients

An optimal representation of acoustic features is an ongoing challenge in automatic speech emotion recognition research. In this study, we proposed Cepstral coefficients based on evolutionary filterbanks as emotional features. It is difficult to guarantee that an individual optimized filterbank provides the best representation for emotion classification. Consequently, we employed six HMM-based binary classifiers that used a specific filterbank, which was optimized by a genetic algorithm to categorize the data into seven emotion classes. These optimized classifiers were applied in a hierarchical manner and outperformed conventional Mel Frequency Cepstral Coefficients in terms of overall emotion classification accuracy. The proposed method using evolutionary-based Cepstral coefficients achieved a weighted average recall of 87.29% on the Berlin database while the same approach but using conventional Cepstral features achieved only 79.63%.

Adaptive Data Boosting Technique for Robust Personalized Speech Emotion in Emotionally-Imbalanced Small-Sample Environments

Personalized emotion recognition provides an individual training model for each target user in order to mitigate the accuracy problem when using general training models collected from multiple users. Existing personalized speech emotion recognition research has a cold-start problem that requires a large amount of emotionally-balanced data samples from the target user when creating the personalized training model. Such research is difficult to apply in real environments due to the difficulty of collecting numerous target user speech data with emotionally-balanced label samples. Therefore, we propose the Robust Personalized Emotion Recognition Framework with the Adaptive Data Boosting Algorithm to solve the cold-start problem. The proposed framework incrementally provides a customized training model for the target user by reinforcing the dataset by combining the acquired target user speech with speech from other users, followed by applying SMOTE (Synthetic Minority Over-sampling Technique)-based data augmentation. The proposed method proved to be adaptive across a small number of target user datasets and emotionally-imbalanced data environments through iterative experiments using the IEMOCAP (Interactive Emotional Dyadic Motion Capture) database.

Speech emotion recognition research: an analysis of research focus

This article analyses research in speech emotion recognition ("SER") from 2006 to 2017 in order to identify the current focus of research, and areas in which research is lacking. The objective is to examine what is being done in this field of research. Searching on selected keywords, we extracted and analysed 260 articles from well-known online databases. The analysis indicates that SER research is an active field of research, dozens of articles being published each year in journals and conference proceedings. The majority of articles concentrate on three critical aspects of SER, namely (1) databases, (2) suitable speech features, and (3) classification techniques to maximize the recognition accuracy of SER systems. Having carried out association analysis of the critical aspects and how they influence the performance of the SER system in term of recognition accuracy, we found that certain combination of databases, speech features and classifiers influence the recognition accuracy of the SER system. We have also suggested aspects of SER that could be taken into consideration in future works based on our review.

Cross-Corpus Speech Emotion Recognition Based on Multiple Kernel Learning of Joint Sample and Feature Matching

Cross-corpus speech emotion recognition, which learns an accurate classifier for new test data using old and labeled training data, has shown promising value in speech emotion recognition research. Most previous works have explored two learning strategies independently for cross-corpus speech emotion recognition: feature matching and sample reweighting. In this paper, we show that both strategies are important and inevitable when the distribution difference is substantially large for training and test data. We therefore put forward a novel multiple kernel learning of joint sample and feature matching (JSFM-MKL) to model them in a unified optimization problem. Experimental results demonstrate that the proposed JSFM-MKL outperforms the competitive algorithms for cross-corpus speech emotion recognition.

Multi-Stage Recognition of Speech Emotion Using Sequential Forward Feature Selection

AbstractThe intensive research of speech emotion recognition introduced a huge collection of speech emotion features. Large feature sets complicate the speech emotion recognition task. Among various feature selection and transformation techniques for one-stage classification, multiple classifier systems were proposed. The main idea of multiple classifiers is to arrange the emotion classification process in stages. Besides parallel and serial cases, the hierarchical arrangement of multi-stage classification is most widely used for speech emotion recognition. In this paper, we present a sequential-forward-feature-selection-based multi-stage classification scheme. The Sequential Forward Selection (SFS) and Sequential Floating Forward Selection (SFFS) techniques were employed for every stage of the multi-stage classification scheme. Experimental testing of the proposed scheme was performed using the German and Lithuanian emotional speech datasets. Sequential-feature-selection-based multi-stage classification outperformed the single-stage scheme by 12–42 % for different emotion sets. The multi-stage scheme has shown higher robustness to the growth of emotion set. The decrease in recognition rate with the increase in emotion set for multi-stage scheme was lower by 10–20 % in comparison with the single-stage case. Differences in SFS and SFFS employment for feature selection were negligible.

The Research of Speech Emotion Recognition Based on Gaussian Mixture Model

A new recognition method based on Gaussian mixture model for speech emotion recognition is proposed in this paper. To improve the effectiveness of feature extraction and accuracy of emotion recognition, extraction of Mel frequency cepstrum coefficient combined with Gaussian mixture model is used to recognize speech emotion. According to feature parameters extraction method by analyzing the principle of vocalization theory, emotion models based on Gaussian mixture model are generated and the similarity of their templates is obtained. A series of experiments is performed with recorded speech based on Gaussian mixture model and indicates the system gains high performance and better robustness.

Graph Learning Based Speaker Independent Speech Emotion Recognition

In this paper, the algorithm based on graph learning and graph embedding framework, Speaker-Penalty Graph Learning (SPGL), is proposed in the research of speech emotion recognition to s ...

A Research of Speech Emotion Recognition Based on Deep Belief Network and SVM

Feature extraction is a very important part in speech emotion recognition, and in allusion to feature extraction in speech emotion recognition problems, this paper proposed a new method of feature extraction, using DBNs in DNN to extract emotional features in speech signal automatically. By training a 5 layers depth DBNs, to extract speech emotion feature and incorporate multiple consecutive frames to form a high dimensional feature. The features after training in DBNs were the input of nonlinear SVM classifier, and finally speech emotion recognition multiple classifier system was achieved. The speech emotion recognition rate of the system reached 86.5%, which was 7% higher than the original method.