Surrey Audio-Visual Expressed Emotion Database Research Articles

Efficient speech emotion recognition using modified feature extraction

Abstract One of the most important issues in human-computer interaction is to create a system that can hear and respond correctly like a human. This has led to the design of the Automatic Speech Emotion Recognition system (SER) that is able to identify different emotional classes by extracting and selecting effective features from speech signals. For this reason, in this study, we propose a novel feature extraction method based on adaptive time-frequency coefficients to improve the SER. The simulations are performed using the Berlin Emotional Speech Database (EMO-DB), the Surrey Audio-Visual Expressed Emotion Database (SAVEE), and the Persian Drama Radio Emotional Corpus (PDREC). The main contribution of our work is to extract novel features, called Adaptive Time-Frequency features, based on the Fractional Fourier Transform and to combine them with Cepstral features. Experimental results show that the proposed method effectively identifies different emotional classes in EMO-DB (97.57% accuracy), SAVEE (80% accuracy), and PDREC (91.46% accuracy) data sets.

Hybrid BBO_PSO and higher order spectral features for emotion and stress recognition from natural speech

The aim of the present study is to select a set of higher order spectral features for emotion/stress recognition system. 50 Bispectral (28 features) and Bicoherence (22 features) based higher order spectral features were extracted from speech signal and its glottal waveform. These features were combined with Inter-Speech 2010 features to further improve the recognition rates. Feature subset selection (FSS) was carried out in this proposed work with the objective of maximizing emotion recognition rate for subject independent with minimum features. The FSS contains two stages: Multi-cluster feature selection was adopted in Stage 1 to reduce feature space and identify relevant feature subset from Interspeech 2010 features. In Stage 2, Biogeography based optimization (BBO), Particle swarm optimization (PSO) and proposed BBO_PSO Hybrid optimization were performed to further reduce the dimension of feature space and identify the most relevant feature subset, which has higher discrimination ability to distinguish different emotional states. The proposed method was tested in three different databases: Berlin emotional speech database (BES), Surrey audio-visual expressed emotion database (SAVEE) and Speech under simulated and actual stress (SUSAS) simulated domain. The proposed feature set was evaluated with subject independent (SI), subject dependent (SD), gender dependent male (GD-male), gender dependent female (GD-female), text independent pairwise speech (TIDPS), and text independent multi-style speech (TIDMSS) experiments by using SVM and ELM classifiers. From the results obtained, it is evident that the proposed method attained accuracies of 93.25% (SI), 100% (SD), 93.75% (GD-male), and 97.58% (GD-female) for BES; 62.38% (SI) and 76.19% (SD) for SAVEE; and 90.09% (TIDMSS), 97.04% (TIDPS − Angry vs. Neutral), 98.89% (TIDPS − Lombard vs. Neutral), 99.07% (TIDPS − Loud vs. Neutral) for SUSAS.

Vocal-based emotion recognition using random forests and decision tree

This paper proposes a new vocal-based emotion recognition method using random forests, where pairs of the features on the whole speech signal, namely, pitch, intensity, the first four formants, the first four formants bandwidths, mean autocorrelation, mean noise-to-harmonics ratio and standard deviation, are used in order to recognize the emotional state of a speaker. The proposed technique adopts random forests to represent the speech signals, along with the decision-trees approach, in order to classify them into different categories. The emotions are broadly categorised into the six groups, which are happiness, fear, sadness, neutral, surprise, and disgust. The Surrey Audio-Visual Expressed Emotion database is used. According to the experimental results using leave-one-out cross-validation, by means of combining the most significant prosodic features, the proposed method has an average recognition rate of $$66.28\%$$66.28%, and at the highest level, the recognition rate of $$78\%$$78% has been obtained, which belongs to the happiness voice signals. The proposed method has $$13.78\%$$13.78% higher average recognition rate and $$28.1\%$$28.1% higher best recognition rate compared to the linear discriminant analysis as well as $$6.58\%$$6.58% higher average recognition rate than the deep neural networks results, both of which have been implemented on the same database.

Bispectral features and mean shift clustering for stress and emotion recognition from natural speech

A new set of features and feature enhancement techniques are proposed to recognize emotion and stress from speech signal. The speech waveforms and the glottal waveforms (derived from the recorded emotional/stress speech waveforms) were processed by using third order statistics called bispectrum and 28 (14 from speech waveforms and 14 from glottal waveforms) bispectral based features. In this work, mean shift clustering was used to enhance the discrimination ability of the extracted Bispectral Features (BSFs). Four classifiers were used to distinguish different emotional and stressed states. The performance of the proposed method is tested with three databases. Different experiments were conducted and recognition rates were achieved in the range between 93.44% and 100% for Berlin emotional speech database (BES), between 73.81% and 97.23% for Surrey audio-visual expressed emotion database (SAVEE), between 93.8% and 100% for speech under simulated and actual stress simulated domain (SUSAS) (recognition of multi-style speech under stress-neutral, loud, lombard and anger) and 100% for SUSAS actual domain (recognition of three different levels of stress – high, medium and low). The obtained results indicate that the proposed bispectral based features and mean shift clustering provide promising results to recognize emotion and stress from speech signal.

A new hybrid PSO assisted biogeography-based optimization for emotion and stress recognition from speech signal

Speech signals and glottal signals convey speakers’ emotional state along with linguistic information. To recognize speakers’ emotions and respond to it expressively is very much important for human-machine interaction. To develop a subject independent speech emotion/stress recognition system, by identifying speaker's emotion from their voices, features from OpenSmile toolbox, higher order spectral features and feature selection algorithm, is proposed in this work. Feature selection plays an important role in overcoming the challenge of dimensionality in several applications. This paper proposes a new particle swarm optimization assisted Biogeography-based algorithm for feature selection. The simulations were conducted using Berlin Emotional Speech Database (BES), Surrey Audio-Visual Expressed Emotion Database (SAVEE), Speech under Simulated and Actual Stress (SUSAS) and also validated using eight benchmark datasets. These datasets are of different dimensions and classes. Totally eight different experiments were conducted and obtained the recognition rates in range of 90.31%–99.47% (BES database), 62.50%–78.44% (SAVEE database) and 85.83%–98.70% (SUSAS database). The obtained results convincingly prove the effectiveness of the proposed feature selection algorithm when compared to the previous works and other metaheuristic algorithms (BBO and PSO).