Emotional Speech Research Articles

Enhancing speech emotion recognition through deep learning and handcrafted feature fusion

In this paper, we introduce an innovative investigation in speech emotion recognition (SER). The proposed model combines deep learning-based and handcrafted audio features to achieve optimal accuracy. The proposed model employs an iterative feature selection and majority voting pipeline to obtain better results by fusing deep learning-based and handcrafted features. The wav2vec2 model and the openSmile audio processing library are used in order to extract audio features from audio data. Then the feature selection and majority voting techniques are used to identify the optimal feature selection methods for diverse feature sets and to combine their strengths. The experiments are performed using a diverse and extensive corpus to ensure the robustness of the proposed method. In the construction of this multi-corpus dataset, we used four well-known benchmark datasets, namely Ravdess, Savee, Crema-D, and Tess. All of these datasets are publicly available. These datasets are combined on six common emotions: sadness, happiness, fear, anger, surprise, and disgust. The resultant dataset comprises 11,511 samples across these categories. The proposed method has been shown to achieve results comparable to those reported in the existing literature. The experimental results indicate that the proposed pipeline leads to a 3 % improvement in classification accuracy. The highest achieved accuracy on the multi-corpus dataset is 92.55 %.

ADAM optimised human speech emotion recogniser based on statistical information distribution of chroma, MFCC, and MBSE features

ADAM optimised human speech emotion recogniser based on statistical information distribution of chroma, MFCC, and MBSE features

TemporalAugmenter: An Ensemble Recurrent Based Deep Learning Approach for Signal Classification

Ensemble modeling has been widely used to solve complex problems as it helps to improve overall performance and generalization. In this paper, we propose a novel TemporalAugmenter approach based on ensemble modeling for augmenting the temporal information capturing for long-term and short-term dependencies in data integration of two variations of recurrent neural networks in two learning streams to obtain the maximum possible temporal extraction. Thus, the proposed model augments the extraction of temporal dependencies. In addition, the proposed approach reduces the preprocessing and prior stages of feature extraction, which reduces the required energy to process the models built upon the proposed TemporalAugmenter approach, contributing towards green AI. Moreover, the proposed model can be simply integrated into various domains including industrial, medical, and human-computer interaction applications. Our proposed approach empirically evaluated the speech emotion recognition, electrocardiogram signal, and signal quality examination tasks as three different signals with varying complexity and different temporal dependency features.

Squeeze-and-excitation 3D convolutional attention recurrent network for end-to-end speech emotion recognition

Speech emotion recognition (SER) is difficult since emotions are complex and dynamic processes involving multiple dimensions and sub-dimensions. Feature extraction is a challenging step in SER, where relevant features are extracted from the speech to identify emotional states accurately. Overcoming these challenges is essential to ensuring the effectiveness and robustness of the SER system. 3D-convolutional neural networks (3D-CNNs) are successfully used for feature extraction in SER. Speech signals can be converted into spectrogram-like representations where one axis represents time, another frequency, and the third can represent additional context or features. For modeling the dynamic nature of speech, a squeeze-and-excitation-based 3D-CNN model is employed for capturing temporal and spatial features of speech. Attention Gated Recurrent Units (AGRU) are applied to the extracted features for learning long-range temporal dependencies and selecting more informative features. The extraction and selection of the spatio-temporal feature representation lead to a hierarchical representation of the input speech. The fusion of squeeze-and-excitation 3D-CNN and AGRU (named SE3D-CARN) is evaluated on two datasets, EMO-BD and IEMOCAP, to identify various emotional states in speech. The proposed SER model reached an accuracy of 94.2% and 81.1% on the EMO-BD and IEMOCAP datasets, respectively.

Speech Emotion Recognition Using Magnitude and Phase Features

Speech Emotion Recognition Using Magnitude and Phase Features

Multi-language: ensemble learning-based speech emotion recognition

Multi-language: ensemble learning-based speech emotion recognition

Speech Emotion Recognition Using Machine Learning

The speech is the most effective means of communication, to recognize the emotions in speech is the most crucial task. In this paper we are using the Artificial Neural Network to recognize the emotions in speech. Hence, providing an efficient and accurate technique for speech based emotion recognition is also an important task. This study is focused on seven basic human emotions (angry, disgust, fear, happy, neutral, surprise, sad). The training and validating accuracy and also lose can be seen in a graph while training the dataset. According to it confusion matrix for model is created. The seven features Frequency, Pitch, Amplitude and formant of speech is used to recognize seven basic emotions from speech. Keywords: Emotion detection, Emotion classification, verbal emotion classification.

Speech Emotion Recognition Using ANN

The speech is the most effective means of communication, to recognize the emotions in speech is the most crucial task. In this paper we are using the Artificial Neural Network to recognize the emotions in speech. Hence, providing an efficient and accurate technique for speech based emotion recognition is also an important task. This study is focused on seven basic human emotions (angry, disgust, fear, happy, neutral, surprise, sad). The training and validating accuracy and also lose can be seen in a graph while training the dataset. According to it confusion matrix for model is created. The seven features Frequency, Pitch, Amplitude and formant of speech is used to recognize seven basic emotions from speech. Keywords: Emotion detection, Emotion classification, verbal emotion classification.

Enhancements in Immediate Speech Emotion Detection: Harnessing Prosodic and Spectral Characteristics

Speech is essential to human communication for expressing and understanding feelings. Emotional speech processing has challenges with expert data sampling, dataset organization, and computational complexity in large-scale analysis. This study aims to reduce data redundancy and high dimensionality by introducing a new speech emotion recognition system. The system employs Diffusion Map to reduce dimensionality and includes Decision Trees and K-Nearest Neighbors(KNN)ensemble classifiers. These strategies are suggested to increase voice emotion recognition accuracy. Speech emotion recognition is gaining popularity in affective computing for usage in medical, industry, and academics. This project aims to provide an efficient and robust real-time emotion identification framework. In order to identify emotions using supervised machine learning models, this work makes use of paralinguistic factors such as intensity, pitch, and MFCC. In order to classify data, experimental analysis integrates prosodic and spectral information utilizing methods like Random Forest, Multilayer Perceptron, SVM, KNN, and Gaussian Naïve Bayes. Fast training times make these machine learning models excellent for real-time applications. SVM and MLP have the highest accuracy at 70.86% and 79.52%, respectively. Comparisons to benchmarks show significant improvements over earlier models.

Educational game as a practice for developing students' critical thinking skills regarding manipulations on the Internet

Introduction. Expanded digitalization widens the gap between the increasing immersion of young people in the digital space and their ability to identify, manage and resist the destructive potential of the information flow of the Internet environment. Increasing threats to personal security and improved technologies for manipulating public opinion and consciousness makes developed critical thinking skills and increased resistance to manipulative and destructive online content particularly relevant for the young generation The paper aims to determine the potential of an educational game in developing critical thinking skills among students in relation to manipulations on the Internet. Materials and methods. 147 students of Tomsk State University (Russian Federation) took part in the pilot study. Research methods were: 1) a questionnaire for self-assessment of the ability to identify manipulative techniques and assess the impact of information on own psycho-emotional state; 2) a test task for forming skills in recognizing manipulative techniques. Statistical data was processed using the Mann Whitney U test and the Wilcoxon t-test. Research results. A pilot experimental study revealed the following: 1) respondents tend to overestimate their ability to determine the manipulative potential of Internet content, as well as their ability to resist it and realize its impact on their psycho-emotional state; 2) after the game, the experimental group demonstrates increased skills in identifying all the five manipulation techniques (the number of respondents able to recognize a conspiracy theory increased by 6.8 times, emotional speech by 1.4 times; scapegoating by 7.6 times; attacking by 1.3 times, and a false dichotomy by 10.7 times); 3) in recognizing three manipulation techniques, the experimental and control groups show significant differences (p <0.001). After the game, the participants demonstrate the skills of accurately recognizing conspiracy theory, scapegoating, and false dichotomy. Conclusion. The suggested gaming practice has a high potential for developing critical thinking skills in relation to manipulation techniques on the Internet (in the context of digital interaction and information impact).

Advances in Speech Emotion Recognition and Analysis: A Review of Applied Machine Learning Methodologies

Abstract: The most important marketing tactics in the current environment is emotion detection based on individual’s interests, allowing extensive customization to cater to the varied needs of customers. For instance, contact centers may start playing music while a caller is on the line if they are agitated. A smart car that slows down when the driver is angry or scared is another illustration.The recognition of speech emotion has become a vital task in the field of computer science department . A very important aspect of human emotional state information is speech emotion that is the source of communication between human and computer interactions . The recognition of speech emotion is currently a very hot topic in the field of machine learning and deep learning and many systems have been build and are currently working on the recognition of these speech emotions using deep learning and neural networks algorithms. In this paper, various types of models and features like Mel-frequency Cepstral Coefficients (MFCC), Modulation spectral features (MSFs), Low-Level Descriptors(LLDs) for speech emotion recognition are categorized. Ultimately, the algorithms like recurrent neural network(RNN), support vector machine(SVM) used in different models are tabulated presenting their advantages and disadvantages along with their features and efficiency.

Creation of a diverse mixed-lingual emotional speech corpus with a framework for enhanced emotion detection

Speech Emotion Recognition (SER) has advanced considerably during the past 20 years. Till date, various SER systems have been developed for monolingual, multilingual and cross corpus contexts. However, in a country like India where numerous languages are spoken and often humans converse in more than one language, a dedicated SER system for mixed-lingual scenario is more crucial to be established which is the focus of this work. A self-recorded database that includes speech emotion samples with 11 diverse Indian languages has been developed. In parallel, a mixed-lingual database is formed with three popular standard databases of Berlin, Baum and SAVEE to represent mixed-lingual environment for western background. A detailed investigation of GeMAPS (Geneva Minimalistic Acoustic Parameter Set) feature set for mixed-lingual SER is performed. A distinct set of MFCC (Mel Frequency Cepstral Coefficients) coefficients derived from sine and cosine-based filter banks enriches the GeMAPS feature set and are proven to be robust for mixed-lingual emotion recognition. Various Machine Learning (ML) and Deep Learning (DL) algorithms have been applied for emotion recognition. The experimental results demonstrate GeMAPS features classified from ML has been quite robust for recognizing all the emotions across the mixed-lingual database of the western languages. However, with diverse recording conditions and languages of the Indian self-recorded database the GeMAPS with enriched features and classified using DL are proven to be significant for mixed-lingual emotion recognition.

IMPLEMENTATION OF THE PRINCIPLE OF EMOTIONALITY IN THE PROCESS OF LANGUAGE TRAINING OF FUTURE SPECIALISTS THROUGH DISTANCE EDUCATION: THE VIEWPOINT OF HIGHER EDUCATION ACQUIRES

The article raises the problem of implementing the principle of emotionality in the process of language training of future specialists in the conditions of distance learning. According to the author’s observations, in the educational scientific field, the principle of emotionality is the subject of research by many scientists. The purpose of the proposed study is to analyze the results of a questionnaire of higher education graduates of various educational fields regarding the problem of implementing the principle of emotionality in the process of language training of future specialists in the distance learning format. The researcher believes that language classes are a fertile ground for the implementation of the principle of emotionality and the formation of the emotional sphere of learners, since language performs an emotional function in the process of any type of activity ‒ learning, education, development, communication, etc., and is a tool for creating positive (or negative) influence on the personality, and therefore the basis of the formation of positive emotional thinking and speech (communication). In order to achieve the goal of the study, we developed a questionnaire and conducted a survey of higher education graduates. Among the respondents are students of “Publishing and editing”, “Journalism”, “International journalism”, “TV host”, “Law”, “Ukrainian language and literature” specialties. Students were asked to answer the following questions: 1. Does the emotional background of the class have an impact on learning and the desire to get an education? 2. Are distance learning sufficiently emotional? 3. In which learning format do you get more positive emotions ‒ offline or online? 4. Name the emotions that specific activities evoke in you. 5. What is the cause of negative emotion? 6. What emotions lead to motivational activities in class? 7. What, in your opinion (forms, techniques, means of teaching), could enliven classes in the Ukrainian language in a distance format? The results of the survey showed that there is a certain discrepancy between the expectations of students regarding the implementation of the principle of emotionality in distance learning classes and the actual state of the situation. The student needs dynamic educational content that increases positive emotions in class and serves as a motivator for educational activities. A positive emotion is provided to the student not so much by the role of a traditional passive subject who is given the necessary knowledge system, but by the role of an active analyst, researcher, coach, moderator, innovator, inventor, representative. Providing a positive emotional background in the conditions of distance learning is the basis for the formation of positive thinking and positive communication, and as a result, a high level of education.

Face and Speech Emotion Recognition System

Abstract: Emotions serve as the cornerstone of human communication, facilitating the expression of one's inner thoughts and feelings to others. Speech Emotion Recognition (SER) represents a pivotal endeavour aimed at deciphering the emotional nuances embedded within a speaker's voice signal. Universal emotions such as neutrality, anger, happiness, and sadness form the basis of this recognition process, allowing for the identification of fundamental emotional states. To achieve this, spectral and prosodic features are leveraged, each offering unique insights into the emotional content of speech. Spectral features, exemplified by the Mel Frequency Cepstral Coefficient (MFCC), provide a detailed analysis of the frequency distribution within speech signals, while prosodic features encompass elements like fundamental frequency, volume, pitch, speech intensity, and glottal parameters, capturing the rhythmic and tonal variations indicative of different emotional states. Through the integration of these features, SER systems can effectively simulate and classify a diverse range of emotional expressions, paving the way for enhanced human-computer interaction and communication technologies

Speech emotion recognition using a combination of variational mode decomposition and Hilbert transform

The objective of automated speech emotion recognition (SER) is to recognize each emotion of the speech signal uniquely and efficiently using machines like computers, mobile devices, etc. The popularity of SER has been widely acknowledged among researchers due to its extensive applicability in practical contexts. The use of SER has shown to be advantageous in several domains, including medical treatment, enhancement of security systems, surveillance operations, online marketing strategies, online educational platforms, online search engines, personal communication, customer relationship management, reinforcement of machine and human connection, and numerous other areas. Numerous authors have used various techniques, including the combination of multiple features (acoustic, non-acoustic, or both) and classifiers (machine learning, deep learning, or both), in order to enhance the efficacy of emotion categorization. In the present work, our objective is to enhance the performance of emotion classification (PECL) by using a combined approach of the variational mode decomposition (VMD) and Hilbert transform (HT) techniques. Using the VMD method, we decomposed the speech signal frame into many sub-signals, or intrinsic mode functions (IMFs). Then HT is applied to each VMD-based IMF signal to find the mode instantaneous amplitude (MIA) and mode instantaneous frequency (MIF) signal vectors. We extracted proposed features such as HT-based approximate entropy (HTAE), HT-based permutation entropy (HTPE), HT-based increment entropy (HTIE), and HT-based sample entropy (HTSE) using each MIA and MIF signal vector. The combination of the proposed HT-based features is called HT-based entropy (HTE) features. Then, we accessed the PECL using the HTE and MFCC features alone and in conjunction with a deep neural network (DNN) classifier. The experiment's results showed that the combinations of the proposed feature (MFCC + HTE) using a DNN classifier outperformed the individual features and obtained a SER accuracy of 86.92% for the EMOVO dataset and 91.63% for the EMO-DB dataset.

Machine Learning Based Speech Emotion Recognition System

In the last decade, there has been significant research into Automatic Speech Emotion Recognition (SER). The primary goal of SER is to improve human-machine interfaces. It can also monitor someone's psychological state for lie detection applications. Recently, speech emotion recognition has found uses in medicine and forensics. This paper recognizes 7 emotions using pitch and prosody features. The majority of speech features used here are in the time domain. A Support Vector Machine (SVM) classifier categorizes the emotions. The Berlin emotional database was used for this task. A good recognition rate of 81% was achieved. The reference paper for this work recognized 4 emotions and obtained a 94.2% recognition rate. However, the reference paper used a more complex hybrid classifier, while this work focuses on recognizing more emotions with a simpler model. . KeyWords: Blindness, Smart Stick, BlinDar, GPS, ESP8266, Internet of Things, RF Tx/Rx, MQ2.

Learning in Deep Radial Basis Function Networks.

Learning in neural networks with locally-tuned neuron models such as radial Basis Function (RBF) networks is often seen as instable, in particular when multi-layered architectures are used. Furthermore, universal approximation theorems for single-layered RBF networks are very well established; therefore, deeper architectures are theoretically not required. Consequently, RBFs are mostly used in a single-layered manner. However, deep neural networks have proven their effectiveness on many different tasks. In this paper, we show that deeper RBF architectures with multiple radial basis function layers can be designed together with efficient learning schemes. We introduce an initialization scheme for deep RBF networks based on k-means clustering and covariance estimation. We further show how to make use of convolutions to speed up the calculation of the Mahalanobis distance in a partially connected way, which is similar to the convolutional neural networks (CNNs). Finally, we evaluate our approach on image classification as well as speech emotion recognition tasks. Our results show that deep RBF networks perform very well, with comparable results to other deep neural network types, such as CNNs.

A Survey on Speech Emotion based Music Recommendation System

Psychological research has proven that music can reduce stress, improve mood, and help release “happy” chemicals like oxytocin, serotonin, and dopamine. It's no surprise that music has always been a popular tool in clinics and in the treatment of many ailments, so as people facing mental health issues continue to wander around the world, coping with mental illness is more important than ever. Although music recommendations already exist, they do not have all the scheduling algorithms that take into account the user's needs. Considering that it is undeniable that most people listen to music on a regular basis, music has also been shown to reduce stress, pain, and blood pressure while improving cognition, memory, and sleep quality. The most comprehensive and practical product that sorts all the good music. Our goal is to improve the emotional state of the user by creating a positive feeling based on the recommended music called "Viby".

Analysis Of Emotions Through Speech Recognition

Speech emotion recognition (SER) is a burgeoning field in AI that analyzes vocal characteristics to understand human emotions. It delves deeper than the literal meaning of words, uncovering emotional cues hidden within speech patterns. Pitch, loudness, and speech rate are just a few features that vary with emotional state. SER utilizes machine learning algorithms to classify these features into categories like happiness, sadness, or anger. This technology offers a treasure trove of possibilities, from enhancing human-computer interaction to revolutionizing customer service and even aiding in mental health assessments. As SER continues to evolve, it holds the potential to transform how we connect with machines, fostering deeper understanding and richer emotional experiences.

A Modified MFCC-Based Deep Learning Method for Emotion Classification from Speech

Speech, which is one of the most effective methods of communication, varies according to the emotions experienced by people and includes not only vocabulary but also information about emotions. With developing technologies, human-machine interaction is also improving. Emotional information to be extracted from voice signals is valuable for this interaction. For these reasons, studies on emotion recognition systems are increasing. In this study, sentiment analysis is performed using the Toronto Emotional Speech Set (TESS) created by University of Toronto. The voice data in the dataset is first preprocessed and then a new CNN-based deep learning method on it is compared. The voice files in the TESS dataset have been first obtained feature maps using the MFCC method, and then classification has been performed with this method based on the proposed neural network model. Separate models have been created with CNN and LSTM models for the classification process. The experiments show that the MFCC-applied CNN model achieves a better result with an accuracy of 99.5% than the existing methods for the classification of voice signals. The accuracy value of the CNN model shows that the proposed CNN model can be used for emotion classification from human voice data.