Emotion Detection Research Articles

A facial and vocal emotion recognition system based on deep learning

The traditional teaching mode, mainly characterized by "cramming" instruction, often overlooks the absorption of knowledge by students during the teaching process. Even in recent years, the emerging "flipped classroom" model heavily relies on teachers' experience and expertise, making it challenging to quantify students' classroom learning states. Among young teachers lacking teaching experience, the replication of excellent teaching models faces difficulties. With the rapid development of artificial intelligence technology, deep learning and other AI techniques are increasingly applied to people's lives. Additionally, emotion computing technology is also becoming more sophisticated. In the field of education, the use of AI technology holds the potential to help teachers systematically and comprehensively assess teaching quality, promoting the sharing of high-quality educational resources nationally and globally. Existing emotion detection technologies often focus on single techniques such as image recognition or natural language processing, resulting in low accuracy in discerning human emotions and making it difficult to accurately identify emotional changes in complex scenarios such as classrooms. Furthermore, there is a lack of comprehensive emotional statistical analysis methods for audio and visual data. In light of these, we have designed and implemented a multi-modal emotion recognition and analysis system. Through testing in classroom settings, this system has the potential to assist classroom teaching and enhance teaching quality.

Deep Learning-based Emotion Recognition Algorithms in Music Performance

In the realm of artificial intelligence and musicology, emotion recognition in music performance has emerged as a pivotal area of research. This paper introduces EmoTrackNet, an integrated deep learning framework that combines sparse attention networks, enhanced one-dimensional residual Convolutional Neural Networks (CNNs) with an improved Inception module, and Gate Recurrent Units (GRUs). The synergy of these technologies aims to decode complex emotional cues embedded in music. Our methodology revolves around leveraging the sparse attention network to efficiently process temporal sequences, thereby capturing the intricate dynamics of musical pieces. The incorporation of the 1D residual CNN with an upgraded Inception module facilitates the extraction of nuanced features from audio signals, encompassing a broad spectrum of musical tones and textures. The GRU component further refines the model’s capability to retain and process sequential information over longer timeframes, essential for understanding evolving emotional expressions in music. We evaluated EmoTrackNet on the Soundtrack dataset a comprehensive collection of music pieces annotated with emotional labels. The results demonstrated remarkable improvements in the accuracy of emotion recognition, outperforming existing models. This enhanced performance can be attributed to the integrated approach, which efficiently combines the strengths of each component, leading to a more robust and sensitive emotion detection system. EmoTrackNet’s novel architecture and promising results pave the way for new avenues in musicology, particularly in understanding and interpreting the emotional depth of musical performances. This framework not only contributes significantly to the field of music emotion recognition but also has potential applications in music therapy, entertainment, and interactive media where emotional engagement is key.

Sentiment Analysis and Facial Expression Recognition in Customer Service Interactions

In the evolving landscape of digital customer service, the need for advanced methods to accurately understand and respond to customer emotions has become critical. Traditional systems often rely solely on textual data, missing non-verbal cues that significantly contribute to the customer's emotional state. This study proposes a combined approach integrating Facial Expression Recognition (FER) and Natural Language Processing (NLP) to enhance emotion detection accuracy in customer service interactions. The FER component employs Convolutional Neural Networks (CNNs) to analyze facial expressions, while the NLP component uses Long Short-Term Memory (LSTM) networks to process textual data. This multimodal system aims to provide a comprehensive understanding of customer emotions by capturing both verbal and non-verbal cues. Experiments demonstrate that the integrated FER and NLP model significantly outperforms standalone models, achieving an accuracy of 92.3%, compared to 85.2% for FER-only and 87.4% for NLP-only models. The results highlight the benefits of a multimodal approach, showing substantial improvements in both training and validation performance. This study also compares the proposed model with other state-of-the-art models such as the Deep Learning Assisted Semantic Text Analysis (DLSTA) and Multimodal Emotion Recognition using Deep Belief Networks (DBN). While DLSTA achieves higher accuracy in text-based emotion detection, and DBNs provide robust emotion classification by integrating various modalities, our model effectively balances the strengths of both visual and textual data. The findings suggest that integrating FER and NLP can significantly enhance the quality of customer service by enabling more empathetic and effective interactions. Future work will focus on optimizing computational efficiency, addressing data variability, and ensuring adaptability across diverse customer service scenarios.

Multi-Modal Emotion Recognition Feature Extraction and Data Fusion Methods Evaluation

Research into emotion detection is crucial because of the wide range of fields that can benefit from it, including healthcare, intelligent customer service, and education. In comparison to unimodal approaches, multimodal emotion recognition (MER) integrates many modalities including text, facial expressions, and voice to provide better accuracy and robustness. This article provides a historical and present-day overview of MER, focusing on its relevance, difficulties, and approaches. We examine several datasets, comparing and contrasting their features and shortcomings; they include IEMOCAP and MELD. Recent developments in deep learning approaches, particularly fusion strategies such as early, late, and hybrid fusion are covered in the literature review. Data redundancy, complicated feature extraction, and real-time detection are among the identified shortcomings. Our suggested technique enhances emotion recognition accuracy by using deep learning to extract features using a hybrid fusion approach. To overcome existing restrictions and advance the area of MER, this study intends to direct future investigations in the right direction. Examining various data fusion strategies, reviewing new methodologies in multimodal emotion identification, and identifying problems and research needs to make up the primary body of this work.

Hcnnxgboost: A Hybrid Cnn-Xgboost Approach for Effective Emotion Detection in Textual Data

in recent years, emotional analysis has become a key focus in computational studies, driven by the need to understand societal sentiments. this exploration is motivated by its many promising applications, such as community well-being evaluation, human-computer interaction, suicide prevention, and personalized recommendations. even with progress in other areas like identifying expressions from facial cues and speech, the study of text-based emotion recognition remains a fascinating field of research because machines struggle to interpret context, especially compared to human capabilities. our research addresses this by introducing a multiclass text-based emotion detection system that combines a cnn architecture with xgboost for improved classification in natural language processing. we pre-process publicly available datasets and use glove pre-trained word embeddings for better text representation. a major contribution of our work is enhancing the feature space by combining cnn probabilities with the original text data. the proposed hcnnxgboost model outperforms all other machine learning and deep learning algorithms across the emoint, isear, and crowdflower datasets, achieving f-scores of 90.1%, 87.4%, and 62.2%, respectively. experimental evaluations on benchmark datasets show better f-scores, confirming the effectiveness of our approach. comparisons with other classifiers highlight the enhanced performance and effectiveness of our hybrid cnnxgboost (hcnnxgboost) model, making it one of the best solutions for emotion classification in natural language processing tasks.

Emotion Recognition on Turkish Mobile Operator Turkcell’s Call Center Calls

A fundamental component of human intelligence is the capacity for feeling. In addition to being founded on logic and reason, human conduct is also greatly influenced by the emotions that people experience. For the purpose of this study, we classified one thousand real-life call center client voice data in the Turkish language based on the way they expressed their emotions using text emotion detection. We made use of Ekman's emotional labeling and techniques from the field of artificial intelligence, such as deep learning and other similar methods.

The study of the effect of preprocessing techniques for emotion detection on Amazon product review dataset

The study of the effect of preprocessing techniques for emotion detection on Amazon product review dataset

Public sentiment on the Israeli–Palestinian conflict: insights from YouTube, Mastodon, and Google Trends

ABSTRACT This research employs a comprehensive methodology to delve into public sentiments surrounding the Israeli–Palestinian conflict, incorporating unconventional data sources—YouTube comments and Mastodon. Departing from the traditional reliance on Twitter, our systematic approach involves keyword-driven content identification, leveraging Google Trends to establish five pivotal keywords. Mastodon searches employed straightforward hashtag strategies, while the intricacies of YouTube required an exclusive focus on official newspaper channels to mitigate polarization risks. Rigorous data cleaning ensued, retaining only English-language texts and eliminating extraneous elements. The resulting dataset was subjected to Sentiment Analysis and Emotion Detection, providing a nuanced understanding of public sentiments across platforms, totaling 253.3 K texts.

The Human Touch in the Age of Artificial Intelligence: A Literature Review on the Interplay of Emotional Intelligence and AI

Although artificial intelligence has made significant strides in analyzing extensive datasets and recognizing intricate patterns, it still struggles to replicate the nuanced understanding of human emotions. This study aims to explore the complex relationship between emotional intelligence (EI) and AI, investigating the potential for AI to enhance our understanding and development of EI. Emotion detection in AI often involves analyzing facial expressions, speech patterns, and physiological cues. However, incorporating cultural diversity and biases into training datasets can hinder accuracy and efficiency. A promising area of future research is the development of AI systems capable of not only perceiving emotions but also demonstrating emotional intelligence. Such systems could revolutionize human-robot interactions, fostering more genuine and empathetic connections. Delving into the relationship between EI and AI can gain a deeper understanding of AI's emotional intelligence, appreciating its capabilities and limitations. This knowledge can pave the way for more compassionate and effective human-AI interactions. This knowledge can inform the development of more sophisticated AI systems that can contribute to our own emotional growth and well-being.

Research Trend and Impact on Student Learning using Artificial Intelligence-Based Emotion Recognation: Systematic Bibliometric Analysis

Emotions have a significant function in the learning process. Good emotions such as enjoyment and curiosity can increase attention and comprehension, whereas negative emotions such as fear and boredom might impair academic achievement. Therefore, identifying and managing students' emotions is a critical step in developing an effective learning environment. In recent years, there has been a substantial progress in the application of artificial intelligence to detect emotions through facial expression analysis. Systematic literature reviews based on bibliometric analysis of artificial intelligence-based emotion recognition studies are still difficult to locate. This paper aims to undertake bibliometric and visualization analyses with VOSviewer for artificial intelligence-based emotion recognition. The evaluation includes 230 scholarly publications on emotion detection investigations indexed by Scopus quartiles Q1–Q4 from the Scopus database in the recent decade, namely 2014–2024. This bibliometric analysis has major implications for increasing academic, practical, and policy-related elements of artificial intelligence-based emotion recognition in education. By recognizing trends and major contributions in this subject, researchers, practitioners, and policymakers may work together to develop a more flexible and supportive learning environment, thereby boosting the overall quality of education.

EfficientNet Convolutional Neural Network with Gram Matrices Modules for Predicting Sadness Emotion

Images are becoming an attractive area of emotional analysis. Recognising emotions in the images of general nature is gaining more and more research attention. Such emotion recognition is more sophisticated and different from conventional computer tasks. Due to human subjectivity, ambiguous judgments, cultural and personal differences, there is no an unambiguous model for such emotion assessment. In this paper, we have chosen sadness as the main emotion, which has significant impact to the richness of human experience and the depth of personal meaning. The main hypothesis of our research is that by extending the capabilities of convolutional neural networks to integrate both deep and shallow layer feature maps, it is possible to improve the detection of sadness emotion in images. We have suggested integration of the different convolutional layers by taking the learned features from the selected layers and applying a pairwise operation to compute the Gram matrices of feature sub-maps. Our findings show that this approach improves the network’s ability to recognize sadness in the context of binary classification, resulting in a higher emotion recognition accuracy. We experimentally evaluated the proposed network for the stated binary classification problem under different parameters and datasets. The results demonstrate that the improved network achieves improved accuracy as compared to the baseline (EfficientNetV2) and the previous state-of-the-art model.

Enhancing emotion detection with synergistic combination of word embeddings and convolutional neural networks

Recognizing emotions in textual data is crucial in a wide range of natural language processing (NLP) applications, from consumer sentiment research to mental health evaluation. The word embedding techniques play a pivotal role in text processing. In this paper, the performance of several well-known word embedding methods is evaluated in the context of emotion recognition. The classification of emotions is further enhanced using a convolutional neural network (CNN) model because of its propensity to capture local patterns and its recent triumphs in text-related tasks. The integration of CNN with word embedding techniques introduced an additional layer to the landscape of emotion detection from text. The synergy between word embedding techniques and CNN harnesses the strengths of both approaches. CNNs extract local patterns and features from sequential data, making them well-suited for capturing relevant information within the embeddings. The results obtained with various embeddings highlight the significance of choosing synergistic combinations for optimum performance. The combination of CNNs and word embeddings proved a versatile and effective approach.

Emotion Detection using EEG Signals in Image and Video Processing: A Survey

Brain serves as the body's processing of knowledge and management centre. The central nervous system directly produces ElectroEncephaloGram (EEG) physiological signals, which are strongly associated with human emotions. In the upcoming years, there will be an increase in interest for identifying emotion using brain waves by EEG (ElectroEncephaloGraphy) signals. It takes efficient and effective signal processing and feature extraction techniques for detection of emotions from human biological brain signals. Current approaches gather valuable information from a fixed number of ElectroEncephaloGraphy (EEG) channels utilizing variety of methodologies. This work analyses the different difficulties and problems associated with EEG signals for emotion identification and provides a comprehensive summary of several contemporary approaches. Pre-processing, feature extraction, and categorization are the first steps in the process of recognizing emotions from EEG signals. The main goal of this survey is to sought and enhance brain signal-based emotion detection ability by comparing all novel and adaptive channel selection technique that recognize the distinct changes in brain activities that varies between individuals and emotional states.

Implementation of Convolutional Neural Networks (CNN) in An Emotion Detection System for Measuring Learning Concentration Levels

Technological advancements have had a significant impact on the education sector, including the application of Convolutional Neural Networks (CNN) for facial image analysis. This research aims to implement CNN to measure students' learning concentration levels. The FER2013 dataset, which includes seven emotion classifications and comprises 28,709 images for training data, is used as the database. The data is processed through rescaling and augmentation to prepare the CNN model. The model consists of several convolutional layers, pooling layers, and fully connected layers designed to extract crucial features from facial images. Evaluation results demonstrate a very high accuracy of 94.95% on training data, indicating that the model effectively recognizes complex patterns within the data. Although there is a higher loss value of 157% and a decreased accuracy of 62.75% on validation data, this suggests that the model possesses a strong foundational capability and can still be improved through further adjustments. With high accuracy in training and promising validation results, the model shows substantial potential for real-world application, where it can assist teachers in understanding students' emotional responses in real-time. The implementation of CNN aids educators in comprehending students' emotional responses and adapting their teaching methods more effectively, thereby creating a more conducive learning environment and enhancing students' academic and social development. These findings also open opportunities for further research to improve the performance and generalization of the model on unseen data, making this technology an increasingly reliable tool in education

Advancing emotion recognition in facial expressions through PCA, RFE, and MLP Integration

Emotion recognition through facial expressions is vital for enhancing human-computer interaction, making systems more intuitive and responsive to user needs. This study introduces an innovative approach to emotion detection, leveraging Principal Component Analysis (PCA) and Recursive Feature Elimination (RFE) for feature extraction and optimization. The methodology focuses on refining facial feature representations to improve classification accuracy, which is critical for accurately detecting emotions like happiness, sadness, fear, and surprise. The approach was applied to two widely recognized facial emotion datasets: the Cohn-Kanade (CK) and the Japanese Female Facial Expression (JAFFE) datasets. By integrating PCA and RFE, the model efficiently selects the most relevant features, enhancing the overall performance of emotion recognition. A comparative analysis with existing deep learning models highlights the advantages of the proposed method. The effectiveness of this approach is further supported by the results, where the model achieves an accuracy of 98.49% on the CK dataset and 96.03% on the JAFFE dataset. These results demonstrate a significant improvement over recent methods, indicating the model's potential for real-world applications.

Integrating IoMT and AI for Proactive Healthcare: Predictive Models and Emotion Detection in Neurodegenerative Diseases

Neurodegenerative diseases, such as Parkinson’s and Alzheimer’s, present considerable challenges in their early detection, monitoring, and management. The paper presents NeuroPredict, a healthcare platform that integrates a series of Internet of Medical Things (IoMT) devices and artificial intelligence (AI) algorithms to address these challenges and proactively improve the lives of patients with or at risk of neurodegenerative diseases. Sensor data and data obtained through standardized and non-standardized forms are used to construct detailed models of monitored patients’ lifestyles and mental and physical health status. The platform offers personalized healthcare management by integrating AI-driven predictive models that detect early symptoms and track disease progression. The paper focuses on the NeuroPredict platform and the integrated emotion detection algorithm based on voice features. The rationale for integrating emotion detection is based on two fundamental observations: (a) there is a strong correlation between physical and mental health, and (b) frequent negative mental states affect quality of life and signal potential future health declines, necessitating timely interventions. Voice was selected as the primary signal for mood detection due to its ease of acquisition without requiring complex or dedicated hardware. Additionally, voice features have proven valuable in further mental health assessments, including the diagnosis of Alzheimer’s and Parkinson’s diseases.

Retraction notice: Emotion detection on webpages using biosensors integrated to a window-based dynamic control system

Retraction notice: Emotion detection on webpages using biosensors integrated to a window-based dynamic control system

Speech Emotion Recognition Using openSMILE and GPT 3.5 Transformer.

In recent years, artificial intelligence, and machine learning (ML) models have advanced significantly, offering transformative solutions across diverse sectors. Emotion recognition in speech has particularly benefited from ML techniques, revolutionizing its accuracy and applicability. This article proposes a method for emotion detection in Romanian speech analysis by combining two distinct approaches: semantic analysis using GPT Transformer and acoustic analysis using openSMILE. The results showed an accuracy of 74% and a precision of almost 82%. Several system limitations were observed due to the limited and low-quality dataset. However, it also opened a new horizon in our research by analyzing emotions to identify mental health disorders.

EEG Based Emotion Detection by Using Modified Tunicate Swarm Optimization Algorithm

EEG Based Emotion Detection by Using Modified Tunicate Swarm Optimization Algorithm

EmotionCast: An Emotion-Driven Intelligent Broadcasting System for Dynamic Camera Switching.

Traditional broadcasting methods often result in fatigue and decision-making errors when dealing with complex and diverse live content. Current research on intelligent broadcasting primarily relies on preset rules and model-based decisions, which have limited capabilities for understanding emotional dynamics. To address these issues, this study proposed and developed an emotion-driven intelligent broadcasting system, EmotionCast, to enhance the efficiency of camera switching during live broadcasts through decisions based on multimodal emotion recognition technology. Initially, the system employs sensing technologies to collect real-time video and audio data from multiple cameras, utilizing deep learning algorithms to analyze facial expressions and vocal tone cues for emotion detection. Subsequently, the visual, audio, and textual analyses were integrated to generate an emotional score for each camera. Finally, the score for each camera shot at the current time point was calculated by combining the current emotion score with the optimal scores from the preceding time window. This approach ensured optimal camera switching, thereby enabling swift responses to emotional changes. EmotionCast can be applied in various sensing environments such as sports events, concerts, and large-scale performances. The experimental results demonstrate that EmotionCast excels in switching accuracy, emotional resonance, and audience satisfaction, significantly enhancing emotional engagement compared to traditional broadcasting methods.