Emotion Detection Research Articles

Implementation of the YOLO Method for Detection of Human Emotions Based on Facial Mimics

Emotion detection through facial expression recognition plays an important role in everyday life, such as how to respond correctly to emotional expressions in social interactions, so that you can establish and build verbal or nonverbal communication with other people and so on. Facial expressions are facial changes in response to a person's emotional state, intentions, or social communication. Face detection is the first step that must be taken in facial analysis, including facial expression recognition. There are many methods that can be used to carry out the face detection process, such as the YOLO method. This YOLO method reframes object detection as a single regression problem, directly from image pixels to bounding box coordinates and class probabilities. By using the YOLO method, the process only needs to look once at the input image, to predict what objects are in the image and where those objects are. Based on the results of the tests carried out, the YOLO method can be used to detect human facial expressions with a success rate of 80%, with neutral, surprise and disgust facial expressions having a good level of accuracy and fear facial expressions having a poor accuracy level. The YOLO method is able to detect facial expressions of humans who wear accessories such as glasses.

Capturing multiple emotions from conversational data using fine tuned transformers

Emotion detection is one of the crucial topics of Natural Language Processing (NLP) in recent years, and now one of the biggest motivating factors in correct identification and interpretation of a wide range of emotional expressions in textual data. This research examines the use of Bidirectional Encoder Representations from Transformers (BERT), a trained transformer model for emotion detection in textual data. This analysis evaluates how good BERT is at identifying emotions such as surprise, anger, fear, happiness and sadness compared with ordinary machine learning as well as deep learning techniques. The addition of weighted emotions approach enhances the model performance and gives a deeper emotional context awareness making it more effective to deal with complicated emotional utterances. The purpose of the research is to develop a fine-tuned BERT model with a weighted emotion framework to enhance the accuracy of emotion classification in conversational text. The context is improving emotion recognition in scenarios where multiple emotions co-exist, addressing limitations in traditional models by capturing subtle and overlapping emotional expressions. In terms of training the methods multiple datasets are considered and also the research examines various models performance. The article further discusses possible application areas of BERT modified in light of NLP.

Advancements in Emotion Classification via Facial and Body Gesture Analysis: A Survey

ABSTRACTMachine vision has had a substantial impact on human‐computer interaction and psychological research. The ability of robots to generate realistic facial expressions has been enhanced by advancements in artificial intelligence, neural networks, and deep learning, thereby nurturing more profound emotive connections between humans and machines. This paper offers an in‐depth look of the most recent developments in emotion classification technologies, with a specific focus on the period from 2019 to 2023. This research addresses the integration of facial and body gesture analysis into emotion detection, emphasising the development of methods such as partial transfer learning, sign‐based measurement systems, and lightweight convolutional neural networks. By analysing the most recent research, the paper not only provides a comprehensive overview of the most advanced methods, but also emphasises their superiority over previous approaches in terms of efficiency, and applicability. The integration of gestures into emotion analysis is underscored as a critical area for further investigation, providing novel opportunities for comprehending and interpreting the intricate layers of nonverbal communication. The objective of this survey is to provide a resource for researchers and practitioners who are interested in advancing the field of emotion classification by utilising innovative methodologies.

Advanced functional connectivity analysis with integrated EEG signal enhancement and GTW-EEG-GAN for emotion detection

Emotion recognition is crucial in human-computer interaction and psychological research, utilizing modalities such as facial expressions, voice intonations, and EEG signals. This research investigates AI-driven techniques by employing Graph Neural Networks (GNNs) on functional connectivity matrices derived from EEG data to advance emotion recognition. Our approach integrates sophisticated preprocessing methods, including Integrated EEG Signal Enhancement (IESE) and novel augmentation techniques such as Gaussian Time Warping Generative Adversarial Network (GTW-EEG-GAN). This integration aims at enhancing data quality and diversity. Signal decomposition using EMD-Wavelet Hybrid Decomposition further refines feature extraction, enabling robust analysis of EEG signals. Functional connectivity metrics capture critical neural interactions intended for precise emotion characterization. The GNN architecture effectively processes these features, achieving significant accuracy improvements. Evaluations on the DEAP dataset demonstrate promising results, attaining 94.5% accuracy before hyperparameter tuning and further improving to 98.6% after tuning, highlighting the system's efficacy. The methodology showcases the integration of advanced signal processing techniques with AI, offering a comprehensive framework for future studies. The findings advocate potential applications in areas such as personalized mental health monitoring, adaptive learning environments, and responsive gaming experiences, demonstrating the broad impact and versatility of AI-driven EEG-based emotion recognition.

EEGER – A Model for Recognition of Human Emotion Using Brain Signal

Emotions significantly impact human thinking, judgment, health, and communication. EEG-based emotion detection has advanced with the use of Brain-Computer Interface (BCI) technology, proving more effective than other physiological data. Despite progress in affective computing, emotion recognition remains a challenge. However, it's increasingly common in brain–machine interfaces, and research shows EEG brain waves are valuable in identifying emotional states. This research introduces a novel automated system for emotion recognition using deep learning techniques on EEG data collected from the GAMEEMO dataset, where participants played emotional assessment games. The system is designed to identify four emotions experienced during gameplay. The proposed model, called EEGER, was trained exclusively on EEG signals and demonstrated a 99.99% accuracy with minimal computational time. Key to its efficiency is the use of LSTM (Long Short-Term Memory) classifiers, which simplify the process by automatically extracting relevant features. The system was tested across different learning rates and epoch values, showing that 10 epochs with a learning rate of 0.0001 were sufficient to achieve the best accuracy. EEGER was also compared with other methods like K-Nearest Neighbor (KNN), Linear Discriminant Analysis (LDA), and Adaptive Boosting (AdaBoost), outperforming them in both accuracy and efficiency. These findings suggest that EEGER offers a promising new approach to EEG-based emotion recognition, optimizing performance with lower complexity and computation time.

Digital transformation of mental health therapy by integrating digitalized cognitive behavioral therapy and eye movement desensitization and reprocessing.

Digital therapy has gained popularity in the mental health field because of its convenience and accessibility. One major benefit of digital therapy is its ability to address therapist shortages. Posttraumatic stress disorder (PTSD) is a debilitating mental health condition that can develop after an individual experiences or witnesses a traumatic event. Digital therapy is an important resource for individuals with PTSD who may not have access to traditional in-person therapy. Cognitive behavioral therapy (CBT) and eye movement desensitization and reprocessing (EMDR) are two evidence-based psychotherapies that have shown efficacy in treating PTSD. This paper examines the mechanisms and clinical symptoms of PTSD as well as the principles and applications of CBT and EMDR. Additionally, the potential of digital therapy, including internet-based CBT, video conferencing-based therapy, and exposure therapy using augmented and virtual reality, is explored. This paper also discusses the engineering techniques employed in digital psychotherapy, such as emotion detection models and text analysis, for assessing patients' emotional states. Furthermore, it addresses the challenges faced in digital therapy, including regulatory issues, hardware limitations, privacy and security concerns, and effectiveness considerations. Overall, this paper provides a comprehensive overview of the current state of digital psychotherapy for PTSD treatment and highlights the opportunities and challenges in this rapidly evolving field.

A REVIEW ON MACHINE LEARNING AND DEEP LEARNING TECHNIQUES FOR TEXTUAL EMOTION ANALYSIS ON SOCIAL NETWORKS

In recent times emotion detection has achieved significant attention in the field of Natural Language Processing (NLP) due to the abundance of text data available on different social network platforms like Twitter, LinkedIn and Reddit. This paper presents a thorough review of existing emotion detection techniques on text analysis. The methodology involves a comparative analysis of different machinelearning and deep learning models, approaches and datasets utilised for emotion detection. The article discusses the limitations and challenges of traditional methods and delves into the theoretical foundations of machine learning and deep learning techniques such as SVM, CNN and BERT. By exploring the current state-of-the-art in emotion detection from social networks, this review aims to provide insight for researchers to advance the field of emotion detection.

Investigating Digital Illiterate Classification Techniques Based on DeepFace Technology

This paper presents an algorithm to identify digital illiterates by analyzing age and emotions from facial recognition. In this paper, digital illiterates refer to people who struggle in using digital devices. The study assumed that older individuals who display surprised or angry facial expressions while using digital devices are more likely to be identified as digitally illiterate. For age detection, the study used MTCNN (Multi-task Cascaded Convolutional Networks), and for emotion detection, it employed the VGG-Face model. MTCNN detects facial features and landmarks to preprocess images and distinguish facial characteristics. The VGG-Face model uses convolution operations to analyze facial images and classify emotional states. The dataset used in the study consisted of 3,000 facial images collected from the internet. The research team categorized the images into faces of individuals aged over 50, angry expressions, and surprised expressions. The dataset included 411 individuals (13.7%) aged over 50, 163 individuals (5.4%) with angry expressions, and 145 individuals (4.8%) with surprised expressions. Accuracy was estimated by comparing the results from the DeepFace algorithm with those from the research team’s classifications. The DeepFace algorithm achieved 95.77% accuracy in detecting individuals aged over 50, 83.45% accuracy for surprise, and 76.07% for anger. The results demonstrate that it is possible to identify digital illiterates based on their age and emotional expressions and could enable the development of personalized services to directly or indirectly support digital illiterates, potentially improving digital accessibility.

Leveraging Large Language Model ChatGPT for enhanced understanding of end-user emotions in social media feedbacks

For software evolution, user feedback has become a meaningful way to improve applications. Recent studies show a significant increase in analyzing end-user feedback from various social media platforms for software evolution. However, less attention has been given to the end-user feedback for low-rating software applications. Also, such approaches are developed mainly on the understanding of human annotators who might have subconsciously tried for a second guess, questioning the validity of the methods. For this purpose, we proposed an approach that analyzes end-user feedback for low-rating applications to identify the end-user opinion types associated with negative reviews (an issue or bug). Also, we utilized Generative Artificial Intelligence (AI), i.e., ChatGPT, as an annotator and negotiator when preparing a truth set for the deep learning (DL) classifiers to identify end-user emotion. For the proposed approach, we first used the ChatGPT Application Programming Interface (API) to identify negative end-user feedback by processing 71853 reviews collected from 45 apps in the Amazon store. Next, a novel grounded theory is developed by manually processing end-user negative feedback to identify frequently associated emotion types, including anger, confusion, disgust, distrust, disappointment, fear, frustration, and sadness. Next, two datasets were developed, one with human annotators using a content analysis approach and the other using ChatGPT API with the identified emotion types. Next, another round is conducted with ChatGPT to negotiate over the conflicts with the human-annotated dataset, resulting in a conflict-free emotion detection dataset. Finally, various DL classifiers, including LSTM, BILSTM, CNN, RNN, GRU, BiGRU and BiRNN, are employed to identify their efficacy in detecting end-users emotions by preprocessing the input data, applying feature engineering, balancing the data set, and then training and testing them using a cross-validation approach. We obtained an average accuracy of 94%, 94%, 93%, 92%, 91%, 91%, and 85%, with LSTM, BILSTM, RNN, CNN, GRU, BiGRU and BiRNN, respectively, showing improved results with the truth set curated with human and ChatGPT. Using ChatGPT as an annotator and negotiator can help automate and validate the annotation process, resulting in better DL performances.

Enhancing Emotion Detection in Textual Data: A Comparative Analysis of Machine Learning Models and Feature Extraction Techniques

Enhancing Emotion Detection in Textual Data: A Comparative Analysis of Machine Learning Models and Feature Extraction Techniques

A facial and vocal emotion recognition system based on deep learning

The traditional teaching mode, mainly characterized by "cramming" instruction, often overlooks the absorption of knowledge by students during the teaching process. Even in recent years, the emerging "flipped classroom" model heavily relies on teachers' experience and expertise, making it challenging to quantify students' classroom learning states. Among young teachers lacking teaching experience, the replication of excellent teaching models faces difficulties. With the rapid development of artificial intelligence technology, deep learning and other AI techniques are increasingly applied to people's lives. Additionally, emotion computing technology is also becoming more sophisticated. In the field of education, the use of AI technology holds the potential to help teachers systematically and comprehensively assess teaching quality, promoting the sharing of high-quality educational resources nationally and globally. Existing emotion detection technologies often focus on single techniques such as image recognition or natural language processing, resulting in low accuracy in discerning human emotions and making it difficult to accurately identify emotional changes in complex scenarios such as classrooms. Furthermore, there is a lack of comprehensive emotional statistical analysis methods for audio and visual data. In light of these, we have designed and implemented a multi-modal emotion recognition and analysis system. Through testing in classroom settings, this system has the potential to assist classroom teaching and enhance teaching quality.

Deep Learning-based Emotion Recognition Algorithms in Music Performance

In the realm of artificial intelligence and musicology, emotion recognition in music performance has emerged as a pivotal area of research. This paper introduces EmoTrackNet, an integrated deep learning framework that combines sparse attention networks, enhanced one-dimensional residual Convolutional Neural Networks (CNNs) with an improved Inception module, and Gate Recurrent Units (GRUs). The synergy of these technologies aims to decode complex emotional cues embedded in music. Our methodology revolves around leveraging the sparse attention network to efficiently process temporal sequences, thereby capturing the intricate dynamics of musical pieces. The incorporation of the 1D residual CNN with an upgraded Inception module facilitates the extraction of nuanced features from audio signals, encompassing a broad spectrum of musical tones and textures. The GRU component further refines the model’s capability to retain and process sequential information over longer timeframes, essential for understanding evolving emotional expressions in music. We evaluated EmoTrackNet on the Soundtrack dataset a comprehensive collection of music pieces annotated with emotional labels. The results demonstrated remarkable improvements in the accuracy of emotion recognition, outperforming existing models. This enhanced performance can be attributed to the integrated approach, which efficiently combines the strengths of each component, leading to a more robust and sensitive emotion detection system. EmoTrackNet’s novel architecture and promising results pave the way for new avenues in musicology, particularly in understanding and interpreting the emotional depth of musical performances. This framework not only contributes significantly to the field of music emotion recognition but also has potential applications in music therapy, entertainment, and interactive media where emotional engagement is key.

Sentiment Analysis and Facial Expression Recognition in Customer Service Interactions

In the evolving landscape of digital customer service, the need for advanced methods to accurately understand and respond to customer emotions has become critical. Traditional systems often rely solely on textual data, missing non-verbal cues that significantly contribute to the customer's emotional state. This study proposes a combined approach integrating Facial Expression Recognition (FER) and Natural Language Processing (NLP) to enhance emotion detection accuracy in customer service interactions. The FER component employs Convolutional Neural Networks (CNNs) to analyze facial expressions, while the NLP component uses Long Short-Term Memory (LSTM) networks to process textual data. This multimodal system aims to provide a comprehensive understanding of customer emotions by capturing both verbal and non-verbal cues. Experiments demonstrate that the integrated FER and NLP model significantly outperforms standalone models, achieving an accuracy of 92.3%, compared to 85.2% for FER-only and 87.4% for NLP-only models. The results highlight the benefits of a multimodal approach, showing substantial improvements in both training and validation performance. This study also compares the proposed model with other state-of-the-art models such as the Deep Learning Assisted Semantic Text Analysis (DLSTA) and Multimodal Emotion Recognition using Deep Belief Networks (DBN). While DLSTA achieves higher accuracy in text-based emotion detection, and DBNs provide robust emotion classification by integrating various modalities, our model effectively balances the strengths of both visual and textual data. The findings suggest that integrating FER and NLP can significantly enhance the quality of customer service by enabling more empathetic and effective interactions. Future work will focus on optimizing computational efficiency, addressing data variability, and ensuring adaptability across diverse customer service scenarios.

Multi-Modal Emotion Recognition Feature Extraction and Data Fusion Methods Evaluation

Research into emotion detection is crucial because of the wide range of fields that can benefit from it, including healthcare, intelligent customer service, and education. In comparison to unimodal approaches, multimodal emotion recognition (MER) integrates many modalities including text, facial expressions, and voice to provide better accuracy and robustness. This article provides a historical and present-day overview of MER, focusing on its relevance, difficulties, and approaches. We examine several datasets, comparing and contrasting their features and shortcomings; they include IEMOCAP and MELD. Recent developments in deep learning approaches, particularly fusion strategies such as early, late, and hybrid fusion are covered in the literature review. Data redundancy, complicated feature extraction, and real-time detection are among the identified shortcomings. Our suggested technique enhances emotion recognition accuracy by using deep learning to extract features using a hybrid fusion approach. To overcome existing restrictions and advance the area of MER, this study intends to direct future investigations in the right direction. Examining various data fusion strategies, reviewing new methodologies in multimodal emotion identification, and identifying problems and research needs to make up the primary body of this work.

Hcnnxgboost: A Hybrid Cnn-Xgboost Approach for Effective Emotion Detection in Textual Data

in recent years, emotional analysis has become a key focus in computational studies, driven by the need to understand societal sentiments. this exploration is motivated by its many promising applications, such as community well-being evaluation, human-computer interaction, suicide prevention, and personalized recommendations. even with progress in other areas like identifying expressions from facial cues and speech, the study of text-based emotion recognition remains a fascinating field of research because machines struggle to interpret context, especially compared to human capabilities. our research addresses this by introducing a multiclass text-based emotion detection system that combines a cnn architecture with xgboost for improved classification in natural language processing. we pre-process publicly available datasets and use glove pre-trained word embeddings for better text representation. a major contribution of our work is enhancing the feature space by combining cnn probabilities with the original text data. the proposed hcnnxgboost model outperforms all other machine learning and deep learning algorithms across the emoint, isear, and crowdflower datasets, achieving f-scores of 90.1%, 87.4%, and 62.2%, respectively. experimental evaluations on benchmark datasets show better f-scores, confirming the effectiveness of our approach. comparisons with other classifiers highlight the enhanced performance and effectiveness of our hybrid cnnxgboost (hcnnxgboost) model, making it one of the best solutions for emotion classification in natural language processing tasks.

The study of the effect of preprocessing techniques for emotion detection on Amazon product review dataset

The study of the effect of preprocessing techniques for emotion detection on Amazon product review dataset

Public sentiment on the Israeli–Palestinian conflict: insights from YouTube, Mastodon, and Google Trends

ABSTRACT This research employs a comprehensive methodology to delve into public sentiments surrounding the Israeli–Palestinian conflict, incorporating unconventional data sources—YouTube comments and Mastodon. Departing from the traditional reliance on Twitter, our systematic approach involves keyword-driven content identification, leveraging Google Trends to establish five pivotal keywords. Mastodon searches employed straightforward hashtag strategies, while the intricacies of YouTube required an exclusive focus on official newspaper channels to mitigate polarization risks. Rigorous data cleaning ensued, retaining only English-language texts and eliminating extraneous elements. The resulting dataset was subjected to Sentiment Analysis and Emotion Detection, providing a nuanced understanding of public sentiments across platforms, totaling 253.3 K texts.

The Human Touch in the Age of Artificial Intelligence: A Literature Review on the Interplay of Emotional Intelligence and AI

Although artificial intelligence has made significant strides in analyzing extensive datasets and recognizing intricate patterns, it still struggles to replicate the nuanced understanding of human emotions. This study aims to explore the complex relationship between emotional intelligence (EI) and AI, investigating the potential for AI to enhance our understanding and development of EI. Emotion detection in AI often involves analyzing facial expressions, speech patterns, and physiological cues. However, incorporating cultural diversity and biases into training datasets can hinder accuracy and efficiency. A promising area of future research is the development of AI systems capable of not only perceiving emotions but also demonstrating emotional intelligence. Such systems could revolutionize human-robot interactions, fostering more genuine and empathetic connections. Delving into the relationship between EI and AI can gain a deeper understanding of AI's emotional intelligence, appreciating its capabilities and limitations. This knowledge can pave the way for more compassionate and effective human-AI interactions. This knowledge can inform the development of more sophisticated AI systems that can contribute to our own emotional growth and well-being.

Research Trend and Impact on Student Learning using Artificial Intelligence-Based Emotion Recognation: Systematic Bibliometric Analysis

Emotions have a significant function in the learning process. Good emotions such as enjoyment and curiosity can increase attention and comprehension, whereas negative emotions such as fear and boredom might impair academic achievement. Therefore, identifying and managing students' emotions is a critical step in developing an effective learning environment. In recent years, there has been a substantial progress in the application of artificial intelligence to detect emotions through facial expression analysis. Systematic literature reviews based on bibliometric analysis of artificial intelligence-based emotion recognition studies are still difficult to locate. This paper aims to undertake bibliometric and visualization analyses with VOSviewer for artificial intelligence-based emotion recognition. The evaluation includes 230 scholarly publications on emotion detection investigations indexed by Scopus quartiles Q1–Q4 from the Scopus database in the recent decade, namely 2014–2024. This bibliometric analysis has major implications for increasing academic, practical, and policy-related elements of artificial intelligence-based emotion recognition in education. By recognizing trends and major contributions in this subject, researchers, practitioners, and policymakers may work together to develop a more flexible and supportive learning environment, thereby boosting the overall quality of education.

EfficientNet Convolutional Neural Network with Gram Matrices Modules for Predicting Sadness Emotion

Images are becoming an attractive area of emotional analysis. Recognising emotions in the images of general nature is gaining more and more research attention. Such emotion recognition is more sophisticated and different from conventional computer tasks. Due to human subjectivity, ambiguous judgments, cultural and personal differences, there is no an unambiguous model for such emotion assessment. In this paper, we have chosen sadness as the main emotion, which has significant impact to the richness of human experience and the depth of personal meaning. The main hypothesis of our research is that by extending the capabilities of convolutional neural networks to integrate both deep and shallow layer feature maps, it is possible to improve the detection of sadness emotion in images. We have suggested integration of the different convolutional layers by taking the learned features from the selected layers and applying a pairwise operation to compute the Gram matrices of feature sub-maps. Our findings show that this approach improves the network’s ability to recognize sadness in the context of binary classification, resulting in a higher emotion recognition accuracy. We experimentally evaluated the proposed network for the stated binary classification problem under different parameters and datasets. The results demonstrate that the improved network achieves improved accuracy as compared to the baseline (EfficientNetV2) and the previous state-of-the-art model.