Emotion Detection Research Articles

Multimodal emotion recognition from audio and video

As humans we want to interact with a machine as we would with a person, in a way that it understands us, advises us, and looks after us with no human supervision. Current systems lack empathy and user understanding in spite of very effective logical reasoning. By predicting the emotions of the users, we are able to identify their needs and cater to them as best as possible. Emotion recognition in video and audio has many potential applications, including conversational agents, recommendation systems as well as systems for smart homes, mental illness care, virtual reality games, remote physical training, education and car-hailing services. The aim of the project is to develop an automatic emotion detection system based on voice and facial expression. We propose a model that highlights contextual, multimodal information for emotion detection and recognition. If systems can understand emotions and respond accordingly to behavioral patterns, we can anticipate artificial agents becoming our cognitive-consulting partners in our daily lives. Additionally, the project can be expanded to make interactions more natural and better suited to handle complex situations.

EmoTunes: Your Emotional Soundtrack

The human face is an important organ of an individual’s body and it especially plays an important role in extraction of an individual‘s behavior and emotional state. Manually segregating the list of songs and generating an appropriate playlist based on an individual’s emotional features is a very tedious, time consuming, labor intensive and upheld task. Various algorithms have been proposed and developed for automating the playlist generation process.However the proposed existing algorithms in use are computationally slow, less accurate and sometimes even require use of additional hardware like EEG or sensors. This proposed system based on facial expression extracted will generate a playlist automatically thereby reducing the effort and time involved in rendering the process manually. Thus the proposed system tends to reduce the computational time involved in obtaining the results and the overall cost of the designed system, thereby increasing the overall accuracy of the system. Testing of the system is done on both user dependent (dynamic) and user independent (static) dataset. Facial expressions are captured using an inbuilt camera. The accuracy of the emotion detection algorithm used in the system for real time images is around 85-90%, while for static images it is around 98- 100%.The proposed algorithm on an average calculated estimation takes around 0.95-1.05 sec to generate an emotion based music playlist. Thus, it yields better accuracy in terms of performance and computational time and reduces the designing cost, compared to the algorithms used in the literature survey. Keywords—Audio Emotion Recognition, Music Information Retrieval, Emotion Extraction Module, Audio Feature Extraction Module, Artificial Neural Networks, Confusion Matrix, Viola and Jones Face Detection

FACIAL IMAGE CAPTIONING USING DNN

Facial analysis, encompassing emotion, age, and gender detection, shows potential in various applications such as human-computer interaction, business, security, and health. This study delves into the development and evaluation of a deep neural network (DNN) model for facial emotion, age, and gender detection. Utilizing a convolutional neural network (CNN) architecture trained on diverse datasets for each task, our model proves effective in predicting facial features. The accuracy of needs assessment is X%, the marginal error (MAE) of age estimation is Y years, and the accuracy of gender classification is Z%.

EmoSonics: Emotion Detection via Voice and Speech Recognition

Understanding emotions from speech is like deciphering a rich tapestry of human expression in the realm of human-computer interaction. It's akin to listening to someone's tone and inflection to discern whether they're happy, surprised, or experiencing a range of other feelings. Researchers use a variety of techniques, from analyzing speech patterns to utilizing advanced technologies like fMRI, to decode these emotional cues. Emotions aren't just simple labels; they're complex and nuanced, demanding sophisticated methods for accurate interpretation. Some methods break emotions down into simple categories, while others embrace the intricacies of human emotion, treating them as continuous variables. Ultimately, the goal is to enhance communication between humans and computers by enabling machines to understand and respond appropriately to our emotional states. This pursuit underscores the significance of emotion detection in speech analysis and highlights the need for continually evolving methodologies in human-computer interaction research.

“The ChatGPT bot is causing panic now – but it’ll soon be as mundane a tool as Excel”: analysing topics, sentiment and emotions relating to ChatGPT on Twitter

AbstractChatGPT, a sophisticated chatbot system by OpenAI, gained significant attention and adoption in 2022 and 2023. By generating human-like conversations, it attracted over 100 million monthly users; however, there are concerns about the social impact of ChatGPT, including panic, misinformation and ethics. Twitter has become a platform for expressing views on ChatGPT and popular NLP approaches like topic modelling, sentiment analysis and emotion detection are commonly used to study public discourses on Twitter. While these approaches have limitations, an analytical process of existing best practices captures the evolving nature of these views. Previous studies have examined early reactions and topics associated with ChatGPT on Twitter but have not fully explored the combination of topics, sentiment and emotions, nor have they explicitly followed existing best practices. This study provides an overview of the views expressed on Twitter about ChatGPT by analysing 88,058 tweets from November 2022 to March 2023 to see if panic and concern were replicated in Twitter discourses. The topics covered human-like text generation, chatbot development, writing assistance, data training, efficiency, impact on business and cryptocurrency. Overall, the sentiment was predominantly positive, indicating that concerns surrounding ChatGPT were not widely replicated. However, sentiment fluctuated, with a decline observed around the launch of ChatGPT Plus. The discourse saw consistent patterns of trust and fear, with trust maintaining a steady presence until a decline potentially influenced by concerns about biases and misinformation. We discuss how our findings build upon existing research regarding ChatGPT by providing trajectories of topics, sentiment and emotions.

Text mining analysis of retail and consumer service leaders' sustainability narratives: Are they actually true?

This study investigates the relationship between the sustainability narratives of retail and consumer services leaders and their actual sustainability practices, with a focus on the impact of these narratives on consumer perceptions and engagement. Firm-generated content (FGC) has become a significant factor influencing consumer reactions, especially in the context of sustainability claims within the retail and consumer services sectors. Our research examines the alignment of linguistic and communicative elements in sustainability communications with the actual sustainability performance of companies employing a multifaceted analytical approach including content/theme analysis, sentiment analysis, emotion detection, and readability assessments. The findings reveal that thematic content related to sustainability, emotional engagement strategies, and the clarity of communication are shown to play vital roles in enhancing consumer trust and engagement with retail and consumer services firms. Moreover, readability and polarity are critical in shaping consumer perceptions of authenticity and reliability in sustainability narratives. The study underscores the importance of transparency and authenticity in sustainability communications, demonstrating that effective messaging can significantly influence consumer engagement with retail and consumer services firms. By bridging the gap between sustainability narratives and actual practices, the research contributes to a deeper understanding of strategic communication's role in fostering sustainable consumer behavior and trust in the retail and consumer services sector.

Application of Computer Vision Techniques to Study the Relationship between Mental Stress and Pupil Diameter among Student Population

Stress is a state of mental tension, which helps us to cope with challenges in our life. It makes us progressive when it is positive, but excessive negative stress that perseveres for a long time leads to a state of depressiveness. Longer stressed stage of a human being changes the size, functionality and frequency of response of many internal and external body parameters. By applying computer vision techniques, these changes of body parameters can be tracked to get useful information about the mental stress for a stress affected person. Many studies show the pupil diameter varies significantly with the effect of stress. Our work is based on the study of variation of pupil diameters of stress affected and not affected university students. With the application of different supervised machine learning algorithms, we have observed that the pupil dilates more in case of stress affected students than non-stressed students. We have also found that the pupils of the students dilates more when they were in positive emotional states than their negative emotional states. This work will be helpful for researchers who are working in the field of emotion detection and recognition and affective disorder analysis.

Depression Detection System Using Facial Recognition Using Python

This paper presents a comprehensive mental health assessment toolkit implemented in Python, consisting of two distinct modules. The first module focuses on depression detection through a quiz, while the second module performs emotion detection using libraries such as DeepFace and OpenCV. The Depression Detection Module aims to provide a self-assessment tool for individuals who suspect they may be experiencing symptoms of depression. The emotion detection module uses advanced computer vision techniques and the DeepFace library to analyze facial expressions and recognize emotional states in real time via a webcam or recorded images. OpenCV is used to capture and process images or video streams, while DeepFace's deep learning models accurately classify emotions, including happiness, sadness, anger, fear, surprise, and neutrality. While the system shows promise in contributing to the field of mental health screening, it is essential to address ethical considerations related to user privacy and consent. Striking a balance between technological advancements and ethical guidelines ensures the responsible and effective deployment of such tools.

Human Emotion Detection Using CNN and Transfer Learning

Facial emotion detection is a critical component in human-computer interaction, mental health assessment, and security systems. In this project, we propose a robust facial emotion detection system leveraging state-of-the-art deep learning techniques. Our system utilizes Convolutional Neural Networks (CNNs) to extract meaningful features from facial images and classify them into five distinct emotional categories: neutral, surprise, sad, happy, and angry. We conducted extensive experiments on a diverse dataset consisting of over 10,000 annotated facial images collected from various sources. Through data augmentation techniques such as rotation, translation, and flipping, we expanded the dataset to enhance model training. Additionally, we employed transfer learning by fine-tuning a pre-trained CNN model, ResNet50, on our dataset to leverage its learned features. This project presents a system for real-time emotion monitoring using computer vision techniques. The system utilizes the Haar Cascade Classifier for face detection in live webcam video streams. Furthermore, we evaluated the system's performance across different lighting conditions, poses, and facial occlusions to assess its robustness in real-world scenarios. Our results indicate that the system maintains consistent performance across diverse conditions, making it suitable for deployment in applications requiring real-time facial emotion recognition.

Recent Trends in Machine and Deep Learning for Verbal and Non-verbal Emotion Detection

Abstract: Emotion recognition, both verbal and non-verbal, is a crucial component of artificial intelligence, psychology, and human-computer interaction. Emotion recognition is an integral component that significantly contributes to the improvement of communication and interaction. The research endeavors to conduct a thorough analysis and synthesis of the most recent developments in Deep Learning (DL) and Machine Learning (ML) techniques. Specifically, the study concentrates on the recognition of both verbal and non-verbal emotions. In contrast to previous research concentrated on verbal or non-verbal emotion detection separately, the study attempts to reconcile the gap between the two by demonstrating how ML and DL can be utilized effectively to detect emotions. The study also examines new methods, including multimodal data and integration of contextual information. Additionally, the research has examined the ethical implications and difficulties associated with emotion detection technologies. Findings have also revealed the wide-reaching implications for various sectors, including healthcare, education, customer service, and entertainment, where comprehending human emotions plays a crucial role in enhancing user experience and outcomes. In conclusion, the study provides invaluable knowledge to practitioners and researchers, which may facilitate the development of more advanced and accurate systems.

Enhancing Multimodal Emotion Recognition through Attention Mechanisms in BERT and CNN Architectures

Emotion detection holds significant importance in facilitating human–computer interaction, enhancing the depth of engagement. By integrating this capability, we pave the way for forthcoming AI technologies to possess a blend of cognitive and emotional understanding, bridging the divide between machine functionality and human emotional complexity. This progress has the potential to reshape how machines perceive and respond to human emotions, ushering in an era of empathetic and intuitive artificial systems. The primary research challenge involves developing models that can accurately interpret and analyze emotions from both auditory and textual data, whereby auditory data require optimizing CNNs to detect subtle and intense emotional fluctuations in speech, and textual data necessitate access to large, diverse datasets to effectively capture nuanced emotional cues in written language. This paper introduces a novel approach to multimodal emotion recognition, seamlessly integrating speech and text modalities to accurately infer emotional states. Employing CNNs, we meticulously analyze speech using Mel spectrograms, while a BERT-based model processes the textual component, leveraging its bidirectional layers to enable profound semantic comprehension. The outputs from both modalities are combined using an attention-based fusion mechanism that optimally weighs their contributions. The proposed method here undergoes meticulous testing on two distinct datasets: Carnegie Mellon University’s Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) dataset and the Multimodal Emotion Lines Dataset (MELD). The results demonstrate superior efficacy compared to existing frameworks, achieving an accuracy of 88.4% and an F1-score of 87.9% on the CMU-MOSEI dataset, and a notable weighted accuracy (WA) of 67.81% and a weighted F1 (WF1) score of 66.32% on the MELD dataset. This comprehensive system offers precise emotion detection and introduces several significant advancements in the field.

FER-BHARAT: a lightweight deep learning network for efficient unimodal facial emotion recognition in Indian context

Humans' ability to manage their emotions has a big impact on their ability to plan and make decisions. In order to better understand people and improve human–machine interaction, researchers in affective computing and artificial intelligence are investigating the detection and recognition of emotions. However, different cultures have distinct ways of expressing emotions, and the existing emotion recognition datasets and models may not effectively capture the nuances of the Indian population. To address this gap, this study proposes custom-built lightweight Convolutional Neural Network (CNN) models that are optimized for accuracy and computational efficiency. These models are trained and evaluated on two Indian emotion datasets: The Indian Spontaneous Expression Dataset (ISED) and the Indian Semi Acted Facial Expression Database (iSAFE). The proposed CNN model with manual feature extraction provides remarkable accuracy improvement of 11.14% for ISED and 4.72% for iSAFE datasets as compared to baseline, while reducing the training time. The proposed model also surpasses the accuracy produced by pre-trained ResNet-50 model by 0.27% ISED and by 0.24% for the iSAFE dataset with significant improvement in training time of approximately 320 s for ISED and 60 s for iSAFE dataset. The suggested lightweight CNN model with manual feature extraction offers the advantage of being computationally efficient and more accurate compared to pre-trained model making it a more practical and efficient solution for emotion recognition among Indians.

Speech Emotion Recognition Based on Machine Learning

The speech is the most effective means of communication, to recognize the emotions in speech is the most crucial task. In this paper we are using the Artificial Neural Network to recognize the emotions in speech. Hence, providing an efficient and accurate technique for speech based emotion recognition is also an important task. This study is focused on seven basic human emotions (angry, disgust, fear, happy, neutral, surprise, sad). The training and validating accuracy and also lose can be seen in a graph while training the dataset.According to it confusion matrix for model is created. The features frequency, pitch, amplitude and format of speech is used ti recognize seven basic emotions from speech. Keywords: Emotion detection, Emotion classification, verbal emotion classification.

Emotion Detection Through Electrocardiogram Signal Classification in an IOT Environment with Deep Neural Networks

An ECG detects the health and rhythm of the heart by measuring the electric activity of the heart. It has also been demonstrated that a person's emotions may influence the electrical activity of the heart. As a result, studying the electrical behaviour of the heart may simply determine a person's cardiac state and emotional wellness. IoT is a new technology that is quickly gaining acceptance throughout the world. Anybody, at any time, from anywhere, may connect to any network or service because to the extraordinary power and capacity of IoT. IoT-enabled devices have revolutionized the medical business by providing new capabilities such as remote patient monitoring and self-monitoring. This research proposed an IoT-based ECG monitoring system that employs a heart rate sensor to generate data and an intelligent hybrid classification algorithm to categorize the data. ECG monitoring has become a widely used method for detecting cardiac problems. The following are the primary contributions of this paper: To begin, this paper describes WISE (Wearable IoT-cloud-based health monitoring system), a one-of-a-kind system for real-time personal health monitoring. WISE makes use of the BASN (body area sensor network) technology to provide real-time health monitoring. WISE rapidly transfers data from the BASN to the cloud, and a lightweight wearable LCD may be included to enable quick access to real-time data. This model can address the issue of class imbalance in the ECG dataset, assisting in the development of an IoT-based smart and accurate healthcare system. Pre-processing, feature extraction, and classification are the three steps in any classification technique, whether it is emotion classification or heart health classification. Sensors are used to collect an ECG signal from a person's outside body. The provided ECG signal is first pre-processed using the Butterworth Filtering Method, which effectively reduces noise from the signal. Following pre-processing, the Adaptive Discrete Wavelet Transform technique is used to anticipate the signal's attributes. Lastly, a decision making classification approach based on relational weights is used to determine if the ECG signal is normal or abnormal.

Emotion Detector and Solution Provider Using Voice Model

Emotion detection from images is one of the most high-powered tasks in today’s world and AI detection plays a fundamental role, this project aims to understand human emotions which play a major role in well-being, this project aims to dive deep into human emotions and provide a variety of solutions, through Human-computer interaction takes place, emotion detection and voice signals are input signal The project is classified into two major components: one where a human emo- tion is detected through camera is detected and facial recognition takes place. Secondly, where a solution is provided to a recorded emotion using a voice model, it creates a conversation which result turn provides a solution based on the collected database. The paper presents a theory-based framework of technologies discusses challenges and behavioural considerations, and gives directions for future practices and use. Index Terms—Voice GPT, Artificial intelligence (AI), Facial emotion recognition (FER), Emotion Detection (ED), Open CV Library (OCL) Convolutional neural networks (CNN), Machine Learning (ML), Tensor flow, GTTP, Keras

Speech Emotion Recognition Using Machine Learning

The speech is the most effective means of communication, to recognize the emotions in speech is the most crucial task. In this paper we are using the Artificial Neural Network to recognize the emotions in speech. Hence, providing an efficient and accurate technique for speech based emotion recognition is also an important task. This study is focused on seven basic human emotions (angry, disgust, fear, happy, neutral, surprise, sad). The training and validating accuracy and also lose can be seen in a graph while training the dataset. According to it confusion matrix for model is created. The seven features Frequency, Pitch, Amplitude and formant of speech is used to recognize seven basic emotions from speech. Keywords: Emotion detection, Emotion classification, verbal emotion classification.

Detecting emotions through EEG signals based on modified convolutional fuzzy neural network

Emotion is a human sense that can influence an individual’s life quality in both positive and negative ways. The ability to distinguish different types of emotion can lead researchers to estimate the current situation of patients or the probability of future disease. Recognizing emotions from images have problems concealing their feeling by modifying their facial expressions. This led researchers to consider Electroencephalography (EEG) signals for more accurate emotion detection. However, the complexity of EEG recordings and data analysis using conventional machine learning algorithms caused inconsistent emotion recognition. Therefore, utilizing hybrid deep learning models and other techniques has become common due to their ability to analyze complicated data and achieve higher performance by integrating diverse features of the models. However, researchers prioritize models with fewer parameters to achieve the highest average accuracy. This study improves the Convolutional Fuzzy Neural Network (CFNN) for emotion recognition using EEG signals to achieve a reliable detection system. Initially, the pre-processing and feature extraction phases are implemented to obtain noiseless and informative data. Then, the CFNN with modified architecture is trained to classify emotions. Several parametric and comparative experiments are performed. The proposed model achieved reliable performance for emotion recognition with an average accuracy of 98.21% and 98.08% for valence (pleasantness) and arousal (intensity), respectively, and outperformed state-of-the-art methods.

Speech Emotion Recognition Using ANN

The speech is the most effective means of communication, to recognize the emotions in speech is the most crucial task. In this paper we are using the Artificial Neural Network to recognize the emotions in speech. Hence, providing an efficient and accurate technique for speech based emotion recognition is also an important task. This study is focused on seven basic human emotions (angry, disgust, fear, happy, neutral, surprise, sad). The training and validating accuracy and also lose can be seen in a graph while training the dataset. According to it confusion matrix for model is created. The seven features Frequency, Pitch, Amplitude and formant of speech is used to recognize seven basic emotions from speech. Keywords: Emotion detection, Emotion classification, verbal emotion classification.

Comparative Evaluation of CNN-LSTM Model for Emotion Detection in Indonesian Text

Comparative Evaluation of CNN-LSTM Model for Emotion Detection in Indonesian Text

Enhancements in Immediate Speech Emotion Detection: Harnessing Prosodic and Spectral Characteristics

Speech is essential to human communication for expressing and understanding feelings. Emotional speech processing has challenges with expert data sampling, dataset organization, and computational complexity in large-scale analysis. This study aims to reduce data redundancy and high dimensionality by introducing a new speech emotion recognition system. The system employs Diffusion Map to reduce dimensionality and includes Decision Trees and K-Nearest Neighbors(KNN)ensemble classifiers. These strategies are suggested to increase voice emotion recognition accuracy. Speech emotion recognition is gaining popularity in affective computing for usage in medical, industry, and academics. This project aims to provide an efficient and robust real-time emotion identification framework. In order to identify emotions using supervised machine learning models, this work makes use of paralinguistic factors such as intensity, pitch, and MFCC. In order to classify data, experimental analysis integrates prosodic and spectral information utilizing methods like Random Forest, Multilayer Perceptron, SVM, KNN, and Gaussian Naïve Bayes. Fast training times make these machine learning models excellent for real-time applications. SVM and MLP have the highest accuracy at 70.86% and 79.52%, respectively. Comparisons to benchmarks show significant improvements over earlier models.