Facial Emotion Recognition Research Articles

Human Emotion Detection Using CNN and Transfer Learning

Facial emotion detection is a critical component in human-computer interaction, mental health assessment, and security systems. In this project, we propose a robust facial emotion detection system leveraging state-of-the-art deep learning techniques. Our system utilizes Convolutional Neural Networks (CNNs) to extract meaningful features from facial images and classify them into five distinct emotional categories: neutral, surprise, sad, happy, and angry. We conducted extensive experiments on a diverse dataset consisting of over 10,000 annotated facial images collected from various sources. Through data augmentation techniques such as rotation, translation, and flipping, we expanded the dataset to enhance model training. Additionally, we employed transfer learning by fine-tuning a pre-trained CNN model, ResNet50, on our dataset to leverage its learned features. This project presents a system for real-time emotion monitoring using computer vision techniques. The system utilizes the Haar Cascade Classifier for face detection in live webcam video streams. Furthermore, we evaluated the system's performance across different lighting conditions, poses, and facial occlusions to assess its robustness in real-world scenarios. Our results indicate that the system maintains consistent performance across diverse conditions, making it suitable for deployment in applications requiring real-time facial emotion recognition.

Decomposing the variance in early maladaptive schemas: the major role of one general factor, the minor role of domains, and their differential relations to facial emotion recognition.

Despite the growing interest in the early maladaptive schemas, the progress in understanding their impacts is decelerated by a lack of clear understanding of their structure. Different composite scores are calculated without a solid ground or a clarified meaning. Here we explain that the schema variance can be theoretically decomposed into three components: schema-specific, domain-specific due to the unmet core needs, and the common variance we call general susceptibility; each can differentially correlate with other substantive variables. Using this framework, we empirically examine the structure of schemas and their relationships to facial emotion recognition, a crucial ability that can widely affect our social interactions. A sample of adults completed an emotion recognition task and the Young Schema Questionnaire. Using different factor models, the specific and shared variance across schemas was analyzed. Then, the relation of these variance components to facial emotion recognition was explored. A general factor explained 27%, 40%, and 64% of the total variance in items, schemas, and domains, respectively. Partialling out the common variance, there was little domain-specific variance remained. Regarding facial emotion recognition, they were not correlated with specific schemas; however, the general susceptibility factor was correlated with anger recognition. The variance decomposition approach to schemas, which uses the bifactor model, may offer a clearer way to explore the impacts of schemas. While domain scores are widely used, their reliability, validity, and meaning are questionable. The generic factor, which is consistently extractable from empirical data, requires further attention.

Revolutionizing getaways: automation and artificial intelligence’s transformative journey in tourism and hospitality

PurposeThis theme issue aims to explore the depth and breadth of automation and artificial intelligence’s (AI’s) impact on the hospitality and tourism industry.Design/methodology/approachThe thematic examination of the articles presented in this theme issue highlights the central results concerning the strategic inquiry, especially regarding their addition to knowledge, which is vital for fostering idea creation and theoretical research beneficial to various stakeholders in the tourism and hospitality sector. Given the swift evolution of these technologies, grasping their utilization, advantages and obstacles is essential for industry experts, decision-makers and scholars.FindingsThis theme issue encapsulates the transformative impact of automation and AI in the tourism and hospitality industry, highlighting the juxtaposition of enhanced operational efficiency against employment security concerns. It emphasizes the need for ethical governance to address AI’s ethical dilemmas and security vulnerabilities. The exploration of consumer attitudes towards AI, particularly in service robots and facial emotion recognition technologies, is deemed crucial for leveraging AI’s full potential in service delivery. Furthermore, AI’s entry into gastronomy prompts a reevaluation of traditional culinary practices, advocating for a balance between technological innovation and the preservation of cultural heritage.Originality/valueThis theme issue stands out for its focused exploration of the transformative effects of automation and AI within the hospitality and tourism industry. Unlike broader discussions on technology in business, it delves into specific AI-driven innovations, operational changes and the resultant shifts in consumer behavior and industry standards. By curating a collection of empirical research, theoretical analyses and content analyses, this issue is providing important insights into how automation and AI are reshaping service delivery, management practices and strategic decision-making in an industry characterized by its intense customer focus and reliance on human-to-human interaction.

FER-BHARAT: a lightweight deep learning network for efficient unimodal facial emotion recognition in Indian context

Humans' ability to manage their emotions has a big impact on their ability to plan and make decisions. In order to better understand people and improve human–machine interaction, researchers in affective computing and artificial intelligence are investigating the detection and recognition of emotions. However, different cultures have distinct ways of expressing emotions, and the existing emotion recognition datasets and models may not effectively capture the nuances of the Indian population. To address this gap, this study proposes custom-built lightweight Convolutional Neural Network (CNN) models that are optimized for accuracy and computational efficiency. These models are trained and evaluated on two Indian emotion datasets: The Indian Spontaneous Expression Dataset (ISED) and the Indian Semi Acted Facial Expression Database (iSAFE). The proposed CNN model with manual feature extraction provides remarkable accuracy improvement of 11.14% for ISED and 4.72% for iSAFE datasets as compared to baseline, while reducing the training time. The proposed model also surpasses the accuracy produced by pre-trained ResNet-50 model by 0.27% ISED and by 0.24% for the iSAFE dataset with significant improvement in training time of approximately 320 s for ISED and 60 s for iSAFE dataset. The suggested lightweight CNN model with manual feature extraction offers the advantage of being computationally efficient and more accurate compared to pre-trained model making it a more practical and efficient solution for emotion recognition among Indians.

Superior facial emotion recognition in adolescents with anorexia nervosa - A replication study.

Anorexia nervosa (AN) has been associated with alterations in the processing of socio-emotional information, including impairments in the recognition of emotions in other people's faces. However, adolescents with AN might not show the impairments found in adult patients. The present study investigated facial emotion recognition in adolescents with AN, aiming to replicate our previous results of superior emotion recognition abilities in adolescents with AN compared to adolescents without mental disorders. Adolescent girls (12-18years) with AN (n=33) were compared to girls without mental disorders (n=41). Participants completed one task requiring identification of emotions (happy, sad, afraid, angry, neutral) in faces and one control task. As expected, adolescents with AN showed superior emotion recognition, with higher accuracy rates specifically for afraid faces. This is the first study replicating previous results on basic emotion recognition in adolescents with AN using (almost) the same methodology. The results suggest that, in contrast to adults, adolescents with AN do not show impairments in facial emotion recognition. The impairments may arise in the longer course of the illness, however, longitudinal studies are necessary to confirm this assumption.

Currency Classification Using Deep Learning

Among the uses of machine learning is the recognition of facial expressions. Based on the features that are derived from an image, it assigns a facial expression to one of the classes of facial expressions. Convolutional Neural Network (CNN) is a classification technique that may also be used to identify patterns in an image. We used the CNN approach to identify facial expressions in our proposed study. To increase the precision of facial emotion recognition, the wavelet transform is used after CNN processing. Seven distinct facial expressions are included in the facial expression image dataset that was obtained from Kaggle. The findings of the face expression recognition experiment utilizing CNN and wavelet transform show that the accuracy is improved, and the output is audible. Index Terms: Facial and Convolutional Neural Nets

Emotion Detector and Solution Provider Using Voice Model

Emotion detection from images is one of the most high-powered tasks in today’s world and AI detection plays a fundamental role, this project aims to understand human emotions which play a major role in well-being, this project aims to dive deep into human emotions and provide a variety of solutions, through Human-computer interaction takes place, emotion detection and voice signals are input signal The project is classified into two major components: one where a human emo- tion is detected through camera is detected and facial recognition takes place. Secondly, where a solution is provided to a recorded emotion using a voice model, it creates a conversation which result turn provides a solution based on the collected database. The paper presents a theory-based framework of technologies discusses challenges and behavioural considerations, and gives directions for future practices and use. Index Terms—Voice GPT, Artificial intelligence (AI), Facial emotion recognition (FER), Emotion Detection (ED), Open CV Library (OCL) Convolutional neural networks (CNN), Machine Learning (ML), Tensor flow, GTTP, Keras

Facial Emotion Recognition using Convolutional Neural Network

Emotion is an expression that human use in expressing their feelings. It can be express through facial expression, body language and voice tone. Humans' facial expression is a major way in conveying emotion since it is the most powerful, natural and universal signal to express humans' emotion condition. However, humans' facial expression has similar patterns, and it is very confusing in recognizing the expression using naked eye. For instance, afraid and surprised is very similar to one another. Thus, this will lead to confusion in determining the facial expression. Hence, this study aims to develop a application for emotion recognition that can recognize emotion based on facial expression in real-time. The Deep Learning based technique, Convolutional Neural Network (CNN) is implemented in this study. The Mobile Net algorithm is deployed to train the model for recognition. There are four types of facial expressions to be recognized which are happy, sad, surprise, and disgusting. Keywords: Facial Emotion Recognition, Deep Learning, CNN, Image Processing.

MULTIMODAL SYSTEM FOR FACIAL EMOTION RECOGNITION BASED ON DEEP LEARNING

Emotions are one of the main ways of communication between people and of expressing attitudes towards objects, products, services, etc. They are divided to verbal and non-verbal classes. Human speech and intonation belong to the first class, and to the second (non-verbal) facial and body emotions, known as body language. The subject of this report is the development of multimodal deep learning system intended to recognize facial and body emotions and their relationship with the scene (weather) in which they occur. It is based on three deep learning neural networks (DNN) each one for recognition of facial emotion, body emotion and weather. Combining their results, we improve significantly the final facial emotion recognition (FER) results.

The role of emotional factors in face processing abilities in autism spectrum conditions

Facial emotion recognition is considered atypical in individuals with autism spectrum conditions (ASC), but emotion recognition abilities vary widely in autistic people, and there are inconsistent findings on the causes of these differences. Research indicates alexithymia may result in facial emotion recognition differences in ASC. Alternatively, mood disorders have been linked to atypical facial emotional expression recognition abilities in neurotypical adults. Investigating both the effects of alexithymia and mood disorders (depression and anxiety) is necessary to establish which of these factors may cause atypical facial emotion recognition in ASC. This study aimed to examine whether alexithymia or mood disorder symptomology is a predictor of atypical facial emotion recognition in individuals with ASC. Ninety-eight non-autistic adults and 80 autistic adults were recruited. Participants completed an online facial processing task to examine emotion and identity recognition abilities, the AQ-28, the TAS-20, and the HADS to measure autism severity, alexithymia symptoms, and depression and anxiety symptoms. Regression-based analyses found that autistic traits and autistic group membership did not predict facial emotion processing abilities after accounting for demographic variables, alexithymia and mood disorders: however, neither alexithymia nor mood disorder symptoms predicted variance in face processing abilities either. Our results concur with previous meta-analyses of facial emotion processing in autism spectrum disorder which report that studies do not always report deficits in face processing in autism: our findings are also not supportive of the model that argues that alexithymia explains facial emotion processing difficulties in autism.

A Survey on Facial Emotion Recognition and Fake Emotion Detection Techniques

Facial expressions play a crucial role in depicting the thoughts and feelings of any human. Moreover, they might even serve as the distinguishing factor in categorizing whether a person is genuine or not. In this survey, the authors have carried out a systematic literature review of the past work done on facial emotion recognition and discrimination of genuine and fake emotions. Various datasets that are previously and currently being used in this field have been surveyed. The main objective of this paper is to study the state-of-the-art techniques that are being used to analyze and identify facial emotions and to distinguish between their genuineness. For this purpose, papers from distinguished publications have been researched and a comprehensive review of the literature has been fulfilled.

Emotion Symphony: Deep Learning Journey Through Human Facial Emotions

Abstract: In today's technologically advanced world, understanding human emotions holds paramount importance in various domains ranging from human-computer interaction to mental health diagnosis. Emotions are primarily conveyed through facial expressions, making facial emotion recognition a crucial area of research. This project, "Emotion Symphony," embarks on a deep learning journey aimed at developing an efficient system for recognizing and interpreting human facial emotions. The project employs state-of-the-art deep learning techniques to extract meaningful features from facial images and map them to corresponding emotional states. Leveraging convolutional neural networks (CNNs) and recurrent neural networks (RNNs), the system is designed to analyse subtle nuances in facial expressions, capturing both spatial and temporal dependencies for accurate emotion recognition. To train the model, extensive datasets comprising diverse facial expressions are utilized, ensuring robustness and generalization of the proposed system. Additionally, data augmentation techniques are employed to enhance the model's ability to handle variations in lighting conditions, poses, and facial expressions.

Disruptive mood dysregulation disorder: the relationship between cardiac vagal tone, emotion recognition, and expressive prosody

Children with disruptive mood dysregulation disorder have difficulties with emotion regulation and recognition which impacts their social functioning. Due to its recent definition, there is limited information on the neural mechanisms of this disorder. This exploratory study examined the role of autonomic heart regulation in children with disruptive mood dysregulation disorder, motivated by suggestions of reduced autonomic regulation (measured by heart rate variability) associated with psychiatric disorders in adults and children. The cross-sectional design sampled two groups of children, one with disruptive mood dysregulation disorder ( n = 15; 14 male) and the other with typical development ( n = 15; 14 male). Heart rate variability, facial emotion recognition accuracy and speed, and prosody were measured. There were no significant differences in heart rate variability between the two groups before, or during, the tasks. However, children with disruptive mood dysregulation disorder experienced significantly greater difficulty recognising fear in faces compared to controls and confused other negatively valenced emotions. There was also a significant, negative correlation between heart rate variability and prosody modulation (verbal expression of emotion) in the control group, a relationship that was absent in the disruptive mood dysregulation disorder group. These results suggest that atypical autonomic regulation during emotionally evocative situations and difficulty with facial emotion recognition may contribute to the challenges experienced by children with disruptive mood dysregulation disorder. This is valuable knowledge for psychologists, which can contribute to individualised, effective, and comprehensive care that addresses the child’s emotional, social, and relational well-being.

Electrocardiogram-based Emotion Recognition using Convolutional Neural Networks

Introduction: Electrocardiogram-based facial emotion identification has been a widely spread field in the last few decades. Due to its non-linearity, non-stationary and noisy properties, it is a very difficult job to create a framework that is capable of recognizing emotions with a high recognition rate. background: Electrocardiogram based facial emotion identification has been widely spread field in last some decades. Due to its non-linearity, non-stationary and noisy properties, it is very difficult job to create a framework that is capable of recognizing emotions with high recognition rate. Method: In this work, we introduce a new framework for facial emotion detection based on feature creation using a topographic representation of ECG signal properties. The feature map is created using deep learning techniques, and further, extricated features are then used for classification techniques to detect facial emotion recognition. Result: The recognition results are achieved on two publicly available facial expression datasets, i.e., Ascertain and Dreamer. We illustrated the usefulness of our framework by comparing results with other existing methods. Conclusion: The recognition results prove that the introduced framework can enhance the identifying rate on various given datasets.

Exploring the Impact of Convolutional Neural Networks on Facial Emotion Detection and Recognition

Emotional analytics is a fascinating blend of psychology and technology, with one of the primary methods for recognizing emotions involving facial expression analysis. Facial emotion detection has advanced significantly, utilizing deep learning algorithms to identify common emotions. In recent years, substantial progress has been made in automatic facial emotion recognition (FER). This technology has been applied across various industries to enhance interactions between humans and machines, particularly in human-centered computing and the emerging field of emotional artificial intelligence (EAI). Researchers focus on improving systems’ capabilities to recognize and interpret human facial expressions and behaviors in diverse contexts. The impact of convolutional neural networks (CNNs) on this field has been profound, as these networks have undergone significant development, leading to diverse architectures designed to address increasingly complex challenges. This article explores the latest advancements in automated emotion recognition using computational intelligence, emphasizing how contemporary deep learning models contribute to the field. It provides a review of recent developments in CNN architectures for FER over the past decade, demonstrating how deep learning-based methods and specialized databases collaborate to achieve highly accurate outcomes.

FACIAL EMOTION RECOGNITION FOR DUSKY SKINNED PEOPLE USING CNN

In computer vision, the Convolutional Neural Network is a very popular and useful model for emotion recognition. So, we are using the one of the CNN architectures to analyse the emotions called VGGNet. It is one of the methods or models of Transfer Learning Technology. It has more layers than its predecessors ImageNet and ResNet which have minimal layers when compared to VGGNet. It has 16 or 19 layers in its model to train the dataset. It can train a large dataset at a time. Our main goal is to accurately recognize the dusky skinned people's emotions. And so, for that we have used more sample images of dusky skinned people to be trained with the existing one. By doing so, we can improve the identification of emotions on dusky skinned people. And VGGNet technology also provide a great support to the improvement of the concept. Keywords: Convolutional Neural Network (CNN), VGGNet architecture, facial emotion recognition, dusky skin.

Implications of depressive mood in OSAHS patients: insights from event-related potential

BackgroundObstructive sleep apnea-hypopnea syndrome (OSAHS) is a chronic breathing disorder characterized by recurrent upper airway obstruction during sleep. Although previous studies have shown a link between OSAHS and depressive mood, the neurobiological mechanisms underlying mood disorders in OSAHS patients remain poorly understood. This study aims to investigate the emotion processing mechanism in OSAHS patients with depressive mood using event-related potentials (ERPs).MethodsSeventy-four OSAHS patients were divided into the depressive mood and non-depressive mood groups according to their Self-rating Depression Scale (SDS) scores. Patients underwent overnight polysomnography and completed various cognitive and emotional questionnaires. The patients were shown facial images displaying positive, neutral, and negative emotions and tasked to identify the emotion category, while their visual evoked potential was simultaneously recorded.ResultsThe two groups did not differ significantly in age, BMI, and years of education, but showed significant differences in their slow wave sleep ratio (P = 0.039), ESS (P = 0.006), MMSE (P < 0.001), and MOCA scores (P = 0.043). No significant difference was found in accuracy and response time on emotional face recognition between the two groups. N170 latency in the depressive group was significantly longer than the non-depressive group (P = 0.014 and 0.007) at the bilateral parieto-occipital lobe, while no significant difference in N170 amplitude was found. No significant difference in P300 amplitude or latency between the two groups. Furthermore, N170 amplitude at PO7 was positively correlated with the arousal index and negatively with MOCA scores (both P < 0.01).ConclusionOSAHS patients with depressive mood exhibit increased N170 latency and impaired facial emotion recognition ability. Special attention towards the depressive mood among OSAHS patients is warranted for its implications for patient care.

AI-Based Facial Emotion Recognition

Facial expression, age, and gender detection have received a lot of interest in recent years due to their wide range of applications in fields including healthcare, security, marketing, and entertainment. With the proliferation of artificial intelligence (AI) techniques, especially deep learning, facial analysis systems' accuracy and efficiency have improved significantly. This paper provides a comprehensive analysis of the most recent advances in AI-based facial expression, age, and gender identification algorithms. We explore the constraints and limitations of current techniques, such as the necessity for big annotated datasets, biases in training data, and interpretability issues in deep learning models. This study seeks to give academics, practitioners, and policymakers, a thorough grasp of the most recent state-of-the-art methodologies and trends in AI-based facial emotion, age, and gender identification, thereby supporting future advancements and responsible usage of these technologies. Keywords: Facial expression, age, gender, identification, deep learning, artificial intelligence

A multi-method assessment of emotional processes predicting longitudinal anxiety symptom trajectories in an adolescent clinical sample

ObjectiveEmotion recognition, reactivity, and regulation are important in the development and maintenance of anxiety disorders. Whether and how these processes differentiate between different trajectories in anxiety remain unclear. The current study examined emotional processes as prospective predictors of anxiety symptom trajectories in psychiatrically hospitalized youth. MethodParticipants were 180 adolescents (Mage = 14.89; SD = 1.35) from a psychiatric inpatient unit. At index hospitalization, participants completed a behavioral task assessing facial emotion recognition, and self-report measures of emotion dysregulation and reactivity. They completed a self-report measure on anxiety symptoms at baseline and 3, 6, 12, and 18 months post-discharge. Latent growth curve analysis was conducted to identify subgroups of individuals based upon their trajectory of anxiety symptoms across 18-months. ANOVAs were used to examine subgroup differences in emotional processing variables. ResultsThree distinct trajectories were identified, a stable moderate-to-high anxiety group, a group with moderate-to-high anxiety at baseline with symptom improvement over time, and a group characterized by relatively stable low-to-moderate anxiety throughout the study. The two initially moderate-to-high anxiety groups scored higher for emotion dysregulation and emotion reactivity at baseline compared to the low-to-moderate anxiety group. Emotion regulation difficulties relating to emotional non-acceptance were higher for the stable moderate-to-high anxiety group than for the moderate-to-high anxiety group that experience symptom improvement over time. ConclusionsThese findings may have clinical implications for discharge planning. Future studies should explore emotion regulation with a focus on non-acceptance of one’s emotional experiences as a potential target of intervention in individuals with elevated anxiety.

Learning from feature and label spaces’ bias for uncertainty-adaptive facial emotion recognition

Developing an accurate deep model for facial emotion recognition is a long-term challenge. It is because the uncertainty of emotions, stemming from the ambiguity of different emotional categories and the difference of subjective annotations, can ruin the ability of model to achieve the desired optimization. This paper constructs two distinct datasets, namely original sample set and ambiguous sample set, to explore an effective ambiguous knowledge transfer method to realize the adaptive awareness of uncertainty in facial emotion recognition. The original sample set is the weakly-augmented data with relatively low uncertainty, as most emotions are clean in reality. Meanwhile, the ambiguous sample set is strongly-augmented data that introduces feature and label bias with regard to emotion, which are with relatively high uncertainty. The proposed framework consists of two sub-nets, which are trained using the original set and the ambiguous set respectively. To achieve uncertainty-adaptive learning for two sub-nets, we introduce two modules. One is the cross-space attention consistency learning module that performs attention coupling across original and ambiguous feature spaces, achieving uncertainty-aware representation learning in feature granularity. The other is the soft-label learning module that models and utilizes uncertainty in label granularity, through aligning the posterior distributions between original label space and ambiguous label space. Experimental studies on public datasets indicate that our method is competitive with the state-of-the-art.