Facial Emotion Recognition Research Articles

Disorder Facial Emotion Recognition with GreyWolf Optimized Gefficient-Net for AutismSpectrum

Disorder Facial Emotion Recognition with GreyWolf Optimized Gefficient-Net for AutismSpectrum

Are you tuned in to others' mind? A cross-modal evaluation of affective theory of mind in people with Parkinson's disease.

Affective Theory of Mind (ToM) is the ability to understand other peoples' emotional states and feelings. Several studies showed impaired affective ToM abilities in people with Parkinson's disease (PD). However, most studies tested this ability by using single-stimulus modality tasks (visual cues). The aim of the present study was to evaluate whether affective ToM abilities are impaired in PD using multiple stimulus modalities and whether they are related to disease duration and cognitive/emotional abilities. Twenty mid-stage, non-demented people with PD and 20 matched controls were evaluated by means of two tasks requiring subjects to infer others' mental states from the eyes' expression - the Reading the Mind in the Eyes (RME) test, and from the emotional prosody - a modified version of the Reading the Mind in the Voice (RMV) test. In people with PD, cognitive function and emotional processes were assessed through cognitive tasks and batteries of facial and prosodic emotion recognition tests. Compared to controls, the PD group showed significantly impaired performances in both RME and RMV tests (p<0.0001). Affective ToM abilities in people with PD were not correlated with cognitive function or disease duration. Visual recognition of anger and auditory recognition of fear were positively correlated with RME (p=0.0028) and RMV (p<0.0001) test scores, respectively. Cross-modal impairment of affective ToM abilities is a non-motor feature of PD, unrelated to disease stage or cognitive status. A dysfunctional amygdala-centred network might represent the shared bases for impairments in fear and anger recognition and affective ToM abilities in PD.

PNasFH-Net: Pyramid neural architecture search forward network for facial emotion recognition in uncontrolled and pose variant environment

PNasFH-Net: Pyramid neural architecture search forward network for facial emotion recognition in uncontrolled and pose variant environment

Facial Expression and Speech Pattern Computing using Modern Techniques of Computer Vision

Two crucial areas of research in the field of emotional computing are recognition of facial emotions and spoken emotion. In this study, we tackle the challenge of precisely identifying and analyzing emotions from speech and facial expressions. A fascinating problem in human-computer interaction, mental health monitoring, social robots, and other fields is the detection of emotions from facial expressions and verbal inputs. By combining different modalities, we can obtain a more complete picture of emotions by utilizing the complimentary information included in facial expressions and speech patterns. Our approach succeeds in a number of crucial goals. In order to accurately extract facial features and analyze facial expressions, it first applies cutting-edge computer vision techniques. Modern speech processing algorithms are also used to extract emotional indicators from speech data and record pertinent acoustic elements. Our solution has two results in total. First of all, it permits in-the-moment emotion recognition, enabling prompt and context-sensitive reactions in programs like virtual assistants or emotion-aware systems. Second, it offers insightful information about human emotions that can help with psychological research, clinical diagnosis, and therapeutic interventions. For the extraction of facial features, the suggested method makes use of the Haar cascade classifier, a well-liked object detection algorithm. Numerous facial characteristics, including the areas around the eyes, the nose, and the lips, are represented by haar-like features. The classifier is trained using a training dataset made up of facial photos that have been labeled with the associated emotions. Preprocessing is done on the labeled dataset to separate out the Haar-like features and produce the positive and negative samples. The trained classifier can then recognize and categorize emotions from facial photos or video streams in real-time. The findings point to the potential for real-time emotion recognition in a variety of real-world settings, enabling more sympathetic and context-sensitive human-machine interfaces.

Deficits in facial emotion recognition and cognitive function among baby boomers.

Facial emotion recognition (FER), a key component of social cognition, plays a critical role in social interactions. In the aging process, FER among older adults holds significant potential as a tool for diagnosing cognitive function or enhancing interpersonal relationships. However, research in this area remains limited. This study aims to address this gap by examining the impact of cognitive function on FER among Korean baby boomers aged 60 to 69. Eighty-one participants completed the Korean version of the Montreal Cognitive Assessment (K-MOCA) and FER tasks. Of the participants, 69% had normal cognition, while 31% had mild cognitive impairment. Participants with normal cognition were 1.59 times more likely to recognize facial expressions correctly than those with impaired cognition (AOR=1.59, p<0.0001). They showed significantly higher odds of recognizing happy (AOR=9.68), anger (AOR=2.25), disgust (AOR=1.95), neutral (AOR=3.02), and surprise (AOR=2.27) expressions (p<0.001). Participants with normal cognition participants also scored higher on overall intensity for every emotion except sadness (p<0.001). Among the seven domains of the MoCA, three sub-domains-visuospatial/executive function, language, and orientation-showed significant associations with overall FER. These results highlight the diagnostic potential of FER tasks in identifying cognitive disorders and enhancing social skills in clinical and practical settings.

Application of Facial Emotion Recognition in Teaching-Learning Process for Quality Assessment and Enhancement

Application of Facial Emotion Recognition in Teaching-Learning Process for Quality Assessment and Enhancement

Contribution of neuroimaging to the understanding of social cognition in epilepsy.

This narrative review aims to identify and summarize existing research to better understand the pathophysiological and neuroanatomical bases of social cognition deficits in people with epilepsy. The neuroanatomical basis of social cognition was primarily examined in healthy subjects. In healthy individuals, social cognition is supported by a complex network of interconnected brain regions. Facial emotion recognition relies on a distributed set of structures, including the occipitotemporal neocortex, the temporoparietal and prefrontal areas, and the putamen with a pivotal role of the amygdala. Theory of mind primarily involves the dorsal medial prefrontal cortex and temporoparietal junction, while empathy engages the anterior insular and cingulate cortices. In people with epilepsy, most functional neuroimaging studies have focused on facial emotion recognition, primarily in patients with temporal lobe epilepsy. Nevertheless, across various domains of social cognition, abnormal activations and disrupted connectivity within social cognition networks are consistently observed, regardless of the focus location. Aberrant connectivity has also been noted in the few studies involving patients with generalized epilepsy. In focal epilepsy, the amygdala remains a central region for facial emotion recognition, irrespective of whether the epilepsy is localized to the temporal or frontal lobes. For theory of mind studies, regions typically identified in healthy individuals, such as the medial prefrontal cortex, exhibit either hyperactivation or reduced activation in people with focal epilepsy, complicating interpretation. In the domain of empathy, a study involving patients with idiopathic generalized epilepsy reported decreased activation in core regions commonly identified in healthy individuals, particularly the anterior cingulate cortex and anterior insula. The limited data available in the literature suggest that key regions shared between social cognition and epilepsy networks consistently contribute to these disruptions and may serve as potential targets for future neuromodulation interventions.

Enhancing highway safety with LSTM-based facial emotion recognition

Enhancing highway safety with LSTM-based facial emotion recognition

Face and Voice Recognition-Based Emotion Analysis System (EAS) to Minimize Heterogeneity in the Metaverse

The metaverse, where users interact through avatars, is evolving to closely mirror the real world, requiring realistic object responses based on users’ emotions. While technologies like eye-tracking and hand-tracking transfer physical movements into virtual spaces, accurate emotion detection remains challenging. This study proposes the “Face and Voice Recognition-based Emotion Analysis System (EAS)” to bridge this gap, assessing emotions through both voice and facial expressions. EAS utilizes a microphone and camera to gauge emotional states, combining these inputs for a comprehensive analysis. It comprises three neural networks: the Facial Emotion Analysis Model (FEAM), which classifies emotions using facial landmarks; the Voice Sentiment Analysis Model (VSAM), which detects vocal emotions even in noisy environments using MCycleGAN; and the Metaverse Emotion Recognition Model (MERM), which integrates FEAM and VSAM outputs to infer overall emotional states. EAS’s three primary modules—Facial Emotion Recognition, Voice Emotion Recognition, and User Emotion Analysis—analyze facial features and vocal tones to detect emotions, providing a holistic emotional assessment for realistic interactions in the metaverse. The system’s performance is validated through dataset testing, and future directions are suggested based on simulation outcomes.

Feature and Decision Levels Fusion for the Synergistic Analysis of Facial Expressions and EEG Signals in the Context of Discrete Emotion Recognition

The undertaking of brain-computer interface emotion recognition represents a challenging task that demands meticulous thinking in order for machines to discern human emotions and respond appropriately. This work aims to improve the efficacy of individual modalities by incorporating multimodality for emotion recognition, which employs two distinct modalities. Electroencephalogram (EEG) data and facial expressions are used as independent modalities for emotion identification, with each modality evaluated separately. To combine the modalities, decision- and feature-level fusion algorithms are used. While EEG-derived emotions are primarily classified as continuous domains of valence and arousal, facial emotions are primarily classified as discrete emotions, which presents a considerable hindrance to the fusion process. Ashford Bird[10]. present the dataset used for EEG experiments at UKCI-2019.the CFEE dataset serves as the foundation for face emotion recognition for the proposed work[9]. EEG signals are analyzed for statistical properties such as mean, standard deviation, skewness, and kurtosis, while the face emotion dataset is used to identify various action units. Given the large number of features, redundant feature removal approaches are used to determine feature efficacy. Using the product rule, decision-level fusion obtains an accuracy of 80%. Accuracy in feature-level fusion is 94.48% for KNN and 98.66% for SVM classification which are comparatively higher than individual average accuracy of facial expression 88.04% and 90.73% for EEG signal.

Exploring the role of epileptic focus lateralization on facial emotion recognition in the spectrum of mesial temporal lobe epilepsy.

Evidence increasingly shows that facial emotion recognition (FER) is impaired in refractory mesial temporal lobe epilepsy (rMTLE), especially in patients with a right focus. This study explores FER in both mild (mMTLE) and refractory forms, examining the influence of epileptic focus lateralization on FER. 50 MTLE patients, categorized by epilepsy severity and focus lateralization, were compared with healthy controls. FER was assessed using the Ekman Faces Test (EFT), which evaluates recognition of six basic emotions, alongside a battery of cognitive and mood tests. mMTLE patients showed selective deficits in recognizing fear and anger, while rMTLE patients displayed broader deficits, affecting all emotions except surprise. Patients with a right focus underperformed across all negative emotions, whereas those with a left focus showed deficits mainly in fear and anger. Analysis indicated that early epilepsy onset was associated with poorer FER in right-focused patients; febrile seizures and mesial temporal sclerosis significantly impacted FER in left-focused patients. MTLE affects circuits of FER even in mild subjects, although to a lesser extent than in refractory ones. Earlier onset of MTLE could disrupt the development of FER, possibly interfering during a critical phase of maturation of its circuits, when the focus is right. Conversely, left MTLE may cause less damage to FER circuits, requiring additional factors such as a history of febrile seizures and/or mesial temporal sclerosis for significant impact. Clinically, refractory and right-sided MTLE might be viewed as risk factors of FER deficits.

Emotions at the heart of learning: exploring the role of teacher emotions in student engagement using facial emotion recognition

Emotions at the heart of learning: exploring the role of teacher emotions in student engagement using facial emotion recognition

EmoSense: Pioneering Facial Emotion Recognition with Precision Through Model Optimization and Face Emotion Constraints

EmoSense: Pioneering Facial Emotion Recognition with Precision Through Model Optimization and Face Emotion Constraints

Scalable Context-Based Facial Emotion Recognition Using Facial Landmarks and Attention Mechanism

Scalable Context-Based Facial Emotion Recognition Using Facial Landmarks and Attention Mechanism

Facial emotion recognition based on upper features and transfer learning

Facial expression recognition (FER) in the upper face focuses on the analysis and recognition of emotions based on features extracted from the upper region of the face. This region typically affects the eyes, eyebrows, forehead, and sometimes the upper cheeks. Since these areas are often less affected by face masks or other facial coverings, FER algorithms can concentrate on capturing and interpreting the relevant facial cues, such as eye movements, eyebrow positions, and forehead wrinkles, to accurately recognize and classify different emotions. By focusing on the upper face, FER systems can mitigate the impact of occlusions caused by masks and still provide meaningful insights into the emotional states of individuals. In this work, a FER approach focusing on the upper region is proposed. Several experiments have been made using the CK+ dataset in addition to a comparison between the emotion recognition scores using the upper and the entire face in order to determine whether this area can reflect the real expressed emotion. The results of our approach are promising compared to previous studies with an accuracy up to 96%.

Facial expression recognition with modified local ternary pattern images using convolutional neural network and extreme learning machine

Facial emotion recognition is a crucial aspect of human-computer interaction systems, education, health and various other fields. The common challenges faced in FER are variation in light. Based on researches, Local ternary Pattern (LTP) as a feature vector that can handle the variation of light. Modified Local Ternary Pattern (MLTP) is more discriminative to variation of light and less sensitive to noise. MLTP is a conventional feature vector that requires manual processing. In contrast to MLTP, the Convolution Neural Network (CNN) architecture has an automated feature extractor. This paper proposes MLTP as input to the CNN architecture for handling the variation of light and for automatic feature extractor. Then, for classifying Extreme Learning Machine is used to reduce the training time of the CNN classifier. The Karolinska Directed Emotional Faces (KDEF) dataset is used to assess the proposed method and got an accuracy of 86.28%.

A novel deep learning approach for facial emotion recognition: application to detecting emotional responses in elderly individuals with Alzheimer’s disease

A novel deep learning approach for facial emotion recognition: application to detecting emotional responses in elderly individuals with Alzheimer’s disease

Transfer Learning Based Facial Emotion Recognition

Transfer Learning Based Facial Emotion Recognition

The differential outcomes procedure for improving the recognition of dynamic facial expressions of emotion in people with autism spectrum disorders and first-degree relatives.

Previous research highlights impairments in the recognition of facial expression of emotion in individuals diagnosed with Autism Spectrum Disorder (ASD). Relatives of people with ASD may exhibit similar, albeit subtler, impairments, referred to as the Broad Autism Phenotype (BAP). Recently, the Differential outcomes procedure (DOP) has been shown to enhance this ability in young adults using dynamic stimuli, with fewer intensity levels required to identify fear and surprise. The present study aimed to extend these findings to adults diagnosed with ASD (ASD group), and relatives of people diagnosed with ASD (BAP group). A Bayesian Generalized Linear Model was employed for statistical inference. The results indicated that the ASD DOP group performed worse than the BAP DOP group in fear trials. The social dimension of autism negatively impacted performance in some conditions, while positive relationships were found between the repetitive behavior dimension and performance for the ASD group. The opposite pattern was observed in the BAP group. These results suggest the importance of considering different dimensions of autism when conducting research on its relationship with other variables. Finally, participants in both ASD and BAP groups required less intensity to identify certain emotions when the DOP was applied, highlighting its potential utility for improving dynamic facial emotion recognition.

Automated compound facial emotion recognition using hybrid deep learning model and DCHBO

Automated compound facial emotion recognition using hybrid deep learning model and DCHBO