Facial Emotion Recognition Research Articles

A Comparison of CNN-based 2D Static Facial Emotion Recognition Techniques

A Comparison of CNN-based 2D Static Facial Emotion Recognition Techniques

Facial emotion recognition in sexual offenders

Facial emotion recognition in sexual offenders

Recognition of Facial Expression with the Help of IoT, AI and Robotics

The emerging field of "Smart Face Recognition" utilizes IoT and machine learning to accurately identify individuals based on their facial characteristics. Various industries such as security, retail, and healthcare are leveraging this technology to enhance customer satisfaction and increase productivity. By combining IoT and machine learning, large amounts of data can be collected from multiple sources, such as cameras and sensors, and used to train algorithms for real-time, precise identification of individuals. This technology is gaining popularity due to its accuracy, speed, and scalability, making it essential for applications like security and access control. Recognizing human facial emotions is a key focus in today's technological landscape, with robotic applications across various sectors highlighting the importance of emotion recognition for effective human-robot interaction. This project aims to develop and implement a new automated system for emotion detection and facial recognition using Artificial Intelligence (AI) and the Internet of Things (IoT).

Advanced Techniques for Real-time Facial Expression Recognition Using Deep Learning

Abstract—Developing systems that can automatically recognize and interpret human emotions from facial expressions is the aim of the quickly expanding field of facial emotion identification and detection research. This technology finds applications in a wide range of areas, including as healthcare, marketing, security, and human-computer interface. Using computer vision and machine learning algorithms, facial emotion recognition systems analyze a face’s features and classify it into numerous emotional categories, including joyful, sorrowful, angry, fearful, and surprised.The three steps in the multi-step process that goes into identifying facial emotions are face detection, facial feature extraction, and emotion categorization. Thanks to recent advances in deep learn- ing, facial emotion detection systems can now identify emotions with high resilience and precision. Further, the development of real-time face expression recognition systems has opened up new avenues for applications such as sentiment analysis, emotional intelligence, and affective computing. This technology could fundamentally alter human-machine interactions and open the way to more compassionate and personalized relationships.A multitude of applications, including virtual assistants, mental health aids, and human-centered technology, will be greatly impacted by the development of systems for identifying and de- tecting facial expressions. Artificial intelligence (AI) technologies that recognize emotions allow people to interact with the digital world in more intelligent and flexible ways. But the complexity of emotion identification lies in the fact that it requires context and geometric elements in addition to facial expressions.

Deficits of Facial Emotion Recognition in Elderly Individuals with Mild Cognitive Impairment.

The study of facial emotion recognition is under-explored in subjects with mild cognitive impairment (MCI). We investigated whether deficits in facial emotion recognition are present in patients with MCI. We also analyzed the relationship between facial emotion recognition and different domains of cognitive function. This study included 300 participants aged 60 years or older with cognitive decline. We evaluated 181 MCI and 119 non-MCI subjects using the Seoul Neuropsychological Screening Battery-Core (SNSB-C) and facial emotion recognition task using six facial expressions (anger, disgust, fear, happiness, sadness and surprise). A Generalized Linear Model (GLM) was used to assess the association between cognitive performance and accuracy of facial emotion recognition and to compare facial emotion recognition in the MCI group based on the impairment of five different domains of cognitive function. The model was adjusted for age, sex, years of education, and depressive symptoms. Patients with MCI had a lower score for accurately recognizing total facial emotion (0.48 vs. 0.53; ρ = 0.0003) and surprise (0.73 vs. 0.81; ρ = 0.0215) when compared to cognitively healthy subjects. We also discovered that frontal/executive function domain (Digit Symbol Coding [DSC, 0.38 vs. 0.49; p < 0.0001], Controlled Oral Word Association Test [COWAT, 0.42 vs. 0.49; p = 0.0001], Korean-Trail Making Test [K-TMT, 0.37 vs. 0.48; p = 0.0073], Korean-Color Word Stroop Test [K-CWST, 0.43 vs. 0.49; p = 0.0219]) and language domain (Korean-Boston Naming Test [S-K-BNT, 0.46 vs. 0.47; p = 0.003]) were statistically associated with the deficits of facial emotion recognition in patients with MCI. We observed a significant association between deficits in facial emotion recognition and cognitive impairment in elderly individuals.

HTNet for micro-expression recognition

Facial expression is related to facial muscle contractions and different muscle movements correspond to different emotional states. For micro-expression recognition, the muscle movements are usually subtle, which has a negative impact on the performance of current facial emotion recognition algorithms. Most existing methods use self-attention mechanisms to capture relationships between tokens in a sequence, but they do not take into account the inherent spatial relationships between facial landmarks. This can result in sub-optimal performance on micro-expression recognition tasks. Therefore, learning to recognize facial muscle movements is a key challenge in the area of micro-expression recognition. In this paper, we propose a Hierarchical Transformer Network (HTNet) to identify critical areas of facial muscle movement. HTNet includes two major components: a transformer layer that leverages the local temporal features and an aggregation layer that extracts local and global semantical facial features. Specifically, HTNet divides the face into four different facial areas: left lip area, left eye area, right eye area and right lip area. The transformer layer is used to focus on representing local minor muscle movement with local self-attention in each area. The aggregation layer is used to learn the interactions between eye areas and lip areas. The experiments on four publicly available micro-expression datasets show that the proposed approach outperforms previous methods by a large margin. The codes and models are available at: https://github.com/wangzhifengharrison/HTNet.

Research and Implementation of a Facial Emotion Recognition System Based on ZYNQ

Research and Implementation of a Facial Emotion Recognition System Based on ZYNQ

FORMICARY SWARM OPTIMIZED DEEP CNN FOR FACIAL EMOTION RECOGNITION FROM HUMAN FACIAL EXPRESSIONS

Facial emotion recognition (FER) is a dominant research area that captures the biological facial features and matches the facial data with existing databases to analyze the individual’s emotional state. Numerous techniques have been formulated for attaining effective FER. However, the occlusions, different head positions, deformed faces, motion blur under unrestricted settings, and complicated backgrounds make it complex to analyze the facial images. In this paper, the formicary swarm optimization-based deep convolutional neural network (FSO-opt DCNN) model is utilized for facial emotion detection for which JAFFE and the RAVDESS facial expression datasets are used. DCNNs are proficient with built-in feature extraction strategies from images to map the various facial expressions to the corresponding emotional states adopted for effective FER. In addition, the intensity, directional, and edge patterns as well as the correlation features extracted utilizing the hybrid textual pattern, RESNET 101 and VGG 16-based correlation modules assist the DCNN to attain the informative feature from the high-resolution images. Further, the formicary swarm optimization (FSO) is incorporated that effectively tunes the DCNN to capture the complex relationships between the learned features that excel the FER capability. Evaluating the metrics, the face recognition model using the RAVDESS dataset achieves notable efficiencies during a training percentage (TP) of 90%, with values of 97.51%, 95.48%, 99.55%, 95.48%, 97.48%, 96.47%, and a minimum loss of 2.49%. Simultaneously, the emotion detection model demonstrates robust efficiencies with metric values of 96.75%, 98.49%, 95.01%, 98.49%, 96.72%, 97.59%, and a minimum loss of 3.25%. Finally, the obtained results reveal the efficacy of the FSO-opt DCNN, particularly in face recognition tasks, as it outperforms the existing models across various datasets, showcasing its versatility and potential in facial analysis applications.

Down-modulation of functional ventral striatum activation for emotional face stimuli in patients with insula damage.

Insula damage results in substantial impairments in facial emotion recognition. In particular, left hemispheric damage appears to be associated with poorer recognition of aversively rated facial expressions. Functional imaging can provide information on differences in the processing of these stimuli in patients with insula lesions when compared to healthy matched controls (HCs). We therefore investigated 17 patients with insula lesions in the chronic stage following stroke and 13 HCs using a passive-viewing task with pictures of facial expressions testing the blood oxygenation dependent (BOLD) effect in predefined regions of interest (ROIs). We expected a decrease in functional activation in an area modulating emotional response (left ventral striatum) but not in the facial recognition areas in the left inferior fusiform gyrus. Quantification of BOLD-response in ROIs but also voxel-based statistics confirmed this hypothesis. The voxel-based analysis demonstrated that the decrease in BOLD in the left ventral striatum was driven by left hemispheric damaged patients (n = 10). In our patient group, insula activation was strongly associated with the intensity rating of facial expressions. In conclusion, the combination of performance testing and functional imaging in patients following circumscribed brain damage is a challenging method for understanding emotion processing in the human brain.

Impact of psychosocial stress on facial emotion recognition in schizophrenia and controls: an experimental study in a forensic sample.

Psychotic disorders have been associated with dysregulated stress reactions and adaptation. Little is known about the neuroendocrine responses to psychosocial stress in justice-involved individuals with schizophrenia. Using an experimental research design, the present study aims to examine differences in the subjective and neuroendocrine responses to psychosocial stress and its impact on facial emotion recognition (FER) and performance on an arithmetic task in chronically ill justice-involved individuals with schizophrenia (PAT) and a healthy control group. PAT undergoing treatment in forensic psychiatric inpatient wards (n = 17) and a healthy control group (n = 17) were assessed regarding sociodemographic and clinical characteristics. Additionally, salivary cortisol levels, measured before and after performing a psychosocial stress task [Montreal Imaging Stress Task (MIST)], and performance on an arithmetic problem-solving task and two FER tasks were recorded. Two participants dropped out, one from each group. Therefore, the final sample consisted of 32 individuals. Significant group differences in FER were recorded. There was a significant rise in subjective perception of momentary strain relating to the induction of psychosocial stress in both groups. Notably, the pre-stress level of subjective strain was higher in the PAT group than controls. Acute psychosocial stress induced an increase in FER performance in a sub-task related to naming emotions in individuals with schizophrenia spectrum disorder. The results underline the importance of psychosocial and therapeutic interventions aimed at strengthening stress resilience in individuals with schizophrenia spectrum disorders.

Medical image intelligent diagnosis system based on facial emotion recognition and convolutional neural network

The facial emotional information of patients is one of the important indicators of their health status. By combining facial emotion recognition technology, medical imaging intelligent diagnostic systems can more comprehensively understand the psychological and physiological status of patients. This study first selected medical imaging datasets from the publicly available Medical Machine Learning Image Standard database and performed necessary preprocessing, including data cleaning, enhancement, and standardization. Next, a CNN model is constructed to extract key features from medical images and facial expression images through structures such as convolutional layers, activation functions, pooling layers, and fully connected layers. In addition, the system also integrates facial emotion recognition module and medical image analysis module, improving the comprehensiveness of diagnosis through feature fusion. Finally, the diagnostic results are displayed through a user interaction interface. Through comparative experiments, this study verified that the proposed CNN model outperforms traditional LSTM and SVM models in terms of accuracy, recall, and precision. The CNN model has shown significant advantages in medical imaging intelligent diagnosis systems, especially achieving a recall rate of 99%, demonstrating extremely high disease recognition ability. The results of this study not only provide new research directions for the field of intelligent diagnosis in medical imaging, but also lay a solid foundation for future clinical applications.

Medical image intelligent diagnosis system based on facial emotion recognition and convolutional neural network

The facial emotional information of patients is one of the important indicators of their health status. By combining facial emotion recognition technology, medical imaging intelligent diagnostic systems can more comprehensively understand the psychological and physiological status of patients. This study first selected medical imaging datasets from the publicly available Medical Machine Learning Image Standard database and performed necessary preprocessing, including data cleaning, enhancement, and standardization. Next, a CNN model is constructed to extract key features from medical images and facial expression images through structures such as convolutional layers, activation functions, pooling layers, and fully connected layers. In addition, the system also integrates facial emotion recognition module and medical image analysis module, improving the comprehensiveness of diagnosis through feature fusion. Finally, the diagnostic results are displayed through a user interaction interface. Through comparative experiments, this study verified that the proposed CNN model outperforms traditional LSTM and SVM models in terms of accuracy, recall, and precision. The CNN model has shown significant advantages in medical imaging intelligent diagnosis systems, especially achieving a recall rate of 99%, demonstrating extremely high disease recognition ability. The results of this study not only provide new research directions for the field of intelligent diagnosis in medical imaging, but also lay a solid foundation for future clinical applications.

Examining learners’ cognitive processes and learning emotions in digital game-based vocabulary learning: evidence from eye-tracking technology and facial emotion recognition

Purpose This research explored the intersection of cognitive processes, emotions and their impacts on digital game-based vocabulary learning (DGVL) among university students. Recognizing the scant research in this area, especially with integrating innovative technologies, this study aims to understand the influence of these elements using advanced monitoring tools. Design/methodology/approach This inquiry was carried out as an observational study involving 44 university students segmented into three English language proficiency levels: high, intermediate and low based on their English course scores. The methodological tools included a portable eye tracker to observe visual behaviors and deep learning technology to identify and analyze the participants’ emotional responses and engagement with the DGVL during the learning process. Findings The results showed that distinct fixation sequences and variations in visual attention during DGVL were correlated with different levels of competency, suggesting a direct correlation between visual engagement and language competence. In addition, emotional transitions, predominantly from engagement (“flow”) to challenge (“frustration”), were common among participants, reflecting the emotional dynamics of learning. Furthermore, all participants consistently focused on the English vocabulary definitions, indicative of their targeted approach to understanding and test preparation. These findings highlighted the intricate dynamics between emotions and cognitive processes in learning environments. Originality/value Contribution of this study shows the interplay of cognitive engagement and emotional experiences in the context of DGVL. It underscored the complex nature of these factors and their collective influence on learners’ visual and emotional engagement, offering valuable implications for educational strategies and technological applications in language learning.

Confrontation with others' emotions changes bodily resonance differently in those with low and high levels of depersonalization.

We present novel research on the cortical dynamics of atypical perceptual and emotional processing in people with symptoms of depersonalization-derealization disorder (DP-DR). We used electroencephalography (EEG)/event-related potentials (ERPs) to delineate the early perceptual mechanisms underlying emotional face recognition and mirror touch in adults with low and high levels of DP-DR symptoms (low-DP and high-DP groups). Face-sensitive visual N170 showed markedly less differentiation for emotional versus neutral face-voice stimuli in the high- than in the low-DP group. This effect was related to self-reported bodily symptoms like disembodiment. Emotional face-voice primes altered mirror touch at somatosensory cortical components P45 and P100 differently in the two groups. In the high-DP group, mirror touch occurred only when seeing touch after being confronted with angry face-voice primes. Mirror touch in the low-DP group, however, was unaffected by preceding emotions. Modulation of mirror touch following angry others was related to symptoms of self-other confusion. Results suggest that others' negative emotions affect somatosensory processes in those with an altered sense of bodily self. Our findings are in line with the idea that disconnecting from one's body and self (core symptom of DP-DR) may be a defence mechanism to protect from the threat of negative feelings, which may be exacerbated through self-other confusion. This article is part of the theme issue 'Sensing and feeling: an integrative approach to sensory processing and emotional experience'.

Psilocybin for the treatment of Alzheimer's disease.

Alzheimer's disease (AD) stands as a formidable neurodegenerative ailment and a prominent contributor to dementia. The scarcity of available therapies for AD accentuates the exigency for innovative treatment modalities. Psilocybin, a psychoactive alkaloid intrinsic to hallucinogenic mushrooms, has garnered attention within the neuropsychiatric realm due to its established safety and efficacy in treating depression. Nonetheless, its potential as a therapeutic avenue for AD remains largely uncharted. This comprehensive review endeavors to encapsulate the pharmacological effects of psilocybin while elucidating the existing evidence concerning its potential mechanisms contributing to a positive impact on AD. Specifically, the active metabolite of psilocybin, psilocin, elicits its effects through the modulation of the 5-hydroxytryptamine 2A receptor (5-HT2A receptor). This modulation causes heightened neural plasticity, diminished inflammation, and improvements in cognitive functions such as creativity, cognitive flexibility, and emotional facial recognition. Noteworthy is psilocybin's promising role in mitigating anxiety and depression symptoms in AD patients. Acknowledging the attendant adverse reactions, we proffer strategies aimed at tempering or mitigating its hallucinogenic effects. Moreover, we broach the ethical and legal dimensions inherent in psilocybin's exploration for AD treatment. By traversing these avenues, We propose therapeutic potential of psilocybin in the nuanced management of Alzheimer's disease.

An efficient facial emotion recognition using convolutional neural network with local sorting binary pattern and whale optimization algorithm

An efficient facial emotion recognition using convolutional neural network with local sorting binary pattern and whale optimization algorithm

Expression perceptive fields explain individual differences in the recognition of facial emotions

Humans can use the facial expressions of another to infer their emotional state, although it remains unknown how this process occurs. Here we suppose the presence of perceptive fields within expression space, analogous to feature-tuned receptive-fields of early visual cortex. We developed genetic algorithms to explore a multidimensional space of possible expressions and identify those that individuals associated with different emotions. We next defined perceptive fields as probabilistic maps within expression space, and found that they could predict the emotions that individuals infer from expressions presented in a separate task. We found profound individual variability in their size, location, and specificity, and that individuals with more similar perceptive fields had similar interpretations of the emotion communicated by an expression, providing possible channels for social communication. Modelling perceptive fields therefore provides a predictive framework in which to understand how individuals infer emotions from facial expressions.

Functional connectivity during facial and vocal emotion recognition: Preliminary evidence for dissociations in developmental change by nonverbal modality

The developmental trajectory of emotion recognition (ER) skills is thought to vary by nonverbal modality, with vocal ER becoming mature later than facial ER. To investigate potential neural mechanisms contributing to this dissociation at a behavioural level, the current study examined whether youth's neural functional connectivity during vocal and facial ER tasks showed differential developmental change across time. Youth ages 8–19 (n = 41) completed facial and vocal ER tasks while undergoing functional magnetic resonance imaging, at two timepoints (1 year apart; n = 36 for behavioural data, n = 28 for neural data). Partial least squares analyses revealed that functional connectivity during ER is both distinguishable by modality (with different patterns of connectivity for facial vs. vocal ER) and across time—with changes in connectivity being particularly pronounced for vocal ER. ER accuracy was greater for faces than voices, and positively associated with age; although task performance did not change appreciably across a 1-year period, changes in latent functional connectivity patterns across time predicted participants' ER accuracy at Time 2. Taken together, these results suggest that vocal and facial ER are supported by distinguishable neural correlates that may undergo different developmental trajectories. Our findings are also preliminary evidence that changes in network integration may support the development of ER skills in childhood and adolescence.

No Pain, No Gain—Giving Real-Time Emotional Feedback in a Virtual Mirror Improves Collaboration in Virtual Teamwork

This study investigates the impact of real-time emotional feedback on the quality of teamwork conducted over videoconferencing. We developed a framework that provides real-time feedback through a virtual mirror based on facial and voice emotion recognition. In an experiment with 28 teams (84 participants), teams collaborated over Zoom (version 5.16.6) to set up a virtual Mars station using custom simulation software (Mars Star City, version 4.0). Participants were divided into 14 experimental teams, which were shown the virtual mirror, and 14 control teams without it. Team performance was measured by the improvement in the Mars simulation output quality. Our analysis using correlation, multi-level regression, and machine learning revealed that fewer interruptions but an increasing number over time correlated with higher performance. Higher vocal arousal and happiness also enhanced performance. We confirmed that female presence in teams boosts performance. SHAP values indicated that high variability in happiness, head movement, and positive facial valence—an “emotional rollercoaster”—positively predicted team performance. The experimental group outperformed the control group, suggesting that virtual mirroring improves virtual teamwork and that interrupting each other more while speaking less, leads to better results.

A Randomized Trial Utilizing EEG Brain Computer Interface to Improve Facial Emotion Recognition in Autistic Adults

A Randomized Trial Utilizing EEG Brain Computer Interface to Improve Facial Emotion Recognition in Autistic Adults