Facial Emotion Research Articles

Negative urgency as a state-level process.

Test whether global self-reports of urgency moderated the within-person associations of affect and impulsive behaviors. Negative urgency is a personality trait that is a risk factor for a range of psychopathology. Although it is assumed that global self-reports of urgency measure individual tendencies to act more impulsively in the face of negative emotions, evidence from ecological momentary assessment studies is mixed. In this Registered Report, we used ecological momentary assessment data from a large sample of young adults (n = 496, age 18-22, 5 surveys per day for 40 days). All forms of momentary impulsivity were impaired in moments when people reported more intense negative emotions, but global self-reports of urgency did not explain individual differences in this association. Moreover, averaged affective states, rather than specific dimensions, affective circumplex, or appraisals, best predicted impulsive states. Results suggest that face-valid interpretations of global self-report of urgency are inaccurate, and it may be important to understand how some people come to understand themselves as high on urgency rather than assuming that people's self-reports of their motivations are accurate. Momentary experiences of emotions globally impact multiple weakly to moderately associated impulsive behaviors, and future research should seek to understand both when and for whom these associations are strongest.

Down-modulation of functional ventral striatum activation for emotional face stimuli in patients with insula damage.

Insula damage results in substantial impairments in facial emotion recognition. In particular, left hemispheric damage appears to be associated with poorer recognition of aversively rated facial expressions. Functional imaging can provide information on differences in the processing of these stimuli in patients with insula lesions when compared to healthy matched controls (HCs). We therefore investigated 17 patients with insula lesions in the chronic stage following stroke and 13 HCs using a passive-viewing task with pictures of facial expressions testing the blood oxygenation dependent (BOLD) effect in predefined regions of interest (ROIs). We expected a decrease in functional activation in an area modulating emotional response (left ventral striatum) but not in the facial recognition areas in the left inferior fusiform gyrus. Quantification of BOLD-response in ROIs but also voxel-based statistics confirmed this hypothesis. The voxel-based analysis demonstrated that the decrease in BOLD in the left ventral striatum was driven by left hemispheric damaged patients (n = 10). In our patient group, insula activation was strongly associated with the intensity rating of facial expressions. In conclusion, the combination of performance testing and functional imaging in patients following circumscribed brain damage is a challenging method for understanding emotion processing in the human brain.

Medical image intelligent diagnosis system based on facial emotion recognition and convolutional neural network

The facial emotional information of patients is one of the important indicators of their health status. By combining facial emotion recognition technology, medical imaging intelligent diagnostic systems can more comprehensively understand the psychological and physiological status of patients. This study first selected medical imaging datasets from the publicly available Medical Machine Learning Image Standard database and performed necessary preprocessing, including data cleaning, enhancement, and standardization. Next, a CNN model is constructed to extract key features from medical images and facial expression images through structures such as convolutional layers, activation functions, pooling layers, and fully connected layers. In addition, the system also integrates facial emotion recognition module and medical image analysis module, improving the comprehensiveness of diagnosis through feature fusion. Finally, the diagnostic results are displayed through a user interaction interface. Through comparative experiments, this study verified that the proposed CNN model outperforms traditional LSTM and SVM models in terms of accuracy, recall, and precision. The CNN model has shown significant advantages in medical imaging intelligent diagnosis systems, especially achieving a recall rate of 99%, demonstrating extremely high disease recognition ability. The results of this study not only provide new research directions for the field of intelligent diagnosis in medical imaging, but also lay a solid foundation for future clinical applications.

Medical image intelligent diagnosis system based on facial emotion recognition and convolutional neural network

The facial emotional information of patients is one of the important indicators of their health status. By combining facial emotion recognition technology, medical imaging intelligent diagnostic systems can more comprehensively understand the psychological and physiological status of patients. This study first selected medical imaging datasets from the publicly available Medical Machine Learning Image Standard database and performed necessary preprocessing, including data cleaning, enhancement, and standardization. Next, a CNN model is constructed to extract key features from medical images and facial expression images through structures such as convolutional layers, activation functions, pooling layers, and fully connected layers. In addition, the system also integrates facial emotion recognition module and medical image analysis module, improving the comprehensiveness of diagnosis through feature fusion. Finally, the diagnostic results are displayed through a user interaction interface. Through comparative experiments, this study verified that the proposed CNN model outperforms traditional LSTM and SVM models in terms of accuracy, recall, and precision. The CNN model has shown significant advantages in medical imaging intelligent diagnosis systems, especially achieving a recall rate of 99%, demonstrating extremely high disease recognition ability. The results of this study not only provide new research directions for the field of intelligent diagnosis in medical imaging, but also lay a solid foundation for future clinical applications.

Examining learners’ cognitive processes and learning emotions in digital game-based vocabulary learning: evidence from eye-tracking technology and facial emotion recognition

Purpose This research explored the intersection of cognitive processes, emotions and their impacts on digital game-based vocabulary learning (DGVL) among university students. Recognizing the scant research in this area, especially with integrating innovative technologies, this study aims to understand the influence of these elements using advanced monitoring tools. Design/methodology/approach This inquiry was carried out as an observational study involving 44 university students segmented into three English language proficiency levels: high, intermediate and low based on their English course scores. The methodological tools included a portable eye tracker to observe visual behaviors and deep learning technology to identify and analyze the participants’ emotional responses and engagement with the DGVL during the learning process. Findings The results showed that distinct fixation sequences and variations in visual attention during DGVL were correlated with different levels of competency, suggesting a direct correlation between visual engagement and language competence. In addition, emotional transitions, predominantly from engagement (“flow”) to challenge (“frustration”), were common among participants, reflecting the emotional dynamics of learning. Furthermore, all participants consistently focused on the English vocabulary definitions, indicative of their targeted approach to understanding and test preparation. These findings highlighted the intricate dynamics between emotions and cognitive processes in learning environments. Originality/value Contribution of this study shows the interplay of cognitive engagement and emotional experiences in the context of DGVL. It underscored the complex nature of these factors and their collective influence on learners’ visual and emotional engagement, offering valuable implications for educational strategies and technological applications in language learning.

Fear is more right lateralized than happiness and anger: Evidence for the motivational hypothesis of emotional face perception?

ABSTRACT Facial emotion processing (FEP) tends to be right hemisphere lateralized. This right-hemispheric bias (RHB) for FEP varies within and between individuals. The aim of the present research was to examine evidence pertaining to the prominent theories of FEP hemispheric bias as measured by a half-emotional half-neutral (no emotion) chimeric faces task. FEP hemispheric bias was indexed using laterality quotients (LQs) calculated from a Chimeric Faces Task completed by 427 adults recruited from the general population aged 18–67 years. Participants indicated which of two identical (but mirrored) emotional-neutral chimeric faces were more emotive. While all investigated emotions (fear, anger, and happiness) were right lateralized, fear was significantly more right lateralized than anger and happiness. These results provide evidence for both the right hemisphere hypothesis and the motivational hypothesis of emotion perception.

Impact of psychosocial stress on facial emotion recognition in schizophrenia and controls: an experimental study in a forensic sample.

Psychotic disorders have been associated with dysregulated stress reactions and adaptation. Little is known about the neuroendocrine responses to psychosocial stress in justice-involved individuals with schizophrenia. Using an experimental research design, the present study aims to examine differences in the subjective and neuroendocrine responses to psychosocial stress and its impact on facial emotion recognition (FER) and performance on an arithmetic task in chronically ill justice-involved individuals with schizophrenia (PAT) and a healthy control group. PAT undergoing treatment in forensic psychiatric inpatient wards (n = 17) and a healthy control group (n = 17) were assessed regarding sociodemographic and clinical characteristics. Additionally, salivary cortisol levels, measured before and after performing a psychosocial stress task [Montreal Imaging Stress Task (MIST)], and performance on an arithmetic problem-solving task and two FER tasks were recorded. Two participants dropped out, one from each group. Therefore, the final sample consisted of 32 individuals. Significant group differences in FER were recorded. There was a significant rise in subjective perception of momentary strain relating to the induction of psychosocial stress in both groups. Notably, the pre-stress level of subjective strain was higher in the PAT group than controls. Acute psychosocial stress induced an increase in FER performance in a sub-task related to naming emotions in individuals with schizophrenia spectrum disorder. The results underline the importance of psychosocial and therapeutic interventions aimed at strengthening stress resilience in individuals with schizophrenia spectrum disorders.

Confrontation with others' emotions changes bodily resonance differently in those with low and high levels of depersonalization.

We present novel research on the cortical dynamics of atypical perceptual and emotional processing in people with symptoms of depersonalization-derealization disorder (DP-DR). We used electroencephalography (EEG)/event-related potentials (ERPs) to delineate the early perceptual mechanisms underlying emotional face recognition and mirror touch in adults with low and high levels of DP-DR symptoms (low-DP and high-DP groups). Face-sensitive visual N170 showed markedly less differentiation for emotional versus neutral face-voice stimuli in the high- than in the low-DP group. This effect was related to self-reported bodily symptoms like disembodiment. Emotional face-voice primes altered mirror touch at somatosensory cortical components P45 and P100 differently in the two groups. In the high-DP group, mirror touch occurred only when seeing touch after being confronted with angry face-voice primes. Mirror touch in the low-DP group, however, was unaffected by preceding emotions. Modulation of mirror touch following angry others was related to symptoms of self-other confusion. Results suggest that others' negative emotions affect somatosensory processes in those with an altered sense of bodily self. Our findings are in line with the idea that disconnecting from one's body and self (core symptom of DP-DR) may be a defence mechanism to protect from the threat of negative feelings, which may be exacerbated through self-other confusion. This article is part of the theme issue 'Sensing and feeling: an integrative approach to sensory processing and emotional experience'.

Modulation of the somatosensory blink reflex under fear

ObjectiveThis study evaluated the isolated and combined effects of fear and PPS paradigms on SBR. MethodThe prospective study was conducted with healthy participants. After stimulation of the right median nerve at the wrist, bilateral recordings were randomized under the following conditions: First experiment (with the right hand on the chair armrest): i. baseline recordings, ii. while watching fearful facial expressions from the Karolinska Emotional Faces battery (fear), iii. post-watching (post-fear), iv. while watching neutral facial expressions from the same battery (neutral), v. Immediately after viewing (post-neutral). Second experiment (right hand 2 cm away from the right eye, PPS): i. reference condition (PPS), ii. while watching fearful facial expressions (PPS-fear), iii. while watching neutral facial expressions (PPS-neutral). In each condition, SBR latency, area, duration, and amplitudes were measured and compared between conditions. ResultsWe included 16 participants. SBR could be recorded in 11 (mean age:30.7 ± 5.2, F/M:5/6). First experiment: SBR amplitude was significantly reduced in fear condition (p = 0.008), and SBR area was reduced considerably in fear and post-fear conditions (p = 0.004) compared to the baseline. Second experiment: The SBR area was higher in the PPS (p = 0.009) compared to the baseline and even higher in the fearPPS compared to the PPS (p = 0.038). In neutral or PPS-neutral conditions, the area of the SBR did not change significantly. ConclusionFear suppressed SBR, but fear increased SBR when a threat stimulus was present. The findings were unrelated to habituation or attention, indicating cortical-amygdala-bulbar connections.

The Emotional Face of Language: Sentiment Analysis of English and Korean Subtitles on Barbie

Barbie: A Touch of Magic, Eposide 5th, an English and Korean subtitled Barbie cartoon, is used in this study to investigate the relationship between language culture and emotions. Finding out how language and culture impact emotional expression and what this means for language acquisition, emotional intelligence development, and cross-cultural communication are the key objectives. An in-depth analysis was done on the emotional intonations that change between the Korean and English subtitles, as well as any possible implications for language learning. The study emphasizes the intricate connection between language and society as well as the significance of including affective components in language instruction materials. The results imply that when learning a language and acquiring cross-cultural communication skills, the degree to which an emotion is communicated matters. The study offers helpful data that can be utilized in the domains of communication techniques, translation, and language instruction. It also emphasizes how important cultural sensitivity is for developing good communication and empathy skills. The purpose of this study is to advance knowledge of the relationship between language and culture and establish the framework for further investigation into this field.

Other ethnicity effects in ensemble coding of facial expressions.

Cultural difference in ensemble emotion perception is an important research question, providing insights into the complexity of human cognition and social interaction. Here, we conducted two experiments to investigate how emotion perception would be affected by other ethnicity effects and ensemble coding. In Experiment 1, two groups of Asian and Caucasian participants were tasked with assessing the average emotion of faces from their ethnic group, other ethnic group, and mixed ethnicity groups. Results revealed that participants exhibited relatively accurate yet amplified emotion perception of their group faces, with a tendency to overestimate the weight of the faces from the other ethnic group. In Experiment 2, Asian participants were instructed to discern the emotion of a target face surrounded by faces from Caucasian and Asian faces. Results corroborated earlier findings, indicating that while participants accurately perceived emotions in faces of their ethnicity, their perception of Caucasian faces was noticeably influenced by the presence of surrounding Asian faces. These findings collectively support the notion that the other ethnicity effect stems from differential emotional amplification inherent in ensemble coding of emotion perception.

An efficient facial emotion recognition using convolutional neural network with local sorting binary pattern and whale optimization algorithm

An efficient facial emotion recognition using convolutional neural network with local sorting binary pattern and whale optimization algorithm

Registered report “Categorical perception of facial expressions of anger and disgust across cultures”

ABSTRACT Previous research has demonstrated that individuals from Western cultures exhibit categorical perception (CP) in their judgments of emotional faces. However, the extent to which this phenomenon characterises the judgments of facial expressions among East Asians remains relatively unexplored. Building upon recent findings showing that East Asians are more likely than Westerners to see a mixture of emotions in facial expressions of anger and disgust, the present research aimed to investigate whether East Asians also display CP for angry and disgusted faces. To address this question, participants from Canada and China were recruited to discriminate pairs of faces along the anger-disgust continuum. The results revealed the presence of CP in both cultural groups, as participants consistently exhibited higher accuracy and faster response latencies when discriminating between-category pairs of expressions compared to within-category pairs. Moreover, the magnitude of CP did not vary significantly across cultures. These findings provide novel evidence supporting the existence of CP for facial expressions in both East Asian and Western cultures, suggesting that CP is a perceptual phenomenon that transcends cultural boundaries. This research contributes to the growing literature on cross-cultural perceptions of facial expressions by deepening our understanding of how facial expressions are perceived categorically across cultures.

Attachment and inter-individual differences in empathy, compassion, and theory of mind abilities

ABSTRACT Social processing, namely the ability to understand others’ cognitive and affective states, is crucial for successful social interaction. It encompasses socio-affective abilities such as empathy and compassion, as well as socio-cognitive abilities such as theory of mind (ToM). This study examined the link between social processing and attachment. Our study goes beyond previous research in that social processing abilities were assessed in a single, state-of-the-art behavioral paradigm using video narratives, the EmpaToM. Attachment was captured with the Adult Attachment Interview (N = 85; 50.60% women, M age = 25.87 ± 4.50 years) measuring participants’ present-day capacity to think about and communicate attachment-relevant information about the past. Additionally, a self-report attachment questionnaire was employed (N = 158). We found that AAI-based attachment security (vs. insecurity) was associated with higher behavioral ToM abilities. Furthermore, self-reported attachment avoidance was negatively correlated with behavioral compassion abilities. Our findings provide further evidence that interview-based and self-reported attachment measures do not converge, but may rather be understood as capturing different facets of attachment that relate to different components of social processing. We conclude that individuals with secure, non-avoidant attachment show social abilities that allow them to better understand others’ thoughts and generate positive, caring emotions in face of others’ distress.

Expression perceptive fields explain individual differences in the recognition of facial emotions

Humans can use the facial expressions of another to infer their emotional state, although it remains unknown how this process occurs. Here we suppose the presence of perceptive fields within expression space, analogous to feature-tuned receptive-fields of early visual cortex. We developed genetic algorithms to explore a multidimensional space of possible expressions and identify those that individuals associated with different emotions. We next defined perceptive fields as probabilistic maps within expression space, and found that they could predict the emotions that individuals infer from expressions presented in a separate task. We found profound individual variability in their size, location, and specificity, and that individuals with more similar perceptive fields had similar interpretations of the emotion communicated by an expression, providing possible channels for social communication. Modelling perceptive fields therefore provides a predictive framework in which to understand how individuals infer emotions from facial expressions.

Relationship between low mood and micro-expression processing: evidence of negative bias in interpreting fleeting facial expressions.

Depression affects the recognition of emotion in facial expressions by reducing the detection accuracy and adding a bias towards negativity. However, no study has examined associations between depression and the recognition of microfacial expressions (fleeting facial cues of emotions in people's faces). Thus, we investigated associations between low mood and micro-expression processing using video stimuli of micro-expressions. We examined whether (i) individuals with low mood had trouble recognizing emotions, (ii) were more likely to perceive happy facial expressions as neutral and neutral facial expressions as sad, and (iii) recognized sad emotional expressions better than control subjects (n = 349). We found that participants with low mood showed poorer performance when judging emotions in faces (p = 0.03). Furthermore, there was a specific deficit among them in recognizing happiness. Lastly, participants with low moods were more likely to perceive neutral faces as sad (p = 0.042). However, no evidence was found that individuals with low moods confused happy faces as neutral or were better than the control group at recognizing sad faces. Our results show that mood affects the perception of emotions in facial expressions, which has the potential to negatively affect interpersonal interactions and ultimately quality of life.

Neural Substrates of Emotion Processing and Cognitive Control Over Emotion in Youth Anxiety: An RDoC-Informed Study Across the Clinical to Nonclinical Continuum of Severity

Clinically anxious youth are hypervigilant to emotional stimuli and display difficulty shifting attention from emotional to nonemotional stimuli, suggesting impairments in cognitive control over emotion. However, it is unknown whether the neural substrates of such biases vary across the clinical-to-nonclinical range of anxiety or by age. Youth aged 7 to 17 years with clinical anxiety (n= 119) or without an anxiety diagnosis (n= 41) matched emotional faces or matched shapes flanked by emotional face distractors during magnetic resonance imaging, probing emotion processing and cognitive control over emotion, respectively. Building from the National Institute of Mental Health Research Domain Criteria (RDoC) framework, clinically anxious youth were sampled across diagnostic categories, and non-clinically affected youth were sampled across minimal-to-subclinical severity. Across both conditions, anxiety severity was associated with hyperactivation in the right inferior parietal lobe, a substrate of hypervigilance. Brain-anxiety associations were also differentiated by attentional state; anxiety severity was associated with greater left ventrolateral prefrontal cortex activation during emotion processing (face matching) and greater activation in the left posterior superior temporal sulcus and temporoparietal junction (and slower responses) during cognitive control over emotion (shape matching). Age also moderated associations between anxiety and cognitive control over emotion, such that anxiety was associated with greater right thalamus and bilateral posterior cingulate cortex activation for children at younger and mean ages, but not for older youth. Aberrant function in brain regions implicated in stimulus-driven attention to emotional distractors may contribute to anxiety in youth. Results support the potential utility of attention modulation interventions for anxiety that are tailored to developmental stage. Dimensional Brain Behavior Predictors of CBT Outcomes in Pediatric Anxiety; https://clinicaltrials.gov; NCT02810171.

Alterations in Neural Activation During Facial Emotion Processing in Adolescent Male Participants With Klinefelter Syndrome.

Klinefelter syndrome (KS) is the most common sex-chromosome aneuploidy (47,XXY), affecting 1 in 500 male participants. The phenotype of male participants with KS includes both physical features, such as tall stature and testicular insufficiency, and behavioral alterations, including difficulties in social functioning, anxiety, and depression. Studies examining underlying neural alterations associated with the behavioral phenotype, however, are sparse. We aimed to address this gap in knowledge using functional magnetic resonance imaging in conjunction with an emotion processing paradigm. Functional magnetic resonance imaging was conducted on 38 children and adolescents with KS ( Mage = 12.85, SD = 2.45) and 47 typical developing (control) boys ( Mage = 12.04, SD = 1.82) as they completed a facial emotion processing task. Group differences in activation occurring during the processing of angry versus neutral faces were examined while controlling for age. The results indicated that relative to typically developing boys, boys with KS exhibited anomalous increases in activation of frontal, temporal, and occipital cortices. Within the KS group, secondary analyses indicated that greater activation in these regions was associated with more internalizing symptoms (e.g., anxiety, depression, withdrawn behaviors) and greater social impairments (e.g., social cognition, social communication, social motivation, social communication and interaction, functional communication). The findings from this study indicate a possible neural correlation for difficulties in social and emotional function in KS and add to a growing body of research aimed at increasing our understanding of neural biomarkers in this condition. Future studies that examine the influence of testosterone-replacement therapy on these differences are warranted.

Interaction of perceived social support and childhood maltreatment on limbic responsivity towards negative emotional stimuli in healthy individuals

Childhood maltreatment (CM) is associated with increased limbic activity, while social support is linked to decreased limbic activity towards negative stimuli. Our study aimed to explore the interaction of perceived social support with CM, and their combined impact on limbic activity in negative emotion processing. A total of 130 healthy individuals (HC) underwent a negative emotional face processing paradigm. They were divided into two groups based on the Childhood Trauma Questionnaire: n = 65 HC without CM matched with n = 65 HC with CM. In a region-of-interest approach of the bilateral amygdala-hippocampus-complex (AHC), regression analyses investigating the association of CM and perceived social support with limbic activity and a social support x CM ANCOVA were conducted. CM was associated with increased AHC activity, while perceived social support tended to be associated with decreased AHC activity during negative emotion processing. The ANCOVA showed a significant interaction in bilateral AHC activity (pFWE ≤ 0.024) driven by a negative association between perceived social support and bilateral AHC activity in HC without CM. No significant association was observed in HC with CM. Exploratory analyses using continuous CM scores support this finding. Our results suggest that CM moderates the link between perceived social support and limbic activity, with a protective effect of perceived social support only in HC without CM. The lack of this effect in HC with CM suggests that CM may alter the buffering effect of perceived social support on limbic functioning, highlighting the potential need for preventive interventions targeting social perception of HC with CM.

No Pain, No Gain—Giving Real-Time Emotional Feedback in a Virtual Mirror Improves Collaboration in Virtual Teamwork

This study investigates the impact of real-time emotional feedback on the quality of teamwork conducted over videoconferencing. We developed a framework that provides real-time feedback through a virtual mirror based on facial and voice emotion recognition. In an experiment with 28 teams (84 participants), teams collaborated over Zoom (version 5.16.6) to set up a virtual Mars station using custom simulation software (Mars Star City, version 4.0). Participants were divided into 14 experimental teams, which were shown the virtual mirror, and 14 control teams without it. Team performance was measured by the improvement in the Mars simulation output quality. Our analysis using correlation, multi-level regression, and machine learning revealed that fewer interruptions but an increasing number over time correlated with higher performance. Higher vocal arousal and happiness also enhanced performance. We confirmed that female presence in teams boosts performance. SHAP values indicated that high variability in happiness, head movement, and positive facial valence—an “emotional rollercoaster”—positively predicted team performance. The experimental group outperformed the control group, suggesting that virtual mirroring improves virtual teamwork and that interrupting each other more while speaking less, leads to better results.