Facial Expression Intensity Research Articles

Autism Identification Based on the Intelligent Analysis of Facial Behaviors: An Approach Combining Coarse- and Fine-Grained Analysis

Background: Facial behavior has emerged as a crucial biomarker for autism identification. However, heterogeneity among individuals with autism poses a significant obstacle to traditional feature extraction methods, which often lack the necessary discriminative power. While deep-learning methods hold promise, they are often criticized for their lack of interpretability. Methods: To address these challenges, we developed an innovative facial behavior characterization model that integrates coarse- and fine-grained analyses for intelligent autism identification. The coarse-grained analysis provides a holistic view by computing statistical measures related to facial behavior characteristics. In contrast, the fine-grained component uncovers subtle temporal fluctuations by employing a long short-term memory (LSTM) model to capture the temporal dynamics of head pose, facial expression intensity, and expression types. To fully harness the strengths of both analyses, we implemented a feature-level attention mechanism. This not only enhances the model’s interpretability but also provides valuable insights by highlighting the most influential features through attention weights. Results: Upon evaluation using three-fold cross-validation on a self-constructed autism dataset, our integrated approach achieved an average recognition accuracy of 88.74%, surpassing the standalone coarse-grained analysis by 8.49%. Conclusions: This experimental result underscores the improved generalizability of facial behavior features and effectively mitigates the complexities stemming from the pronounced intragroup variability of those with autism, thereby contributing to more accurate and interpretable autism identification.

Perceptual sensitivity to labeling stereotyped emotion expressions: Associations with age and subclinical psychopathology symptoms from childhood through early adulthood.

This study investigates (a) age-related differences in how the intensity of stereotyped facial expressions influence the emotion label children, adolescents, and adults assign to that face and (b) how this perceptual sensitivity relates to subclinical symptoms of psychopathology. In 2015-2016, 184 participants aged 4-25 years viewed posed stereotypes of angry, fearful, sad, and happy expressions morphed with neutral expressions at 10%-90% intensity. Thin plate regression smoothing splines were used to chart nonlinear associations between age and the perceptual threshold participants needed to assign the emotion label expected based on cultural consensus. Results suggest that sensitivity to labeling stereotypical happy faces as "happy" peaked by age 4. Sensitivity to perceiving stereotypical angry faces as "angry" increased from ages 4 to 7 and then plateaued. In contrast, sensitivity to perceiving stereotypical fearful and sad faces demonstrated protracted development, not reaching a plateau until ages 15 and 16, respectively. Reduction in selecting the "I don't know" response was the primary driver of these age-related changes. Stereotyped fear expressions required the highest intensity to be labeled as such and showed the most marked change in perceptual threshold across development. Interestingly, lower intensity morphs of stereotypical fear faces were frequently labeled "sad." Furthermore, perceiving lower intensity fear morphs was associated with fewer internalizing and externalizing symptoms in participants aged 7-19. This study describes the development of perceptual sensitivity to labeling stereotypical expressions of emotion according to cultural consensus and shows that how people perceive and categorize ambiguous facial expressions is associated with vulnerability to psychopathology. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

The Influence of Emotional Facial Expression Intensity on Decoding Accuracy: High Intensity Does Not Yield High Accuracy

AbstractThis study examined the relationship between the intensity of emotional expressions in facial stimuli and receivers' decoding accuracy for six basic emotions: anger, disgust, fear, happiness, sadness, and surprise. A laboratory experiment was conducted using the forced‐choice method, in which the intensity of each stimulus was manipulated at every 10% interval using the morphing technique. To explore whether a linear relationship would be observed when the intensity was finely manipulated at 10% intervals, a hierarchical multiple regression analysis was performed. The mean percentage of correct responses for each stimulus was the dependent variable, and the linear, quadratic, and cubic terms of the stimulus intensity were the independent variables. The results showed that the linear model was not adopted as the final model for all facial expressions; that is, the effect of the squared term of intensity was significant for anger, disgust, fear, and sadness, while the effect of the cubic term of intensity was significant for happiness and surprise. Our findings indicate that a higher intensity of emotional expression does not yield higher decoding accuracy.

Lightweight facial expression estimation for mobile computing in portable device

AbstractFacial expression recognition has been studied for many years, especially with the development of deep learning. However, the existing researches still have the following two issues. Firstly, the intensity of facial expression is neglected. Secondly, the deep learning based approaches cannot be directly deployed in the devices with limited resources. In order to tackle these two issues, this paper proposes a lightweight facial expression estimation method using a shallow ordinal regression algorithm, which is deployed in a portable smart device for mobile computing in IoTs. Compared with classification based facial expression recognition methods, ordinal regression considers the intensity of facial expression to achieve better mean absolute error (MAE), which is validated by experiments on several public facial expression datasets. The simulation in portable device also demonstrates its effectiveness for mobile computing.

Can Perceivers Differentiate Intense Facial Expressions? Eye Movement Patterns.

Recent research on intense real-life faces has shown that although there was an objective difference in facial activities between intense winning faces and losing faces, viewers failed to differentiate the valence of such expressions. In the present study, we explored whether participants could perceive the difference between intense positive facial expressions and intense negative facial expressions in a forced-choice response task using eye-tracking techniques. Behavioral results showed that the recognition accuracy rate for intense facial expressions was significantly above the chance level. For eye-movement patterns, the results indicated that participants gazed more and longer toward the upper facial region (eyes) than the lower region (mouth) for intense losing faces. However, the gaze patterns were reversed for intense winning faces. The eye movement pattern for successful differentiation trials did not differ from failed differentiation trials. These findings provided preliminary evidence that viewers can utilize intense facial expression information and perceive the difference between intense winning faces and intense losing faces produced by tennis players in a forced-choice response task.

Making or breaking the case for a plain face – Is human perception of canine facial expressivity influenced by physical appearance?

Abstract Facial communication is important in both human-human interactions and human-dog interactions. Individual factors, such as experience, relationship history, and mood, to name a few, influence the reception of facial signals/expressions. But superficial facial features are also significant in human communication, and likely impact communication between dogs and humans. For example, humans are better at evaluating the frequency and intensity of facial expressions in dogs that have plainer faces, if the dog is familiar to them, which could be related to a preference for non-complexity (human faces are generally much less physically diverse than dog faces). This study explored the effect of the physical complexity of dog and human faces on the perceived expressiveness of neutral-faced, unknown individuals of both species. Results indicate that when looking at static images of unknown dogs and humans, facial complexity has minimal impact on how expressive people perceive them to be. However, dogs are consistently ranked as more expressive than humans, and people who live with dogs tend to rank neutral-face dogs of all facial complexity levels as more expressive compared to the rankings of humans who do not live with dogs – which we hypothesize may be the result of a desire/tendency of dog owners to “read meaning” into dog faces.

Facing emotions: real-time fMRI-based neurofeedback using dynamic emotional faces to modulate amygdala activity.

Maladaptive functioning of the amygdala has been associated with impaired emotion regulation in affective disorders. Recent advances in real-time fMRI neurofeedback have successfully demonstrated the modulation of amygdala activity in healthy and psychiatric populations. In contrast to an abstract feedback representation applied in standard neurofeedback designs, we proposed a novel neurofeedback paradigm using naturalistic stimuli like human emotional faces as the feedback display where change in the facial expression intensity (from neutral to happy or from fearful to neutral) was coupled with the participant's ongoing bilateral amygdala activity. The feasibility of this experimental approach was tested on 64 healthy participants who completed a single training session with four neurofeedback runs. Participants were assigned to one of the four experimental groups (n = 16 per group), i.e., happy-up, happy-down, fear-up, fear-down. Depending on the group assignment, they were either instructed to "try to make the face happier" by upregulating (happy-up) or downregulating (happy-down) the amygdala or to "try to make the face less fearful" by upregulating (fear-up) or downregulating (fear-down) the amygdala feedback signal. Linear mixed effect analyses revealed significant amygdala activity changes in the fear condition, specifically in the fear-down group with significant amygdala downregulation in the last two neurofeedback runs as compared to the first run. The happy-up and happy-down groups did not show significant amygdala activity changes over four runs. We did not observe significant improvement in the questionnaire scores and subsequent behavior. Furthermore, task-dependent effective connectivity changes between the amygdala, fusiform face area (FFA), and the medial orbitofrontal cortex (mOFC) were examined using dynamic causal modeling. The effective connectivity between FFA and the amygdala was significantly increased in the happy-up group (facilitatory effect) and decreased in the fear-down group. Notably, the amygdala was downregulated through an inhibitory mechanism mediated by mOFC during the first training run. In this feasibility study, we intended to address key neurofeedback processes like naturalistic facial stimuli, participant engagement in the task, bidirectional regulation, task congruence, and their influence on learning success. It demonstrated that such a versatile emotional face feedback paradigm can be tailored to target biased emotion processing in affective disorders.

The eyes as the exclamation mark of the face: exploring the relationship between eye size, intensity of female facial expressions and attractiveness in a range of emotions.

The eyes play an important role in communicating emotions and shape the determination of other facial attributes. Here, we explored the relationship between eye size, perceived intensity and attractiveness of facial expressions. A sample of N = 63 participants (men and women, aged 18-35) rated attractiveness and emotional intensity for images displaying emotionally expressive women's faces with digitally manipulated eye size (15% smaller, unchanged, or 15% larger). The analysis of perceived intensity showed an interaction parameter between eye size and gender. Female individuals reported differences when comparing unchanged and larger eyes; male participants showed differences across all eye size comparisons (smaller-unchanged, smaller-larger, unchanged-larger). Regarding perceived attractiveness, faces with smaller eyes registered lower mean scores than both unchanged and larger. The lowest intensity level was associated with neutral faces and the highest with fearful ones. Faces displaying happiness were perceived as the most attractive. Larger eyes seem to make emotions more intense and attractive. We suggest that the more intense phenomenon serves an evolutive purpose, as it might encourage caretaking behavior.

Voluntary imitation of dynamic facial expressions in attention deficit hyperactivity disorder: a facial-behavior analysis

ABSTRACT Objective The difficulties involved in social interaction among children with attention deficit hyperactivity disorder (ADHD) have been shown in many studies. Based on the knowledge that the imitation of facial expressions is a key factor in social interaction and functionality, the focus of prior studies has been on the evaluation of facial expressions in individuals with ADHD. However, little is known about voluntary facial mimicry in individuals with ADHD. In this context, we aimed to evaluate the voluntary-facial-imitation intensity of dynamic facial expressions in children with ADHD. Method Forty-one children with ADHD and 53 typically developing children were included in the study. Participants were presented with a video of six basic emotions and neutral facial expressions selected from the EU-Emotion Stimulus Set via a screen. After each emotion, the instruction “now imitate it” was given. While the children watched the video, their faces were recorded with a webcam. The intensity of the children’s voluntary facial imitations was examined with a computer vision program (Openface) that performs facial analysis on recorded videos. Results There was no significant difference between the groups in terms of facial emotion recognition accuracy. In group comparisons of voluntary facial mimicry, children with ADHD showed a significantly higher imitation intensity after emotional expressions of sadness, surprise and fear. There was no difference between the groups after the emotions of happiness, anger and disgust. Conclusion This non-obtrusive, noninvasive, and cost-effective method allowed us to measure the quantitative differences in facial mimicry between children with ADHD and typically developing children. Our results contribute new information to the literature by indicating which emotions can be used in the evaluation of social communication skills, as well as intervention targets for these skills, in children with ADHD.

The effect of masks on the emotion perception of a facial crowd

The present study investigated the effect of facial masks on people’s ability to perceive emotions in crowds. We presented faces with the bottom halves occluded by masks or full faces without occlusion. In two sequentially presented crowds, we varied the number of faces, emotional valence, and intensity of facial expressions, examining the impact of masks on the perception of crowd emotion. Participants reported which of the two crowds they would avoid based on the crowds’ average emotions. The participants’ ability to judge the average emotion of a crowd, especially a crowd expressing happiness, was impaired when the crowd wore masks. For faces covered by masks, crowd emotion judgments were more negatively biased than those without masks. However, participants could still distinguish the emotional intensities of a crowd wearing masks above chance. Additionally, participants responded more quickly to a crowd with more people without compromising accuracy, despite the perceptual challenges imposed by facial masks. Our results suggest that under ambiguous social situations in which individuals’ emotions are partially hidden by masks, a large group may provide stronger social cues than a small group, thereby promoting communication and regulating social behaviors.

Predisposing Variables in Children with Risk of Disruptive Mood: A Clinical Case–Control Study

ABSTRACT Disruptive mood dysregulation disorder (DMDD) is defined in the DSM-V as frequent, severe temper outbursts that significantly impair functioning in different environments. This was a second-stage of follow-up study, conducted to screen the frequency of DMDD in an elementary school. In the first-stage of our study, 453 children between ages 7–11 were evaluated in terms of DMDD high-risk with Children Behavior Check List (CBCL). Of the children, 30 high-risk and 30 low-risk children for DMDD according to CBCL agreed to participate this clinical case–control study. Diagnoses of anxiety disorder, attention deficit and hyperactivity disorder (ADHD), and oppositional defiant disorder were more common among children in the high-risk group than the control group. Symptom Checklist-90-Revised (SCL-90-R) mothers’ interpersonal, anger, and paranoid subscale scores were higher in the DMDD high-risk group than the control group. Children in the DMDD high-risk group scored higher than the control group in all SRS subscales. In the Diagnostic Analysis of Nonverbal Accuracy (DANVA) test, the DMDD high-risk group had higher error rates for fearful and intense facial expressions. Multiple linear regression analysis showed that having a diagnosis of ADHD, high maternal SCL-90-R anger score, and presence of a paternal psychiatric diagnosis increased the high-risk of DMDD.

Facial expression of pain: Sex differences in the discrimination of varying intensities.

It has been proposed that women are better than men at recognizing emotions and pain experienced by others. They have also been shown to be more sensitive to variations in pain expressions. The objective of the present study was to explore the perceptual basis of these sexual differences by comparing the visual information used by men and women to discriminate between different intensities of pain facial expressions. Using the data-driven Bubbles method, we were able to corroborate the woman advantage in the discrimination of pain intensities that did not appear to be explained by variations in empathic tendencies. In terms of visual strategies, our results do not indicate any qualitative differences in the facial regions used by men and women. However, they suggest that women rely on larger regions of the face that seems to completely mediate their advantage. This utilization of larger clusters could indicate either that women integrate simultaneously and more efficiently information coming from different areas of the face or that they are more flexible in the utilization of the information present in these clusters. Women would then opt for a more holistic or flexible processing of the facial information, while men would rely on a specific yet rigid integration strategy. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Mask wearing provides psychological ease but does not affect facial expression intensity estimation.

During the COVID-19 pandemic, wearing a face mask became a global daily practice. Japanese people were already accustomed to wearing masks due to their collectivistic culture, which prioritizes conformity and group harmony. In such a culture, where individuals are concerned about how others perceive them and their actions, wearing masks can be a self-protective action to prevent, escape, or reduce the severity of perceived negative feelings. Previous studies indicate that people experiencing anxiety tend to have negative biases when evaluating emotional expressions on faces. Therefore, we hypothesized that wearing a mask can reduce the negative feelings caused by social pressure, emotion processing, especially intensity perception. While our findings confirmed that wearing a mask reduced negative feelings caused by social pressure, there was no significant change in emotion intensity recognition performance. This null result might be attributed to the small effect size of the association between negative bias in emotion processing and an individual's state. In future studies, it would be valuable to include participants from non-collectivistic cultures to gain a broader understanding of the impact of wearing masks on emotion processing.

Ostracism modulates children's recognition of emotional facial expressions.

Ostracism has been shown to induce considerable physiological, behavioral and cognitive changes in adults. Previous research demonstrated its effects on children's cognitive and behavioral abilities, but less is known about its impact on their capacity to recognize subtle variations in social cues. The present study aimed at investigating whether social manipulations of inclusion and ostracism modulate emotion recognition abilities in children, and whether this modulation varies across childhood. To do so, 5- and 10-year-old children participated in a computer-based ball tossing game called Cyberball during which they were either included or ostracized. Then, they completed a facial emotion recognition task in which they were required to identify neutral facial expressions, or varying levels of intensity of angry and fearful facial expressions. Results indicated lower misidentification rates for children who were previously ostracized as compared to children who were previously included, both at 5 and 10 years of age. Moreover, when looking at children's accuracy and sensitivity to facial expressions, 5-year-olds' decoding abilities were affected by the social manipulation, while no difference between included and ostracized participants was observed for 10-year-olds. In particular, included and ostracized 10-year-old children as well as ostracized 5-year-olds showed higher accuracy and sensitivity for expressions of fear as compared to anger, while no such difference was observed for included 5-year-olds. Overall, the current study presents evidence that Cyberball-induced inclusion and ostracism modulate children's recognition of emotional faces.

Oxytocin normalizes the implicit processing of fearful faces in psychopathy: a randomized crossover study using fMRI

Adults with antisocial personality disorder with (ASPD + P) and without (ASPD – P) psychopathy commit the majority of violent crimes. Empathic processing abnormalities are particularly prominent in psychopathy, but effective pharmacological interventions have yet to be identified. Oxytocin modulates neural responses to fearful expressions in healthy populations. The current study investigates its effects in violent antisocial men. In a placebo-controlled, randomized crossover design, 34 violent offenders (19 ASPD + P; 15 ASPD – P) and 24 healthy non-offenders received 40 IU intranasal oxytocin or placebo and then completed an fMRI morphed faces task examining the implicit processing of fearful facial expressions. Increasing intensity of fearful facial expressions failed to appropriately modulate activity in the bilateral mid-cingulate cortex in violent offenders with ASPD + P, compared with those with ASPD – P. Oxytocin abolished these group differences. This represents evidence of neurochemical modulation of the empathic processing of others’ distress in psychopathy.

Role of facial familiarity and emotional expression intensity in ensemble emotion perception

When looking at groups of people, we can extract information from the different faces to derive properties of the group, such as its average facial emotion, although how this average is computed remains a matter of debate. Here, we examined whether our participants’ personal familiarity with the faces in the group, as well as the intensity of the facial expressions, biased ensemble perception. Participants judged the average emotional expression of ensembles of four different identities whose expressions depicted either neutral, angry, or happy emotions. For the angry and happy expressions, the intensity of the emotion could be either low (e.g., slightly happy) or high (very happy). When all the identities in the ensemble were unfamiliar, the presence of any high intensity emotional face biased ensemble perception towards its emotion. However, when a familiar face was present in the ensemble, perception was biased towards the familiar face’s emotion regardless of its intensity. These findings reveal that how we perceive the average emotion of a group is influenced by both the emotional intensity and familiarity of the faces comprising the group, supporting the idea that different faces may be weighted differently in ensemble perception. These findings have important implications for the judgements we make about a group’s overall emotional state may be biased by individuals within the group.

Differences Between East Asians and Westerners in the Mental Representations and Visual Information Extraction Involved in the Decoding of Pain Facial Expression Intensity.

The online version contains supplementary material available at 10.1007/s42761-023-00186-1.

Revisiting emotion recognition in different types of temporal lobe epilepsy: The influence of facial expression intensity

Temporal lobe epilepsy (TLE) can induce various difficulties in recognizing emotional facial expressions (EFE), particularly for negative valence emotions. However, these difficulties have not been systematically examined according to the localization of the epileptic focus. For this purpose, we used a forced-choice recognition task in which faces expressing fear, sadness, anger, disgust, surprise, or happiness were presented in different intensity levels from moderate to high intensity. The first objective of our study was to evaluate the impact of emotional intensity on the recognition of different categories of EFE in TLE patients compared to control participants. The second objective was to assess the effect of localizationof epileptic focus on the recognition of EFE in patients with medial temporal lobe epilepsy (MTLE) associated or not with hippocampal sclerosis (HS), or lateral temporal lobe epilepsy (LTLE). The results showed that the 272 TLE patients and the 68 control participants were not differently affected by the intensity of EFE. However, we obtained group differences within the clinical population when we took into account the localization of the temporal lobe epileptic focus. As predicted, TLE patients were impaired in recognizing fear and disgust relative to controls. Moreover, the scores of these patients varied according to the localization of the epileptic focus, but not according to the cerebral lateralization of TLE. The facial expression of fear was less well recognized by MTLE patients, with or without HS, and the expression of disgust was less well recognized by LTLE as well as MTLE without HS patients. Moreover, emotional intensity modulated differently the recognition of disgust and surprise of the three patient groups underlying the relevance of using moderate emotional intensity to distinguish the effect of epileptic focus localization. These findings should be taken into account for interpreting the emotional behaviors and deserve to befurther investigated before considering TLE surgical treatment or social cognition interventions in TLE patients.

Training machine learning algorithms for automatic facial coding: The role of emotional facial expressions' prototypicality.

Automatic facial coding (AFC) is a promising new research tool to efficiently analyze emotional facial expressions. AFC is based on machine learning procedures to infer emotion categorization from facial movements (i.e., Action Units). State-of-the-art AFC accurately classifies intense and prototypical facial expressions, whereas it is less accurate for non-prototypical and less intense facial expressions. A potential reason might be that AFC is typically trained with standardized and prototypical facial expression inventories. Because AFC would be useful to analyze less prototypical research material as well, we set out to determine the role of prototypicality in the training material. We trained established machine learning algorithms either with standardized expressions from widely used research inventories or with unstandardized emotional facial expressions obtained in a typical laboratory setting and tested them on identical or cross-over material. All machine learning models' accuracies were comparable when trained and tested with held-out dataset from the same dataset (acc. = [83.4% to 92.5%]). Strikingly, we found a substantial drop in accuracies for models trained with the highly prototypical standardized dataset when tested in the unstandardized dataset (acc. = [52.8%; 69.8%]). However, when they were trained with unstandardized expressions and tested with standardized datasets, accuracies held up (acc. = [82.7%; 92.5%]). These findings demonstrate a strong impact of the training material's prototypicality on AFC's ability to classify emotional faces. Because AFC would be useful for analyzing emotional facial expressions in research or even naturalistic scenarios, future developments should include more naturalistic facial expressions for training. This approach will improve the generalizability of AFC to encode more naturalistic facial expressions and increase robustness for future applications of this promising technology.

Insomnia and impacts on facial expression recognition accuracy, intensity and speed: A meta-analysis

Facial expressions provide nonverbal cues that are important for delivering and interpreting human emotions. Previous studies have shown that the ability to interpret facial emotions correctly could be partially impaired in sleep-deprived people. People with insomnia might also suffer from sleep loss, so we assumed that facial expression recognition ability might also be impaired in people with insomnia. Despite a growing body of research exploring insomnia's potential impacts on facial expression recognition, conflicting results have been reported, and no systematic review of this research body has been conducted. In this study, after screening 1100 records identified through database searches, six articles examining insomnia and facial expression recognition ability were included in a quantitative synthesis. The main outcomes were classification accuracy (ACC), reaction time (RT), and intensity rating—the three most studied facial expression processing variables. Subgroup analysis was performed to identify altered perceptions according to the facial expressions of four emotions—happiness, sadness, fear, and anger—used to examine insomnia and emotion recognition. The pooled standard mean differences (SMDs) and corresponding 95% confidence intervals (CIs) demonstrated that facial expression recognition among people with insomnia was less accurate (SMD = −0.30; 95% CI: −0.46, −0.14) and slower (SMD = 0.67; 95% CI: 0.18, −1.15) compared to good sleepers. The classification ACC of fearful expression was lower in the insomnia group (SMD = −0.66; 95% CI: −1.02, −0.30). This meta-analysis was registered using PROSPERO.