Recognition Of Dynamic Facial Expressions Research Articles

Idiosyncratic fixation patterns generalize across dynamic and static facial expression recognition

Facial expression recognition (FER) is crucial for understanding the emotional state of others during human social interactions. It has been assumed that humans share universal visual sampling strategies to achieve this task. However, recent studies in face identification have revealed striking idiosyncratic fixation patterns, questioning the universality of face processing. More importantly, very little is known about whether such idiosyncrasies extend to the biological relevant recognition of static and dynamic facial expressions of emotion (FEEs). To clarify this issue, we tracked observers’ eye movements categorizing static and ecologically valid dynamic faces displaying the six basic FEEs, all normalized for time presentation (1 s), contrast and global luminance across exposure time. We then used robust data-driven analyses combining statistical fixation maps with hidden Markov Models to explore eye-movements across FEEs and stimulus modalities. Our data revealed three spatially and temporally distinct equally occurring face scanning strategies during FER. Crucially, such visual sampling strategies were mostly comparably effective in FER and highly consistent across FEEs and modalities. Our findings show that spatiotemporal idiosyncratic gaze strategies also occur for the biologically relevant recognition of FEEs, further questioning the universality of FER and, more generally, face processing.

The role of the differential outcomes procedure and schizotypy in the recognition of dynamic facial expressions of emotions

Emotional facial expression recognition is a key ability for adequate social functioning. The current study aims to test if the differential outcomes procedure (DOP) may improve the recognition of dynamic facial expressions of emotions and to further explore whether schizotypal personality traits may have any effect on performance. 183 undergraduate students completed a task where a face morphed from a neutral expression to one of the six basic emotions at full intensity over 10 s. Participants had to press spacebar as soon as they identified the emotion and choose which had appeared. In the first block, participants received no outcomes. In the second block, a group received specific outcomes associated to each emotion (DOP group), while another group received non-differential outcomes after correctly responding (NOP group). Employing generalized linear models (GLMs) and Bayesian inference we estimated different parameters to answer our research goals. Schizotypal personality traits did not seem to affect dynamic emotional facial expression recognition. Participants of the DOP group were less likely to respond incorrectly to faces showing Fear and Surprise at fewer intensity levels. This may suggest that the DOP could lead to better identification of the main features that differentiate each facial expression of emotion.

Enhanced spatial-temporal learning network for dynamic facial expression recognition

The recognition of dynamic facial expressions has received increasing attention since they can better reflect the real expression process of emotion than a static image. However, due to various factors such as subtle variation differences, pose, occlusion, and illumination, it has been a challenging vision task to obtain discriminative expression features in dynamic facial expression recognition. Traditional CNN-based deep learning networks lack global and temporal contextual expression understanding, which tends to affect the final recognition of dynamic expressions. Therefore, we propose an enhanced spatial–temporal learning network (ESTLNet) for more robust dynamic facial expression recognition, which consists of a spatial fusion learning module (SFLM) and a temporal transformer enhancement module (TTEM). First, the SFLM obtains a more expressive spatial feature representation through a dual-channel feature fusion learning module. Then, the TTEM extracts more valid temporal contextual expression features based on the above spatial features through an encoder constructed by cascading a self-attention learning network and an effective gated feed-forward network. Finally, the co-enhanced spatial–temporal model approach is assessed on the four broadly used dynamic expression datasets (DFEW, AFEW, CK+, and Oulu-CASIA). Extensive experimental outcomes demonstrate that our approach surpasses several existing state-of-the-art methods, leading to notable enhancements in performance.

The Interplay between Chronotype and Emotion Regulation in the Recognition of Facial Expressions of Emotion.

Emotion regulation strategies affect the experience and processing of emotions and emotional stimuli. Chronotype has also been shown to influence the processing of emotional stimuli, with late chronotypes showing a bias towards better processing of negative stimuli. Additionally, greater eveningness has been associated with increased difficulties in emotion regulation and preferential use of expressive suppression strategies. Therefore, the present study aimed to understand the interplay between chronotype and emotion regulation on the recognition of dynamic facial expressions of emotion. To that end, 287 participants answered self-report measures and performed an online facial emotion recognition task from short video clips where a neutral face gradually morphed into a full-emotion expression (one of the six basic emotions). Participants should press the spacebar to stop each video as soon as they could recognize the emotional expression, and then identify it from six provided labels/emotions. Greater eveningness was associated with shorter response times (RT) in the identification of sadness, disgust and happiness. Higher scores of expressive suppression were associated with longer RT in identifying sadness, disgust, anger and surprise. Expressive suppression significantly moderated the relationship between chronotype and the recognition of sadness and anger, with chronotype being a significant predictor of emotion recognition times only at higher levels of expressive suppression. No significant effects were observed for cognitive reappraisal. These results are consistent with a negative bias in emotion processing in late chronotypes and increased difficulty in anger and sadness recognition for expressive suppressor morning-types.

Neural models for the cross-species recognition of dynamic facial expressions

Dynamic facial expression recognition is an essential skill of primate communication. While the neural mechanisms to recognize static facial expressions has been extensively investigated, they remain largely unclear for dynamic facial expressions. We studied plausible neural encoding mechanisms, exploiting highly controlled and realistic stimulus sets generated by computer graphics, which are also used in electrophysiological experiments. METHODS: Combining mechanisms from physiologically plausible neural models for the recognition of dynamic bodies (Giese & Poggio, 2003), static faces (Giese & Leopold, 2005) and architectures from computer vision (Simonyan et al., 2014), we devised models for the recognition of dynamic facial expressions. The first model is example-based, and encodes dynamic expressions as temporal sequences of snapshots, exploiting a sequence-selective recurrent neural network. The second model exploits norm-referenced encoding, where face-space neurons are tuned to the differences between the actual stimulus frame and a reference face, the neutral facial expression. The dynamic expressions are recognized by differentiating the responses of these face-space neurons. We tested the models with high-quality human and monkey avatars, animated with motion capture data from both species, controlling expression by motion morphing. RESULTS: Both models recognize reliably dynamic facial expressions of humans and monkeys. However, the predicted behaviour of face-tuned neurons is very different for both models. The norm-referenced model shows a highly gradual, almost linear dependence of the neuron activity with the expressivity of the stimuli, while the neurons in the example-based model show very limited capability of generalization to expressions with varying strength. We also explored if the models explain the human capability of humans to recognize human expressions on monkey faces (Taubert et al. 2020). CONCLUSIONS: The two physiologically plausible models accomplish the recognition of dynamic faces and make distinguishable predictions for physiological experiments, where norm-referenced encoding might support transfer of expression recognition across different head shapes.

Physiologically-inspired models for the recognition of dynamic facial expressions

Physiologically-inspired models for the recognition of dynamic facial expressions

Social cognition and sex: Are men and women really different?

Social cognition includes the ability to represent other people's intentions and beliefs, and the ability to share and recognise the emotions of others. Here, the main aim was to assess the possible presence of sex differences across four aspects of social cognition: (1) recognition of dynamic facial expressions; (2) representation of other people's mental states (both affective and cognitive Theory of Mind, ToM); (3) empathy; (4) identification and regulation of one's own emotions. Measures assessing social cognition were administrated to two hundred ten participants equally divided between men and women. Results showed no significant sex differences in affective and cognitive ToM, in the recognition of emotional facial expressions (with the exception of anger: women were more accurate than men), and in the ability to identify and regulate one's own emotions. A different result was found for empathy, with women reporting higher scores than men. No significant differences between women during follicular vs. luteal phase of menstrual cycle for all the social cognition measures were found. These results are discussed in light of the existing literature. To our knowledge, this study represents one of the few attempts to analyse in a single work sex differences across multiple areas of social cognition.

Strategy Shift Toward Lower Spatial Frequencies in the Recognition of Dynamic Facial Expressions of Basic Emotions: When It Moves It Is Different.

Facial expressions of emotion play a key role in social interactions. While in everyday life, their dynamic and transient nature calls for a fast processing of the visual information they contain, a majority of studies investigating the visual processes underlying their recognition have focused on their static display. The present study aimed to gain a better understanding of these processes while using more ecological dynamic facial expressions. In two experiments, we directly compared the spatial frequency (SF) tuning during the recognition of static and dynamic facial expressions. Experiment 1 revealed a shift toward lower SFs for dynamic expressions in comparison to static ones. Experiment 2 was designed to verify if changes in SF tuning curves were specific to the presence of emotional information in motion by comparing the SF tuning profiles for static, dynamic, and shuffled dynamic expressions. Results showed a similar shift toward lower SFs for shuffled expressions, suggesting that the difference found between dynamic and static expressions might not be linked to informative motion per se but to the presence of motion regardless its nature.

Anxiety Body Odors as Context for Dynamic Faces: Categorization and Psychophysiological Biases.

Body odors (BOs) can convey social information. In particular, their effects are maximal when their presence is paired with meaningful social contexts. Static faces have been widely used as social stimuli. However, they miss a key feature of our phenomenological experience, characterized by multisensory dynamic stimulations. Here, we investigate how BO sampled from individuals experiencing a transitory anxiety state, (a) induce a stress response and (b) bias the recognition of dynamic facial expressions, compared with BO of relaxed individuals. Participants ( n=46) categorized the emotion of a face, morphing from a neutral expression to either an angry or happy expression, during exposure to either BO condition. In addition, their cardiac activity was measured. Exposure to anxiety BO increased the accuracy of dynamic facial recognition and reduced cardiac parasympathetic activity. These results suggest that in social situations that simulate part of the multisensory and dynamic features of real-life social contexts, anxiety BOs will induce a stress response in recipients, modulating both arousal and cognitive-emotional skills but facilitating emotional facial processing.

Tracking the recognition of static and dynamic facial expressions of emotion across the life span.

The effective transmission and decoding of dynamic facial expressions of emotion is omnipresent and critical for adapted social interactions in everyday life. Thus, common intuition would suggest an advantage for dynamic facial expression recognition (FER) over the static snapshots routinely used in most experiments. However, although many studies reported an advantage in the recognition of dynamic over static expressions in clinical populations, results obtained from healthy participants are contrasted. To clarify this issue, we conducted a large cross-sectional study to investigate FER across the life span in order to determine if age is a critical factor to account for such discrepancies. More than 400 observers (age range 5-96) performed recognition tasks of the six basic expressions in static, dynamic, and shuffled (temporally randomized frames) conditions, normalized for the amount of energy sampled over time. We applied a Bayesian hierarchical step-linear model to capture the nonlinear relationship between age and FER for the different viewing conditions. Although replicating the typical accuracy profiles of FER, we determined the age at which peak efficiency was reached for each expression and found greater accuracy for most dynamic expressions across the life span. This advantage in the elderly population was driven by a significant decrease in performance for static images, which was twice as large as for the young adults. Our data posit the use of dynamic stimuli as being critical in the assessment of FER in the elderly population, inviting caution when drawing conclusions from the sole use of static face images to this aim.

Investigating the recognition of static and dynamic facial expressions of emotion in MCI patients

Investigating the recognition of static and dynamic facial expressions of emotion in MCI patients

Eye fixation patterns for categorizing static and dynamic facial expressions.

Facial expressions of emotion are dynamic in nature, but most studies on the visual strategies underlying the recognition of facial emotions have used static stimuli. The present study directly compared the visual strategies underlying the recognition of static and dynamic facial expressions using eye tracking and the Bubbles technique. The results revealed different eye fixation patterns with the 2 kinds of stimuli, with fewer fixations on the eye and mouth area during the recognition of dynamic than static expressions. However, these differences in eye fixations were not accompanied by any systematic differences in the facial information that was actually processed to recognize the expressions. (PsycINFO Database Record

Emotional recognition of dynamic facial expressions before and after cochlear implantation in adults with progressive deafness

Visual processing has been extensively explored in deaf subjects in the context of verbal communication, through the assessment of speech reading and sign language abilities. However, little is known about visual emotional processing in adult progressive deafness, and after cochlear implantation. The goal of our study was thus to assess the influence of acquired post-lingual progressive deafness on the recognition of dynamic facial emotions that were selected to express canonical fear, happiness, sadness, and anger. A total of 23 adults with post-lingual deafness separated into two groups; those assessed either before (n = 10) and those assessed after (n = 13) cochlear implantation (CI); and 13 normal hearing (NH) individuals participated in the current study. Participants were asked to rate the expression of the four cardinal emotions, and to evaluate both their emotional valence (unpleasant-pleasant) and arousal potential (relaxing-stimulating). We found that patients with deafness were impaired in the recognition of sad faces, and that patients equipped with a CI were additionally impaired in the recognition of happiness and fear (but not anger). Relative to controls, all patients with deafness showed a deficit in perceiving arousal expressed in faces, while valence ratings remained unaffected. The current results show for the first time that acquired and progressive deafness is associated with a reduction of emotional sensitivity to visual stimuli. This negative impact of progressive deafness on the perception of dynamic facial cues for emotion recognition contrasts with the proficiency of deaf subjects with and without CIs in processing visual speech cues (Rouger et al., 2007; Strelnikov et al., 2009; Lazard and Giraud, 2017). Altogether these results suggest there to be a trade-off between the processing of linguistic and non-linguistic visual stimuli.

Tracking the recognition of static and dynamic facial expressions of emotion across life span

Tracking the recognition of static and dynamic facial expressions of emotion across life span

Eye movements and spatial frequency utilization during the recognition of static and dynamic facial expressions

Eye movements and spatial frequency utilization during the recognition of static and dynamic facial expressions

Reduced recognition of dynamic facial emotional expressions and emotion-specific response bias in children with an autism spectrum disorder.

Emotion labelling was evaluated in two matched samples of 6-14-year old children with and without an autism spectrum disorder (ASD; N = 45 and N = 50, resp.), using six dynamic facial expressions. The Emotion Recognition Task proved to be valuable demonstrating subtle emotion recognition difficulties in ASD, as we showed a general poorer emotion recognition performance, in addition to some emotion-specific impairments in the ASD group. Participants' preference for selecting a certain emotion label, irrespective of the stimulus presented, played an important role in our results: response bias-corrected data still showed an overall decreased emotion recognition performance in ASD, but no emotion-specific impairments anymore. Moreover, ASD traits and empathy were correlated with emotion recognition performance.

Recognition of static and dynamic facial expressions: a study review

In emotion research, criticism has been directed to the use of static facial expressions, especially concerning its supposedly low ecological validity. In the present work, we performed a review of studies that directly compared the recognition of emotions using static and dynamic facial expressions. Behavioral, neuroimaging, brain damage and facial electromyography studies, published since 1993 were included. Overall, facial motion seems to promote emotional recognition. Neuroimaging and brain damage studies sustain the idea of a dissociation between the systems responsible to process static and dynamic expressions. Electromyography studies indicated that dynamic expressions tend to elicit more intense responses of facial mimic and are related to a higher physiological activation. Those findings support the hypothesis that dynamic facial expressions are ecologically more valid and therefore more appropriate to emotion research.

High-level after-effects in the recognition of dynamic facial expressions

High-level after-effects in the recognition of dynamic facial expressions

Recognition and discrimination of prototypical dynamic expressions of pain and emotions

Facial expressions of pain and emotions provide powerful social signals, which impart information about a person’s state. Unfortunately, research on pain and emotion expression has been conducted largely in parallel with few bridges allowing for direct comparison of the expressive displays and their impact on observers. Moreover, although facial expressions are highly dynamic, previous research has relied mainly on static photographs. Here we directly compare the recognition and discrimination of dynamic facial expressions of pain and basic emotions by naïve observers. One-second film clips were recorded in eight actors displaying neutral facial expressions and expressions of pain and the basic emotions of anger, disgust, fear, happiness, sadness and surprise. Results based on the Facial Action Coding System (FACS) confirmed the distinct (and prototypical) configuration of pain and basic emotion expressions reported in previous studies. Volunteers’ evaluations of those dynamic expressions on intensity, arousal and valence demonstrate the high sensitivity and specificity of the observers’ judgement. Additional rating data further suggest that, for comparable expression intensity, pain is perceived as more arousing and more unpleasant. This study strongly supports the claim that the facial expression of pain is distinct from the expression of basic emotions. This set of dynamic facial expressions provides unique material to explore the psychological and neurobiological processes underlying the perception of pain expression, its impact on the observer, and its role in the regulation of social behaviour.

Violent Media Consumption and the Recognition of Dynamic Facial Expressions

This study assessed the speed of recognition of facial emotional expressions (happy and angry) as a function of violent media consumption. Color photos of calm facial expressions morphed to either an angry or a happy facial expression. Participants were asked to make a speeded identification of the emotion (happiness or anger) during the morph. Results indicated that, independent of trait aggressiveness, participants high in violent media consumption responded slower to depictions of happiness and faster to depictions of anger than participants low in violent media consumption. Implications of these findings are discussed with respect to current models of aggressive behavior.