Processing Of Emotional Expressions Research Articles

Temporal Dynamics of Natural Static Emotional Facial Expressions Decoding: A Study Using Event- and Eye Fixation-Related Potentials.

This study aims at examining the precise temporal dynamics of the emotional facial decoding as it unfolds in the brain, according to the emotions displayed. To characterize this processing as it occurs in ecological settings, we focused on unconstrained visual explorations of natural emotional faces (i.e., free eye movements). The General Linear Model (GLM; Smith and Kutas, 2015a,b; Kristensen et al., 2017a) enables such a depiction. It allows deconvolving adjacent overlapping responses of the eye fixation-related potentials (EFRPs) elicited by the subsequent fixations and the event-related potentials (ERPs) elicited at the stimuli onset. Nineteen participants were displayed with spontaneous static facial expressions of emotions (Neutral, Disgust, Surprise, and Happiness) from the DynEmo database (Tcherkassof et al., 2013). Behavioral results on participants’ eye movements show that the usual diagnostic features in emotional decoding (eyes for negative facial displays and mouth for positive ones) are consistent with the literature. The impact of emotional category on both the ERPs and the EFRPs elicited by the free exploration of the emotional faces is observed upon the temporal dynamics of the emotional facial expression processing. Regarding the ERP at stimulus onset, there is a significant emotion-dependent modulation of the P2–P3 complex and LPP components’ amplitude at the left frontal site for the ERPs computed by averaging. Yet, the GLM reveals the impact of subsequent fixations on the ERPs time-locked on stimulus onset. Results are also in line with the valence hypothesis. The observed differences between the two estimation methods (Average vs. GLM) suggest the predominance of the right hemisphere at the stimulus onset and the implication of the left hemisphere in the processing of the information encoded by subsequent fixations. Concerning the first EFRP, the Lambda response and the P2 component are modulated by the emotion of surprise compared to the neutral emotion, suggesting an impact of high-level factors, in parieto-occipital sites. Moreover, no difference is observed on the second and subsequent EFRP. Taken together, the results stress the significant gain obtained in analyzing the EFRPs using the GLM method and pave the way toward efficient ecological emotional dynamic stimuli analyses.

Effects of Repeated Concussions and Sex on Early Processing of Emotional Facial Expressions as Revealed by Electrophysiology.

Concussions affect the processing of emotional stimuli. This study aimed to investigate how sex interacts with concussion effects on early event-related brain potentials (ERP) measures (P1, N1) of emotional facial expressions (EFE) processing in asymptomatic, multi-concussion athletes during an EFE identification task. Forty control athletes (20 females and 20 males) and 43 multi-concussed athletes (22 females and 21 males), recruited more than 3 months after their last concussion, were tested. Participants completed the Beck Depression Inventory II, the Beck Anxiety Inventory, the Post-Concussion Symptom Scale, and an Emotional Facial Expression Identification Task. Pictures of male and female faces expressing neutral, angry, and happy emotions were randomly presented and the emotion depicted had to be identified as fast as possible during EEG acquisition. Relative to controls, concussed athletes of both sex exhibited a significant suppression of P1 amplitude recorded from the dominant right hemisphere while performing the emotional face expression identification task. The present study also highlighted a sex-specific suppression of the N1 component amplitude after concussion which affected male athletes. These findings suggest that repeated concussions alter the typical pattern of right-hemisphere response dominance to EFE in early stages of EFE processing and that the neurophysiological mechanisms underlying the processing of emotional stimuli are distinctively affected across sex. (JINS, 2018, 24, 673-683).

Dynamic Interactions between Emotion Perception and Action Preparation for Reacting to Social Threat: A Combined cTBS-fMRI Study.

Expressions of emotion are powerful triggers for situation-appropriate responses by the observer. Of particular interest regarding the preparation of such adaptive actions are parietal and premotor cortices, given their potential for interaction with the amygdala (AMG), which is known to play a crucial role in the processing of affective information and in motor response. We set out to disentangle the respective roles of the inferior parietal lobule (IPL) and ventral premotor cortex (PMv) in humans in the processing of emotional body expressions by assessing remote effects of continuous theta burst stimulation (cTBS) in the action network and in AMG. Participants were presented with blocks of short videos showing either angry or neutral whole-body actions. The experiment consisted of three fMRI sessions: two sessions were preceded by stimulation of either right IPL (rIPL) or right PMv (rPMv); and a third session assessed baseline activity. Interestingly, whereas at baseline the left AMG did not differentiate between neutral and angry body postures, a significant difference between these conditions emerged after stimulation of either rIPL or rPMv, with much larger responses to angry than to neutral stimuli. In addition, the effects of cTBS stimulation and emotion were also observed in two other action-relevant areas, the supplementary motor area and the superior parietal cortex. Together, these results show how areas involved in action and emotion perception and in action preparation interact dynamically.

Time course of influence on the allocation of attentional resources caused by unconscious fearful faces

Emotionally affective stimuli have priority in our visual processing even in the absence of conscious processing. However, the influence of unconscious emotional stimuli on our attentional resources remains unclear. Using the continuous flash suppression (CFS) paradigm, we concurrently recorded and analyzed visual event-related potential (ERP) components evoked by the images of suppressed fearful and neutral faces, and the steady-state visual evoked potential (SSVEP) elicited by dynamic Mondrian pictures. Fearful faces, relative to neutral faces, elicited larger late ERP components on parietal electrodes, indicating emotional expression processing without consciousness. More importantly, the presentation of a suppressed fearful face in the CFS resulted in a significantly greater decrease in SSVEP amplitude which started about 1–1.2 s after the face images first appeared. This suggests that the time course of the attentional bias occurs at about 1 s after the appearance of the fearful face and demonstrates that unconscious fearful faces may influence attentional resource allocation. Moreover, we proposed a new method that could eliminate the interaction of ERPs and SSVEPs when recorded concurrently.

Affective theory of mind inferences contextually influence the recognition of emotional facial expressions

ABSTRACTThe recognition of emotional facial expressions is often subject to contextual influence, particularly when the face and the context convey similar emotions. We investigated whether spontaneous, incidental affective theory of mind inferences made while reading vignettes describing social situations would produce context effects on the identification of same-valenced emotions (Experiment 1) as well as differently-valenced emotions (Experiment 2) conveyed by subsequently presented faces. Crucially, we found an effect of context on reaction times in both experiments while, in line with previous work, we found evidence for a context effect on accuracy only in Experiment 1. This demonstrates that affective theory of mind inferences made at the pragmatic level of a text can automatically, contextually influence the perceptual processing of emotional facial expressions in a separate task even when those emotions are of a distinctive valence. Thus, our novel findings suggest that language acts as a contextual influence to the recognition of emotional facial expressions for both same and different valences.

A comparison for machine processing of emotional body language and facial expression

A comparison for machine processing of emotional body language and facial expression

Impaired Emotional Mirroring in Parkinson's Disease-A Study on Brain Activation during Processing of Facial Expressions.

Affective dysfunctions are common in patients with Parkinson's disease, but the underlying neurobiological deviations have rarely been examined. Parkinson's disease is characterized by a loss of dopamine neurons in the substantia nigra resulting in impairment of motor and non-motor basal ganglia-cortical loops. Concerning emotional deficits, some studies provide evidence for altered brain processing in limbic- and lateral-orbitofrontal gating loops. In a second line of evidence, human premotor and inferior parietal homologs of mirror neuron areas were involved in processing and understanding of emotional facial expressions. We examined deviations in brain activation during processing of facial expressions in patients and related these to emotion recognition accuracy. 13 patients and 13 healthy controls underwent an emotion recognition task and a functional magnetic resonance imaging (fMRI) measurement. In the Emotion Hexagon test, participants were presented with blends of two emotions and had to indicate which emotion best described the presented picture. Blended pictures with three levels of difficulty were included. During fMRI scanning, participants observed video clips depicting emotional, non-emotional, and neutral facial expressions or were asked to produce these facial expressions themselves. Patients performed slightly worse in the emotion recognition task, but only when judging the most ambiguous facial expressions. Both groups activated inferior frontal and anterior inferior parietal homologs of mirror neuron areas during observation and execution of the emotional facial expressions. During observation, responses in the pars opercularis of the right inferior frontal gyrus, in the bilateral inferior parietal lobule and in the bilateral supplementary motor cortex were decreased in patients. Furthermore, in patients, activation of the right anterior inferior parietal lobule was positively related to accuracy in the emotion recognition task. Our data provide evidence for a contribution of human homologs of monkey mirror areas to the emotion recognition deficit in Parkinson's disease.

The role of the cerebellum in explicit and incidental processing of facial emotional expressions: A study with transcranial magnetic stimulation

Growing evidence suggests that the cerebellum plays a critical role in non-motor functions, contributing to cognitive and affective processing. In particular, the cerebellum might represent an important node of the “limbic” network, underlying not only emotion regulation but also emotion perception and recognition. Here, we used transcranial magnetic stimulation (TMS) to shed further light on the role of the cerebellum in emotional perception by specifically testing cerebellar contribution to explicit and incidental emotional processing. In particular, in three different experiments, we found that TMS over the (left) cerebellum impaired participants’ ability to categorize facial emotional expressions (explicit task) and to classify the gender of emotional faces (incidental emotional processing task), but not the gender of neutral faces. Overall, our results indicate that the cerebellum is involved in perceiving the emotional content of facial stimuli, even when this is task irrelevant.

Impaired mixed emotion processing in the right ventrolateral prefrontal cortex in schizophrenia: an fMRI study

BackgroundSchizophrenia has a negative effect on the activity of the temporal and prefrontal cortices in the processing of emotional facial expressions. However no previous research focused on the evaluation of mixed emotions in schizophrenia, albeit they are frequently expressed in everyday situations and negative emotions are frequently expressed by mixed facial expressions.MethodsAltogether 37 subjects, 19 patients with schizophrenia and 18 healthy control subjects were enrolled in the study. The two study groups did not differ in age and education. The stimulus set consisted of 10 fearful (100%), 10 happy (100%), 10 mixed fear (70% fear and 30% happy) and 10 mixed happy facial expressions. During the fMRI acquisition pictures were presented in a randomized order and subjects had to categorize expressions by button press.ResultsA decreased activation was found in the patient group during fear, mixed fear and mixed happy processing in the right ventrolateral prefrontal cortex (VLPFC) and the right anterior insula (RAI) at voxel and cluster level after familywise error correction. No difference was found between study groups in activations to happy facial condition. Patients with schizophrenia did not show a differential activation between mixed happy and happy facial expression similar to controls in the right dorsolateral prefrontal cortex (DLPFC).ConclusionsPatients with schizophrenia showed decreased functioning in right prefrontal regions responsible for salience signaling and valence evaluation during emotion recognition. Our results indicate that fear and mixed happy/fear processing are impaired in schizophrenia, while happy facial expression processing is relatively intact.

Association between Neuroticism and Emotional Face Processing

Neuroticism is one of the “Big Five” personality factors and is characterized by a tendency to experience negative affect. We aimed to investigate how neuroticism influences the neural correlates for processing of emotional facial expressions. 68 healthy participants were presented with emotional dynamic facial stimuli, i.e. happy, neutral or angry, during functional MRI. Brain activations for the contrasts emotional vs. neutral, happy vs. neutral and angry vs. neutral were correlated with individuals’ neuroticism scores as obtained by the NEO Five Factor Inventory questionnaire and additionally investigated for gender differences. The bilateral medial temporal gyrus (MTG) was identified as key region in the processing of emotional faces and activations within this region correlated with individual neuroticism scores. Although female participants showed significantly stronger activation differences between emotional and neutral facial expressions in the left MTG, the correlation between activation and neuroticism scores did not show any significant gender differences. Our results offer for the first time a biological correlate within the face processing network for enhanced reactivity of neurotic individuals to emotional facial expressions which occurs similarly for both male and female participants.

Neurophysiological differentiation between preattentive and attentive processing of emotional expressions on French vowels

The present electrophysiological study investigated the processing of emotional prosody by minimizing as much as possible the effect of emotional information conveyed by the lexical-semantic context. Emotionally colored French vowels (i.e., happiness, sadness, fear, and neutral) were presented in a mismatch negativity (MMN) oddball paradigm. Both the MMN, i.e., an event-related potential (ERP) component thought to reflect preattentive change detection, and the P3a, i.e., an ERP marker of involuntary orientation of attention toward deviant stimuli, were significantly modulated by the emotional deviants compared to the neutral ones. Critically, the largest amplitude (MMN, P3a) and the shortest peak latency (MMN) were observed for fear deviants, all other things being equal. Taken together, the present findings lend support to a sequential neurocognitive model of emotion processing (Scherer, 2001) which postulates, among other checks, a first stage of automatic emotion detection (MMN) followed by a second stage of subjective evaluation of the stimulus or event (P3a). Consistently with previous studies, our data suggest that among the six universal emotions, fear could have a special status probably because of its adaptive role in the evolution of the human species.

Minimal group membership biases early neural processing of emotional expressions.

Mere affiliation with a social group alters people's perception of other individuals. One suggested mechanism behind such influence is that group membership triggers divergent visual facial representations for in-group and out-group members, which could constrain face processing. Here, using electroencephalography (EEG) under functional magnetic resonance imagery (fMRI) during a group categorization task, we investigated the impact of mere affiliation to an arbitrary group on the processing of emotional faces. The results indicate that in- and out-group members trigger differential event-related potential activity, appearing 150ms after presentation of group membership information, which correlated with medial prefrontal fMRI activity. Additionally, EEG activity in the earliest stages of face processing (30-100ms after expression onset) dissociated unexpected group-related emotions (in-group anger and out-group joy) from expected ones and correlated with temporo-parietal junction fMRI activity. We discuss the possibility that such dissociation may result from top-down influences from divergent representations for in-group and out-group members. Taken together, the present results suggest that mere membership in an arbitrary group polarized expectations which constrain the first steps of face processing.

Automatic processing of emotional facial expressions as a function of social anhedonia

Anhedonia is an important feature of major depression and schizophrenia-spectrum disorders. Few neuroimaging studies have investigated neural alterations in high anhedonia, isolated from other psychopathological variables, by including only participants without clinical diagnoses. The present study examined healthy individuals scoring high (N = 18) vs. low (N = 19) in social anhedonia, who were carefully selected from a sample of N = 282 participants. To examine differences in automatic brain responses to social-affective stimuli between high vs. low social anhedonia participants, we used functional magnetic resonance imaging. To assess early, automatic stages of emotion processing, we administered a paradigm presenting brief (33ms), backward-masked happy, sad, and neutral facial expressions. Individuals high in social anhedonia demonstrated increased activation in the bilateral thalamus and left red nucleus in response to masked sad faces relative to individuals low in social anhedonia. No significant group differences in brain activation emerged in other regions known to be involved in emotion and reward processing, including the amygdala and nucleus accumbens. Our results suggest that high social anhedonia in otherwise healthy individuals is associated with exaggerated automatic reactivity in the thalamus, which is a brain structure that has been implicated in the mediation of attentional processes.

Facial emotion recognition deficits in children with and without attention deficit hyperactivity disorder: a behavioral and neurophysiological approach.

The current study examined the facial emotion recognition ability with a simultaneous assessment of behavioral and neurophysiological data in children with and without attention deficit hyperactivity disorder (ADHD) aged 7-17 years using a facial emotion matching task and event-related potential measurements (event-related potential components N170 and N250 at T5 and T6) in an emotional continuous performance task. Group differences and interaction effects of children's performance (both behavioral and neurophysiological) were evaluated between children with ADHD and children without ADHD as well as between younger and older children. No deficit in facial emotion recognition was found for children with ADHD compared with children without ADHD even with neurophysiological parameters. However, in terms of developmental differences, the younger children differentiated in their behavioral and neurophysiological performance from the older children. No interaction was detected between the experimental groups and the age groups, indicating that developmental progression in terms of emotional processes did not differ between children with and without ADHD. This study indicates that the facial emotion recognition is above all an age-dependent function with later processing of facial emotion expressions in younger compared with older children and suggests that a facial emotion recognition deficit is secondary in children with ADHD and might occur only with specific emotions or ADHD subtypes, but not in the whole ADHD population.

Processing of Facial Expressions in Autism: a Systematic Review of EEG/ERP Evidence

The aim of this systematic review is to examine electrophysiological differences in emotional facial expression processing in autism spectrum disorder (ASD). We identified 55 articles using PubMed. After application of the rejection criteria, in a two-step procedure, based on abstract and full-text reading, we selected 16 studies for review. We identified group differences and emotion by group interactions in most of the studies during facial emotion-processing tasks. Nine articles reported main effects of facial expression type. Smaller amplitudes and delayed latencies were consistently reported to emotional expressions in ASD subjects compared to controls. Right hemisphere dominance was found in both groups. Our systematic review confirms facial emotion-processing deficits in ASD, which are often linked with emotion recognition deficits.

Processing of Unattended Emotional Facial Expressions: Correlates of Visual Field Bias in Women.

Lateralization in emotional processing is a matter of ongoing debate. Various factors can influence lateralized emotional processing, including stimulus location, emotional valence, and gender. In the present study, we aim to elucidate how unattended emotional facial expressions shown at different locations in the visual field influence behavioral responses, eye movement, and neural responses in a sample of healthy women. Our female participants viewed fearful, happy and neutral faces presented at central and peripheral (left or right) locations while keeping their gaze locked on a central fixation crosshairs and indicating stimulus location via button presses. Throughout the experiment, we monitored fixation and gaze shifts by means of eye tracking. We analyzed eye movements, neural and behavioral responses from n = 18 participants with excellent tracking and task performance. Face stimuli presented in the left hemifield entailed the fastest reactions irrespective of face valence. Unwarranted gaze shifts away from central fixation were rare and mainly directed at peripherally presented stimuli. A distributed neural network comprising the right amygdala, left temporal pole, left middle temporal gyrus, right superior frontal gyrus, and right posterior putamen differentially responded to centrally presented fearful faces, and to peripherally presented neutral and happy faces, especially when they appeared in the left hemifield. Our findings point to a visual field bias on the behavioral and neural level in our female sample. Reaction times, eye movements and neural activations varied according to stimulus location. An interactive effect of face location with face valence was present at the neural level but did not translate to behavioral or eye movement responses.

Sensing emotion in voices: Negativity bias and gender differences in a validation study of the Oxford Vocal ('OxVoc') sounds database.

Emotional expressions are an essential element of human interactions. Recent work has increasingly recognized that emotional vocalizations can color and shape interactions between individuals. Here we present data on the psychometric properties of a recently developed database of authentic nonlinguistic emotional vocalizations from human adults and infants (the Oxford Vocal ‘OxVoc’ Sounds Database; Parsons, Young, Craske, Stein, & Kringelbach, 2014). In a large sample (n = 562), we demonstrate that adults can reliably categorize these sounds (as ‘positive,’ ‘negative,’ or ‘sounds with no emotion’), and rate valence in these sounds consistently over time. In an extended sample (n = 945, including the initial n = 562), we also investigated a number of individual difference factors in relation to valence ratings of these vocalizations. Results demonstrated small but significant effects of (a) symptoms of depression and anxiety with more negative ratings of adult neutral vocalizations (R2 = .011 and R2 = .008, respectively) and (b) gender differences in perceived valence such that female listeners rated adult neutral vocalizations more positively and infant cry vocalizations more negatively than male listeners (R2 = .021, R2 = .010, respectively). Of note, we did not find evidence of negativity bias among other affective vocalizations or gender differences in perceived valence of adult laughter, adult cries, infant laughter, or infant neutral vocalizations. Together, these findings largely converge with factors previously shown to impact processing of emotional facial expressions, suggesting a modality-independent impact of depression, anxiety, and listener gender, particularly among vocalizations with more ambiguous valence.

The Tromso Infant Faces Database (TIF): Development, Validation and Application to Assess Parenting Experience on Clarity and Intensity Ratings

Newborns and infants are highly depending on successfully communicating their needs; e.g., through crying and facial expressions. Although there is a growing interest in the mechanisms of and possible influences on the recognition of facial expressions in infants, heretofore there exists no validated database of emotional infant faces. In the present article we introduce a standardized and freely available face database containing Caucasian infant face images from 18 infants 4 to 12 months old. The development and validation of the Tromsø Infant Faces (TIF) database is presented in Study 1. Over 700 adults categorized the photographs by seven emotion categories (happy, sad, disgusted, angry, afraid, surprised, neutral) and rated intensity, clarity and their valance. In order to examine the relevance of TIF, we then present its first application in Study 2, investigating differences in emotion recognition across different stages of parenthood. We found a small gender effect in terms of women giving higher intensity and clarity ratings than men. Moreover, parents of young children rated the images as clearer than all the other groups, and parents rated “neutral” expressions as more clearly and more intense. Our results suggest that caretaking experience provides an implicit advantage in the processing of emotional expressions in infant faces, especially for the more difficult, ambiguous expressions.

How does gaze direction affect facial processing in social anxiety? —An ERP study

Previous behavioral studies have demonstrated an effect of eye gaze direction on the processing of emotional expressions in adults with social anxiety. However, specific brain responses to the interaction between gaze direction and facial expressions in social anxiety remain unclear. The present study aimed to explore the time course of such interaction using event-related potentials (ERPs) in participants with social anxiety. High socially anxious individuals and low socially anxious individuals were asked to identify the gender of angry or neutral faces with direct or averted gaze while their behavioral performance and electrophysiological data were monitored. We found that identification of angry faces with direct but not averted gaze elicited larger N2 amplitude in high socially anxious individuals compared to low socially anxious individuals, while identification of neutral faces did not produce any gaze modulation effect. Moreover, the N2 was correlated with increased anxiety severity upon exposure to angry faces with direct gaze. Therefore, our results suggest that gaze direction modulates the processing of threatening faces in social anxiety. The N2 component elicited by angry faces with direct gaze could be a state-dependent biomarker of social anxiety and may be an important reference biomarker for social anxiety diagnosis and intervention.

Are event-related potentials to dynamic facial expressions of emotion related to individual differences in the accuracy of processing facial expressions and identity?

Despite a wealth of knowledge about the neural mechanisms behind emotional facial expression processing, little is known about how they relate to individual differences in social cognition abilities. We studied individual differences in the event-related potentials (ERPs) elicited by dynamic facial expressions. First, we assessed the latent structure of the ERPs, reflecting structural face processing in the N170, and the allocation of processing resources and reflexive attention to emotionally salient stimuli, in the early posterior negativity (EPN) and the late positive complex (LPC). Then we estimated brain-behavior relationships between the ERP factors and behavioral indicators of facial identity and emotion-processing abilities. Structural models revealed that the participants who formed faster structural representations of neutral faces (i.e., shorter N170 latencies) performed better at face perception (r = -.51) and memory (r = -.42). The N170 amplitude was not related to individual differences in face cognition or emotion processing. The latent EPN factor correlated with emotion perception (r = .47) and memory (r = .32), and also with face perception abilities (r = .41). Interestingly, the latent factor representing the difference in EPN amplitudes between the two neutral control conditions (chewing and blinking movements) also correlated with emotion perception (r = .51), highlighting the importance of tracking facial changes in the perception of emotional facial expressions. The LPC factor for negative expressions correlated with the memory for emotional facial expressions. The links revealed between the latency and strength of activations of brain systems and individual differences in processing socio-emotional information provide new insights into the brain mechanisms involved in social communication.