Processing Of Facial Expressions Research Articles

Probing the processing of facial expressions in monkeys via time perception and eye tracking.

Accurately recognizing facial expressions is essential for effective social interactions. Non-human primates (NHPs) are widely used in the study of the neural mechanisms underpinning facial expression processing, yet it remains unclear how well monkeys can recognize the facial expressions of other species such as humans. In this study, we systematically investigated how monkeys process the facial expressions of conspecifics and humans using eye-tracking technology and sophisticated behavioral tasks, namely the temporal discrimination task (TDT) and face scan task (FST). We found that monkeys showed prolonged subjective time perception in response to Negative facial expressions in monkeys while showing longer reaction time to Negative facial expressions in humans. Monkey faces also reliably induced divergent pupil contraction in response to different expressions, while human faces and scrambled monkey faces did not. Furthermore, viewing patterns in the FST indicated that monkeys only showed bias toward emotional expressions upon observing monkey faces. Finally, masking the eye region marginally decreased the viewing duration for monkey faces but not for human faces. By probing facial expression processing in monkeys, our study demonstrates that monkeys are more sensitive to the facial expressions of conspecifics than those of humans, thus shedding new light on inter-species communication through facial expressions between NHPs and humans.

ДИНАМІКА ЕЕГ ПІД ЧАС СПРИЙНЯТТЯ ЕМОЦІЙНИХ ВИРАЗІВ ОБЛИЧ ЗА РІЗНОГО ЧАСУ ДЕМОНСТРАЦІЇ: ДОСЛІДЖЕННЯ ПОТЕНЦІАЛІВ, ПОВ'ЯЗАНИХ ІЗ ПОДІЯМИ

The face is one of the most significant elements of non-verbal communication, which allows humans to quickly get various types of information about a specific person. Therefore, understanding facial expressions is crucial for normal social interaction, as it lets us determine the emotional states and intentions of other people. The purpose of this study was to investigate the influence of the time factor on the happy and fearful facial expression processing, reflected in the structure and components of event-related potentials (ERPs). Accordingly, two parallel examinations were conducted with stimulus presentation periods of 500 ms and 100 ms. For the obtained data, two separate analyses (for happy and fearful faces) of the averaged ERP curves corresponding to different stimulus exposure periods were executed. The initial stages of processing (EPN, P3) demonstrated that the difference in the structure and amplitude characteristics of the ERP curve was not that significant, but it demonstrated an increase in attentional resources involvement and a strengthening of the primary emotional analysis. Later ERP components (N400, LPP) showed a more compelling difference, that reflected the complexity of semantic decoding, subsequent conscious evaluation, and processes related to decision-making. Thus, reducing the time of image display from 500 ms to 100 ms created the expected difficulties for the perception and processing of emotional facial expressions and was not sufficient for the normal course of these processes.

Positive and negative facial valence perception are modulated differently by eccentricity in the parafovea

Understanding whether people around us are in a good, bad or neutral mood can be critical to our behavior, both when looking directly at them or when they are in our peripheral visual field. However, facial expressions of emotions are often investigated at central visual field or at locations right or left of fixation. Here we assumed that perception of facial emotional valence (the emotion’s pleasantness) changes with distance from central visual field (eccentricity) and that different emotions may be influenced differently by eccentricity. Participants (n = 58) judged the valence of emotional faces across the parafovea (≤ 4°, positive (happy), negative (fearful), or neutral)) while their eyes were being tracked. As expected, performance decreased with eccentricity. Positive valence perception was least affected by eccentricity (accuracy reduction of 10–19% at 4°) and negative the most (accuracy reduction of 35–38% at 4°), and this was not a result of speed-accuracy trade-off or response biases. Within-valence (but not across-valence) performance was associated across eccentricities suggesting perception of different valences is supported by different mechanisms. While our results may not generalize to all positive and negative emotions, they indicate that beyond-foveal investigations can reveal additional characteristics of the mechanisms that underlie facial expression processing and perception.

Expectation modulates the preferential processing of task-irrelevant fear in the attentional blink: evidence from event-related potentials

BackgroundReporting the second of the two targets is impaired when it occurs 200–500 ms after the first, the phenomenon in the study of consciousness is the attentional blink (AB). In the AB task, both the emotional salience and the expectation of the second target increase the likelihood of that target being consciously reported. Yet, little is known about how expectations modulate the prioritized processing of affective stimuli. We examined the role of expecting fearful expression when processing fear in an AB task. Participants were presented with an AB task where the 2nd target (T2) is either a fearful face or a neutral face, and had to report the target's gender. The frequency of fearful to neutral faces on a given block was manipulated, such that participants could either expect more or less fearful faces.ResultsIn the Experiment 1, we found that fearful faces were more likely to be recognized than neutral faces during the blink period (lag3) when participants were not expecting a fearful face (low fear-expectation); however, high fear-expectation increased the discrimination of fearful T2 than neutral T2 outside the blink period (lag8). In the Experiment 2, we assessed ERP brain activity in response to perceived T2 during the blink period. The results revealed that fearful faces elicited larger P300 amplitudes compared to neutral faces, but only in the low fear-expectation condition, suggesting that expecting a fearful expression can suppress the processing of task-irrelevant facial expression and unexpected fearful expression can break through this suppression. Fearful T2 elicited larger vertex positive potential (VPP) amplitudes than neutral T2, and this affective effect was independent of fear-expectation. Since no effect of expectation was found on the VPP amplitude while P300 exhibited significant interaction between expectation and expression, this suggests that expectations modulate emotional processing at a later stage, after the fearful face has been differentially processed.ConclusionsThese results provided clear evidence for the contribution of the expectation to the prioritized processing of second affective stimuli in the AB.

Temporal Dynamics of Positive and Negative Facial Expression Processing

Temporal Dynamics of Positive and Negative Facial Expression Processing

The relationship between ensemble representations of facial information

People accurately evaluate various types of facial information (gaze direction, facial expression, facial identity, and gender) of multiple faces. Considering such varieties, summarizing abilities of facial information might vary depending on its type because it is either changeable (e.g., gaze direction and expression) or invariant (e.g., identity and gender). The current study investigated the relationship between the averaging abilities of facial information using an individual difference approach and a dual-task paradigm to understand the effect of facial information type on the ensemble coding of facial information. We conducted two online experiments on the relationship between the averaging abilities of facial expressions and gaze direction (Experiment 1) and those of facial expressions and gender (Experiment 2). Participants were asked to estimate the average of each piece of facial information in the first and second blocks (single task), respectively, and both sequentially in the third and fourth blocks (dual task). We found that most of the error autocorrelations of facial information were high, indicating high measurement reliability. Participants’ abilities to average facial expressions were correlated with those to average gaze directions, but not with those to average gender information. That is, the ensemble processing of facial expressions is related to gaze directions, but not genders. These results suggest that ensemble representations of facial information regarding changeable properties differ from those of invariant ones.

Mask-wearing selectivity alters observers’ face perception

Face masks became prevalent across the globe as an efficient tool to stop the spread of COVID-19. A host of studies already demonstrated that masks lead to changes in facial identification and emotional expression processing. These changes were documented across ages and were consistent even with the increased exposure to masked faces. Notably, mask-wearing also changes the state of the observers in regard to their own bodies and other agents. Previous research has already demonstrated a plausible association between observers’ states and their perceptual behaviors. Thus, an outstanding question is whether mask-wearing would alter face recognition abilities. To address this question, we conducted a set of experiments in which participants were asked to recognize non-masked faces (Experiment 1), masked faces (Experiment 2) and novel objects (Experiment 3) while they were either masked or unmasked. Mask wearing hindered face perception abilities but did not modulate object recognition ability. Finally, we demonstrated that the decrement in face perception ability relied on wearing the mask on distinctive facial features (Experiment 4). Together, these findings reveal a novel effect of mask-wearing on face recognition. We discuss these results considering the plausible effect of somatosensory stimulation on visual processing as well as the effect of involuntary perspective taking.

Event-related potential response to drivers' facial expressions in an online car-hailing scene.

Recognizing facial expressions is crucial for adaptive social interaction. Prior empirical research on facial expression processing has primarily focused on isolated faces; however, facial expressions appear embedded in surrounding scenes in everyday life. In this study, we attempted to demonstrate how the online car-hailing scene affects the processing of facial expression. This study examined the processing of drivers' facial expressions in scenes by recording event-related potentials, in which neutral or happy faces embedded in online car-hailing orders were constructed (with type of vehicle, driver rating, driver surname, and level of reputation controlled). A total of 35 female volunteers participated in this experiment and were asked to judge which facial expressions that emerged in scenes of online car-hailing were more trustworthy. The results revealed an interaction between facial expression scenes, brain areas, and electrode sites in the late positive potential, which indicated that happy faces elicited larger amplitudes than did neutral ones in the parietal areas and that scenes with happy facial expressions had shorter latencies than did those with neutral ones. As expected, the late positive potential evoked by happy facial expressions in a scene was larger than that evoked by neutral ones, which reflected motivated attention and motivational response processes. This study highlights the importance of scenes as context in the study of facial expression processing.

Impact of mask-wearing on child and adolescent psychosocial development: a systematic review

Abstract Background Wearing face masks in public is recommended under certain circumstances in order to prevent infectious diseases transmitted through droplets. The objective was to compile all German and English research results from peer-reviewed journal articles using a sensitive literature search on the effects of mask-wearing for preventing infectious diseases on the psychosocial development of children and adolescents. Methods A systematic review was conducted considering different study designs (search period up until 12 July 2021). The risk of bias in the studies was determined using a risk of bias procedure. A descriptive-narrative synthesis of the results was performed. A search update will be conducted shortly before the conference. Results Thirteen studies were included, and the overall risk of bias was estimated to be high in all primary studies. There are some indications from the included surveys that children, adolescents, and their teachers in (pre)schools perceived facial expression processing as impaired due to mask-wearing, which were confirmed by several experimental studies. Two studies reported psychological symptoms like anxiety and stress as well as concentration and learning problems due to wearing a mask during the COVID-19 pandemic. One survey study during the 2002/2003 SARS pandemic examined oral examination performance in English as a foreign language and showed no difference between the “mask” and “no mask” conditions. The results are preliminary and may be extended due to the search update. Conclusions Only little evidence can be derived on the effects of face masks on different developmental areas of children and adolescents based on the small number of studies. There is a lack of research data regarding the following outcomes: psychological development, language development, emotional development, social behaviour, school success, and participation. Further cluster-randomized controlled trials or longitudinal studies are required. Key messages • Empirical and experimental evidence shows that mask-wearing impairs facial expression among children, adolescents, and teachers in (pre)schools. • Research on the following developmental areas is missing: psychological development, language development, emotional development, social behaviour, school success, and participation.

Affective touch during the processing of angry facial expressions: an event-related potential study

It has been shown that affective touch can have stress-buffering effects. The current event-related potential (ERP) study investigated whether affective touch can reduce emotional distress and associated late positivity while viewing angry facial expressions. A total of 122 females (mean age = 23 years) were randomly assigned to one of three groups to either receive slow/soft brushing of their forearm (affective touch), fast brushing (nonaffective touch), or no touch while viewing images depicting angry and neutral facial expressions. The participants rated their affective state (valence, arousal) before and after the experiment. They also rated the perceived intensity of the angry facial expressions and the pleasantness of touch during the experiment. Components of the Late Positive Potential (LPP) in response to the images that are associated with stimulus significance, attention allocation, and emotion regulation (early LPP: 400–1000 ms; late LPP: 1000–3000 ms) were extracted for a frontal and a centroparietal cluster. Affective touch was associated with reduced amplitudes of the late LPP in the frontal cluster but did not affect centroparietal LPPs (early, late). Affective touch was rated as more pleasant than nonaffective touch but did not change reported valence, arousal, and perceived anger intensity. In conclusion, affective touch modulated a neural indicator of stimulus significance but did not influence self-report measures. More naturalistic touch settings might enhance the effects.

Enhanced mirror neuron network activity and effective connectivity during live interaction among female subjects

Facial expressions are indispensable in daily human communication. Previous neuroimaging studies investigating facial expression processing have presented pre-recorded stimuli and lacked live face-to-face interaction. Our paradigm alternated between presentations of real-time model performance and pre-recorded videos of dynamic facial expressions to participants. Simultaneous functional magnetic resonance imaging (fMRI) and facial electromyography activity recordings, as well as post-scan valence and arousal ratings were acquired from 44 female participants. Live facial expressions enhanced the subjective valence and arousal ratings as well as facial muscular responses. Live performances showed greater engagement of the right posterior superior temporal sulcus (pSTS), right inferior frontal gyrus (IFG), right amygdala and right fusiform gyrus, and modulated the effective connectivity within the right mirror neuron system (IFG, pSTS, and right inferior parietal lobule). A support vector machine algorithm could classify multivoxel activation patterns in brain regions involved in dynamic facial expression processing in the mentalizing networks (anterior and posterior cingulate cortex). These results indicate that live social interaction modulates the activity and connectivity of the right mirror neuron system and enhances spontaneous mimicry, further facilitating emotional contagion.

Neural responses to social partners' facial expressions are modulated by their social status in an interactive situation

Recently, several studies have found a recognition advantage for facial expressions, particularly angry expressions, when they appear on high-status faces rather than low-status faces. In the present study, we used event-related potentials (ERPs) to explore the influence of social status on the neural responses to others' facial expressions in a context of performance monitoring. Specifically, we used an interactive rank-inducing task (i.e., time estimation task) to manipulate social partners' status (high versus low) and then told participants that they would receive social feedback (i.e., happy, neutral, or angry) from social partners with high or low status after completing the task. ERP results revealed the preferential processing of high-status targets at both early (P1/N170/FRN) and late (P3) temporal stages of facial expression processing. Notably, larger FRN amplitudes elicited by feedback from high-status partners were observed in happy, neutral, and angry expression contexts, whereas larger P3 amplitudes elicited by feedback from high-status partners were only evident in both neutral and angry expression contexts but not in happy expression context. Together, the present study extended previous studies by showing that the perception of facial expressions could be modulated by target status at multiple stages.

Degenerate pathway for processing smile and other emotional expressions in congenital facial palsy: an hdEEG investigation.

Influential theoretical models argue that an internal simulation mechanism (motor or sensorimotor simulation) supports the recognition of facial expressions. However, despite numerous converging sources of evidence, recent studies testing patients with congenital facial palsy (i.e. Moebius syndrome) seem to refute these theoretical models. However, these results do not consider the principles of neuroplasticity and degeneracy that could support the involvement of an alternative neural processing pathway in these patients. In the present study, we tested healthy participants and participants with Moebius syndrome in a highly sensitive facial expression discrimination task and concomitant high-density electroencephalographic recording. The results, both at the scalp and source levels, indicate the activation of two different pathways of facial expression processing in healthy participants and participants with Moebius syndrome, compatible, respectively, with a dorsal pathway that includes premotor areas and a ventral pathway. Therefore, these results support the reactivation of sensorimotor representations of facial expressions (i.e. simulation) in healthy subjects, in the place of an alternative processing pathway in subjects with congenital facial palsy.This article is part of the theme issue ‘Cracking the laugh code: laughter through the lens of biology, psychology and neuroscience’.

Neural response and representation: Facial expressions in scenes.

Previous studies have shown that the brain generates expectations based on scenes, which affects facial expression recognition. However, although facial expressions are known to interact with perception, the mechanism underlying this interaction remains poorly understood. Here, we used frequency labeling and decoding techniques to reveal the effects of scene-based expectation on the amplitude and representational strength of neural activity. We also reduced the relative reliability between expectation and sensory input by blurring facial expressions to further investigate the effects of this relative reliability on the pattern of neural activation and representation. Participants viewed emotional changes in unblurred or blurred facial expressions, which flickered at a rate of 6Hz within a scene. We found that facial expressions that were congruent with the emotional significance of the scene elicited a larger steady-state visual evoked potential amplitude than did facial expressions that were incongruent with the emotional significance of a scene, in both unblurred and blurred conditions. We also found that expected facial expression representations were stronger than unexpected representations during the unblurred condition. In the blurred condition, unexpected representations were stronger than expected representations. Taken together, these results suggested that facial expression processing in the visual cortex is modulated by top-down signals. The relative reliability of expectation and sensory input moderated the influence of a scene on facial expression representation. Furthermore, our study showed that neural activation amplitudes did not correspond to representational strength.

Representational structure of fMRI/EEG responses to dynamic facial expressions

Face perception provides an excellent example of how the brain processes nuanced visual differences and transforms them into behaviourally useful representations of identities and emotional expressions. While a body of literature has looked into the spatial and temporal neural processing of facial expressions, few studies have used a dimensionally varying set of stimuli containing subtle perceptual changes. In the current study, we used 48 short videos varying dimensionally in their intensity and category (happy, angry, surprised) of expression. We measured both fMRI and EEG responses to these video clips and compared the neural response patterns to the predictions of models based on image features and models derived from behavioural ratings of the stimuli. In fMRI, the inferior frontal gyrus face area (IFG-FA) carried information related only to the intensity of the expression, independent of image-based models. The superior temporal sulcus (STS), inferior temporal (IT) and lateral occipital (LO) areas contained information about both expression category and intensity. In the EEG, the coding of expression category and low-level image features were most pronounced at around 400 ms. The expression intensity model did not, however, correlate significantly at any EEG timepoint. Our results show a specific role for IFG-FA in the coding of expressions and suggest that it contains image and category invariant representations of expression intensity.

The role of discriminability in face perception: Interference processing of expression, gender, and gaze.

Eye gaze plays a fundamental role in social interaction and facial recognition. However, interference processing between gaze and other facial variants (e.g., expression) and invariant information (e.g., gender) remains controversial and unclear, especially the role of facial information discriminability in interference. A Garner paradigm was used to conduct two experiments. This paradigm allows simultaneous investigation of the mutual influence of two kinds of facial information in one experiment. In Experiment 1, we manipulated facial expression discriminability and investigated its role in interference processing of gaze and facial expression. The results show that individuals were unable to ignore expression when classifying gaze with both high and low discriminability but could ignore gaze when classifying expression with high discriminability only. In Experiment 2, we manipulated gender discriminability and investigated its function in interference processing of gaze and gender. Participants were unable to ignore gender when classifying gaze with both high and low discriminability but could ignore gaze when classifying gender with low discriminability only. The results indicate that gaze categorization is affected by facial expression and gender regardless of facial information discriminability, whereas interference of gaze on facial expression and gender depends on the degree of discriminability. The present study provides evidence that the processing of gaze and other variant and invariant information is interdependent.

Mirror neurons and empathy-related regions in psychopathy: Systematic review, meta-analysis, and a working model

ABSTRACT Mirror neurons have been associated with empathy. People with psychopathic traits present low levels of empathy. To analyze this, a systematic review of fMRI studies of people with psychopathic traits during an emotional facial expression processing task was performed. The regions of interest were structures associated with the mirror neuron system: ventromedial prefrontal cortex (vmPFC), inferior parietal lobe (IPL), inferior frontal gyrus and superior temporal sulcus. The analysis was also extended to structures related to affective empathy (insula, amygdala and anterior cingulate cortex) and to two more emotional processing areas (orbitofrontal cortex and fusiform gyrus). Hypoactivation was more frequently observed in regions of the mirror neuron system from people with high psychopathic traits, as well as in the emotional processing structures, and those associated with affective empathy, except for the insula, where it presented higher activity. Differences were observed for all types of emotions. The results suggest that the mirror neuron system is altered in psychopathy and their relationship with affective empathy deficits is discussed.

Infant fecal microbiota composition and attention to emotional faces.

The gut microbiota has been suggested to influence neurodevelopment in rodents. Preliminary human studies have associated fecal microbiota composition with features of emotional and cognitive development as well as differences in thalamus-amygdala connectivity. Currently, microbiota-gut-brain axis studies cover heterogenous set of infant and child brain developmental phenotypes, while microbiota associations with more fine-grained aspects of brain development remain largely unknown. Here (N = 122, 53% boys), we investigated the associations between infant fecal microbiota composition and infant attention to emotional faces, as bias for faces is strong in infancy and deviations in early processing of emotional facial expressions may influence the trajectories of social-emotional development. The fecal microbiota composition was assessed at 2.5 months of age and analyzed with 16S rRNA gene sequencing. Attention to emotional faces was assessed with an age-appropriate face-distractor paradigm, using neutral, happy, fearful, and scrambled faces and salient distractors, at 8 months of age. We observed an association between a lower abundance of Bifidobacterium and a higher abundance of Clostridium with an increased "fear bias," that is, attention toward fearful versus happy/neutral faces. This data suggests an association between early microbiota and later fear bias, a well-established infant phenotype of emotionally directed attention. However, the clinical significance or causality of our findings remains to be assessed. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Processing of Facial Expressions of Emotions and Pain in Alzheimer’s Disease

Alzheimer's disease (AD) is a neurodegenerative pathology that disrupts processing of facial expressions of emotion. The impairment was demonstrated for negative emotions in tasks of matching, discriminating, and labeling facial expressions but no study has included the expression of pain in its protocol. The objective was to study the processing of emotional facial expressions in AD with a particular interest in pain expression. Twenty-seven controls, 15 mild AD patients, and 15 moderate AD patients had to perform four emotional tasks: identification of facial expressions, matching pain expressions, discriminating the intensity of pain expressions, and judging pain intensity. Some emotions were less efficiently recognized by AD patients compared to controls (p < 0.001), specifically fear from the mild stage (p < 0.05), pain and disgust from the moderate stage (p < 0.05 and p < 0.001 respectively). The Exploratory Factor Analysis showed that recognition of pain and recognition of other discreet emotions were underpinned by two different latent factors. Performances on pain expression matching task and pain intensity discrimination task did not differ by group. (p = 0.334 and p = 0.787 respectively). Finally, moderate AD patients judged the pain less intensively than the Control group for both, moderate, and severe pain intensity (p < 0.001). Our data suggest that AD disrupts the recognition of pain expression along with recognition of fear and disgust. Additionally, AD patients seem to underestimate pain intensity compared to controls. The self-rated pain scales should be adapted to the pain processing deficit of AD patients.

Emotion concept in perception of facial expressions: Effects of emotion-label words and emotion-laden words

Emotion perception, the ability to detect and categorize the emotions of others, is an important component of social competence. The basic emotion theory and constructed emotion theory remain controversial over the role of emotional semantic information in the processing of facial expressions. In the present study, in order to explore the mechanism of emotional semantic information in different processing stages of facial expressions, we used the ERPs to investigate the effects of emotion-label words (e.g., happy, disgusted) and emotion-laden words (e.g., flower, maggot) on the perception of happy and disgusted faces. The results showed that disgusted faces were more susceptible to emotion words, and there were differences between word types and consistency conditions in the N170 component. More importantly, disgusted faces showed a significant N400 effect in the emotion-label words condition, that is, the amplitude in the emotion inconsistent condition was negative compared with that in the emotion consistent condition. The processing of happy faces was not significantly affected by the emotional information of words, and no significant effects related to consistency or to word types were found. These results found that disgusted faces exhibited emotional semantic conflict in the inconsistent emotion-label words condition, suggesting that emotion words moderated the perception of disgusted faces, supporting the hypothesis that emotional semantic information influenced the perception of facial expressions.