Face Processing System Research Articles

Predictions enable top-down pattern separation in the macaque face-processing hierarchy

Distinguishing faces requires well distinguishable neural activity patterns. Contextual information may separate neural representations, leading to enhanced identity recognition. Here, we use functional magnetic resonance imaging to investigate how predictions derived from contextual information affect the separability of neural activity patterns in the macaque face-processing system, a 3-level processing hierarchy in ventral visual cortex. We find that in the presence of predictions, early stages of this hierarchy exhibit well separable and high-dimensional neural geometries resembling those at the top of the hierarchy. This is accompanied by a systematic shift of tuning properties from higher to lower areas, endowing lower areas with higher-order, invariant representations instead of their feedforward tuning properties. Thus, top-down signals dynamically transform neural representations of faces into separable and high-dimensional neural geometries. Our results provide evidence how predictive context transforms flexible representational spaces to optimally use the computational resources provided by cortical processing hierarchies for better and faster distinction of facial identities.

A familiar face and person processing area in the human temporal pole

How does the brain process the faces of familiar people? Neuropsychological studies have argued for an area of the temporal pole (TP) linking faces with person identities, but magnetic susceptibility artifacts in this region have hampered its study with fMRI. Using data acquisition and analysis methods optimized to overcome this artifact, we identify a familiar face response in TP, reliably observed in individual brains. This area responds strongly to visual images of familiar faces over unfamiliar faces, objects, and scenes. However, TP did not just respond to images of faces, but also to a variety of high-level social cognitive tasks, including semantic, episodic, and theory of mind tasks. The response profile of TP contrasted with a nearby region of the perirhinal cortex that responded specifically to faces, but not to social cognition tasks. TP was functionally connected with a distributed network in the association cortex associated with social cognition, while PR was functionally connected with face-preferring areas of the ventral visual cortex. This work identifies a missing link in the human face processing system that specifically processes familiar faces, and is well placed to integrate visual information about faces with higher-order conceptual information about other people. The results suggest that separate streams for person and face processing reach anterior temporal areas positioned at the top of the cortical hierarchy.

Altered effective connectivity among face-processing systems in major depressive disorder.

Neuroimaging studies have revealed abnormal functional interaction during the processing of emotional faces in patients with major depressive disorder (MDD), thereby enhancing our comprehension of the pathophysiology of MDD. However, it is unclear whether there is abnormal directional interaction among face-processing systems in patients with MDD. A group of patients with MDD and a healthy control group underwent a face-matching task during functional magnetic resonance imaging. Dynamic causal modelling (DCM) analysis was used to investigate effective connectivity between 7 regions in the face-processing systems. We used a Parametric Empirical Bayes model to compare effective connectivity between patients with MDD and controls. We included 48 patients and 44 healthy controls in our analyses. Both groups showed higher accuracy and faster reaction time in the shape-matching condition than in the face-matching condition. However, no significant behavioural or brain activation differences were found between the groups. Using DCM, we found that, compared with controls, patients with MDD showed decreased self-connection in the right dorsolateral prefrontal cortex (DLPFC), amygdala, and fusiform face area (FFA) across task conditions; increased intrinsic connectivity from the right amygdala to the bilateral DLPFC, right FFA, and left amygdala, suggesting an increased intrinsic connectivity centred in the amygdala in the right side of the face-processing systems; both increased and decreased positive intrinsic connectivity in the left side of the face-processing systems; and comparable task modulation effect on connectivity. Our study did not include longitudinal neuroimaging data, and there was limited region of interest selection in the DCM analysis. Our findings provide evidence for a complex pattern of alterations in the face-processing systems in patients with MDD, potentially involving the right amygdala to a greater extent. The results confirm some previous findings and highlight the crucial role of the regions on both sides of face-processing systems in the pathophysiology of MDD.

Self-reported eye contact sensitivity and face processing in chromosome 22q11.2 deletion syndrome

ABSTRACT Introduction 22q11.2 deletion syndrome (22qDS) has been associated with varying levels of social impairments, and with atypical visual scanning of faces. The present study explored whether self-reported sensitivity to eye contact might be related to these phenomena. Method Individuals with confirmed 22qDS were interviewed about their experience and possible discomfort with eye contact. In cases where individuals expresesed discomfort, they were subsequently asked about coping mechanisms used to deal with this discomfort. In addition to self-reported eye contact discomfort, gaze to emotional faces was examined using eye tracking. Results In the subgroup of individuals who reported discomfort during eye contact, eye tracking results revealed a lower amount of gaze in the eyes of neutral faces, as well as the absence of the typical left visual field (LVF) bias, indicative of alterations in hemispheric lateralization. This subgroup also scored lower on a measure of everyday functioning. Conclusions Our results show that, by simply asking individuals with this social and communicative disorder about eye gaze discomfort, we may better understand the specific challenges that they experience. Moreover, information gained from such first-person reports together with eye-tracking measures further informs about the integrity of their face processing system, as well as about the nature and degree of impairment in this population.

Investigating the Face Inversion Effect in Autism Across Behavioral and Neural Measures of Face Processing

Face processing is foundational to human social cognition, is central to the hallmark features of autism spectrum disorder (ASD), and shapes neural systems and social behavior. Highly efficient and specialized, the face processing system is sensitive to inversion, demonstrated by reduced accuracy in recognition and altered neural response to inverted faces. Understanding at which mechanistic level the autistic face processing system may be particularly different, as measured by the face inversion effect, will improve overall understanding of brain functioning in autism. To synthesize data from the extant literature to determine differences of the face processing system in ASD, as measured by the face inversion effect, across multiple mechanistic levels. Systematic searches were conducted in the MEDLINE, Embase, Web of Science, and PubMed databases from inception to August 11, 2022. Original research that reported performance-based measures of face recognition to upright and inverted faces in ASD and neurotypical samples were included for quantitative synthesis. All studies were screened by at least 2 reviewers. This systematic review and meta-analysis was conducted according to the 2020 Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline. Multiple effect sizes were extracted from studies to maximize information gain and statistical precision and used a random-effects, multilevel modeling framework to account for statistical dependencies within study samples. Effect sizes were calculated as a standardized mean change score between ASD and neurotypical samples (ie, Hedges g). The primary outcome measure was performance difference between upright and inverted faces during face recognition tasks. Measurement modality, psychological construct, recognition demand, sample age, sample sex distribution, and study quality assessment scores were assessed as moderators. Of 1768 screened articles, 122 effect sizes from 38 empirical articles representing data from 1764 individual participants (899 ASD individuals and 865 neurotypical individuals) were included in the meta-analysis. Overall, face recognition performance differences between upright and inverted faces were reduced in autistic individuals compared with neurotypical individuals (g = -0.41; SE = 0.11; 95% credible interval [CrI], -0.63 to -0.18). However, there was considerable heterogeneity among effect sizes, which were explored with moderator analysis. The attenuated face inversion effect in autistic individuals was more prominent in emotion compared with identity recognition (b = 0.46; SE = 0.26; 95% CrI, -0.08 to 0.95) and in behavioral compared with electrophysiological measures (b = 0.23; SE = 0.24; 95% CrI, -0.25 to 0.70). This study found that on average, face recognition in autism is less impacted by inversion. These findings suggest less specialization or expertise of the face processing system in autism, particularly in recognizing emotion from faces as measured in behavioral paradigms.

EEG alpha band functional connectivity reveals distinct cortical dynamics for overt and covert emotional face processing

Current knowledge regarding how the focus of our attention during face processing influences neural responses largely comes from neuroimaging studies reporting on regional brain activations. The present study was designed to add novel insights to this research by studying how attention can differentially impact the way cortical regions interact during emotional face processing. High-density electroencephalogram was recorded in a sample of fifty-two healthy participants during an emotional face processing task. The task required participants to either attend to the expressions (i.e., overt processing) or attend to a perceptual distractor, which rendered the expressions task-irrelevant (i.e., covert processing). Functional connectivity in the alpha band was estimated in source space and modeled using graph theory to quantify whole-brain integration and segregation. Results revealed that overt processing of facial expressions is linked to reduced cortical segregation and increased cortical integration, this latter specifically for negative expressions of fear and sadness. Furthermore, we observed increased communication efficiency during overt processing of negative expressions between the core and the extended face processing systems. Overall, these findings reveal that attention makes the interaction among the nodes involved in face processing more efficient, also uncovering a connectivity signature of the prioritized processing mechanism of negative expressions, that is an increased cross-communication within the nodes of the face processing network.

Overwhelmed by the man in the moon? Pareidolic objects provoke increased amygdala activation in autism

An interesting feature of the primate face detection system results in the perception of illusory faces in objects, or pareidolia. These illusory faces do not per se contain social information, such as eye-gaze or specific identities, yet they activate the cortical brain face-processing network, possibly via the subcortical route, including the amygdala. In autism spectrum disorder (ASD), aversion to eye-contact is commonly reported, and so are alterations in face processing more generally, yet the underlying reasons are not clear. Here we show that in autistic participants (N=37), but not in non-autistic controls (N=34), pareidolic objects increase amygdala activation bilaterally (right amygdala peak: X = 26, Y = −6, Z = −16; left amygdala peak X = −24, Y = −6, Z = −20). In addition, illusory faces engage the face-processing cortical network significantly more in ASD than in controls. An early imbalance in the excitatory and inhibitory systems in autism, affecting typical brain maturation, may be at the basis of an overresponsive reaction to face configuration and to eye contact. Our data add to the evidence of an oversensitive subcortical face processing system in ASD.

Dynamic, naturalistic faces embedded in a narrative elicit responses in the distributed face processing system

Dynamic, naturalistic faces embedded in a narrative elicit responses in the distributed face processing system

The origins of evil: From lesions to the functional architecture of the antisocial brain

In the past decades, a growing body of evidence has suggested that some individuals may exhibit antisocial behaviors following brain lesions. Recently, some authors have shown that lesions underpinning antisocial behaviors may disrupt a particular brain network during resting-state. However, it remains unknown whether these brain lesions may alter specific mental processes during tasks. Therefore, we conducted meta-analytic co-activation analyses on lesion masks of 17 individuals who acquired antisocial behaviors following their brain lesions. Each lesion mask was used as a seed of interest to examine their aberrant co-activation network using a database of 143 whole-brain neuroimaging studies on antisocial behaviors (n = 5,913 subjects). We aimed to map the lesion brain network that shows deficient activity in antisocial population against a null distribution derived from 655 control lesions. We further characterized the lesion-based meta-analytic network using term-based decoding (Neurosynth) as well as receptor/transporter density maps (JuSpace). We found that the lesion meta-analytic network included the amygdala, orbitofrontal cortex, ventro- and dorso-medial prefrontal cortex, fusiform face area, and supplementary motor area (SMA), which correlated mainly with emotional face processing and serotoninergic system (5-HT1A and 5-HTT). We also investigated the heterogeneity in co-activation networks through data-driven methods and found that lesions could be grouped in four main networks, encompassing emotional face processing, general emotion processing, and reward processing. Our study shows that the heterogeneous brain lesions underpinning antisocial behaviors may disrupt specific mental processes, which further increases the risk for distinct antisocial symptoms. It also highlights the importance and complexity of studying brain lesions in relationship with antisocial behaviors.

Recognition of Masked Faces in the Era of the Pandemic: No Improvement Despite Extensive Natural Exposure

Face masks, which became prevalent across the globe during the COVID-19 pandemic, have had a negative impact on face recognition despite the availability of critical information from uncovered face parts, especially the eyes. An outstanding question is whether face-mask effects would be attenuated following extended natural exposure. This question also pertains, more generally, to face-recognition training protocols. We used the Cambridge Face Memory Test in a cross-sectional study (N = 1,732 adults) at six different time points over a 20-month period, alongside a 12-month longitudinal study (N = 208). The results of the experiments revealed persistent deficits in recognition of masked faces and no sign of improvement across time points. Additional experiments verified that the amount of individual experience with masked faces was not correlated with the mask effect. These findings provide compelling evidence that the face-processing system does not easily adapt to visual changes in face stimuli, even following prolonged real-life exposure.

Developmental changes within the extended face processing network: A cross‐sectional functional magnetic resonance imaging study

In the field of face processing, the so-called "core network" has been intensively researched. Its neural activity can be reliably detected in children and adults using functional magnetic resonance imaging (fMRI). However, the core network's counterpart, the so-called "extended network," has been less researched. In the present study, we compared children's and adults' brain activity in the extended system, in particular in the amygdala, the insula, and the inferior frontal gyrus (IFG). Using fMRI, we compared the brain activation pattern between children aged 7-9 years and adults during an emotional face processing task. On the one hand, children showed increased activity in the extended face processing system in relation to adults, particularly in the left amygdala, the right insula, and the left IFG. On the other hand, lateralization indices revealed a "leftward bias" in children's IFG compared to adults. These results suggest that brain activity associated with face processing is characterized by a developmental decrease in activity. They further show that the development is associated with a rightward migration of face-related IFG activation, possibly due to the competition for neural space between several developing brain functions ("developmental competition hypothesis").

Single-Unit Recordings Reveal the Selectivity of a Human Face Area.

The exquisite capacity of primates to detect and recognize faces is crucial for social interactions. Although disentangling the neural basis of human face recognition remains a key goal in neuroscience, direct evidence at the single-neuron level is limited. We recorded from face-selective neurons in human visual cortex in a region characterized by functional magnetic resonance imaging (fMRI) activations for faces compared with objects. The majority of visually responsive neurons in this fMRI activation showed strong selectivity at short latencies for faces compared with objects. Feature-scrambled faces and face-like objects could also drive these neurons, suggesting that this region is not tightly tuned to the visual attributes that typically define whole human faces. These single-cell recordings within the human face processing system provide vital experimental evidence linking previous imaging studies in humans and invasive studies in animal models.SIGNIFICANCE STATEMENT We present the first recordings of face-selective neurons in or near an fMRI-defined patch in human visual cortex. Our unbiased multielectrode array recordings (i.e., no selection of neurons based on a search strategy) confirmed the validity of the BOLD contrast (faces-objects) in humans, a finding with implications for all human imaging studies. By presenting faces, feature-scrambled faces, and face-pareidolia (perceiving faces in inanimate objects) stimuli, we demonstrate that neurons at this level of the visual hierarchy are broadly tuned to the features of a face, independent of spatial configuration and low-level visual attributes.

The other-race effect on the McGurk effect in infancy

This study investigated the difference in the McGurk effect between own-race-face and other-race-face stimuli among Japanese infants from 5 to 9 months of age. The McGurk effect results from infants using information from a speaker’s face in audiovisual speech integration. We hypothesized that the McGurk effect varies with the speaker’s race because of the other-race effect, which indicates an advantage for own-race faces in our face processing system. Experiment 1 demonstrated the other-race effect on audiovisual speech integration such that the infants ages 5–6 months and 8–9 months are likely to perceive the McGurk effect when observing an own-race-face speaker, but not when observing an other-race-face speaker. Experiment 2 found the other-race effect on audiovisual speech integration regardless of irrelevant speech identity cues. Experiment 3 confirmed the infants’ ability to differentiate two auditory syllables. These results showed that infants are likely to integrate voice with an own-race-face, but not with an other-race-face. This implies the role of experiences with own-race-faces in the development of audiovisual speech integration. Our findings also contribute to the discussion of whether perceptual narrowing is a modality-general, pan-sensory process.

Altered effective connectivity of the extended face processing system in depression and its association with treatment response: findings from the YoDA-C randomized controlled trial.

Depression is commonly associated with fronto-amygdala dysfunction during the processing of emotional face expressions. Interactions between these regions are hypothesized to contribute to negative emotional processing biases and as such have been highlighted as potential biomarkers of treatment response. This study aimed to investigate depression associated alterations to directional connectivity and assess the utility of these parameters as predictors of treatment response. Ninety-two unmedicated adolescents and young adults (mean age 20.1; 56.5% female) with moderate-to-severe major depressive disorder and 88 healthy controls (mean age 19.8; 61.4% female) completed an implicit emotional face processing fMRI task. Patients were randomized to receive cognitive behavioral therapy for 12 weeks, plus either fluoxetine or placebo. Using dynamic causal modelling, we examined functional relationships between six brain regions implicated in emotional face processing, comparing both patients and controls and treatment responders and non-responders. Depressed patients demonstrated reduced inhibition from the dlPFC to vmPFC and reduced excitation from the dlPFC to amygdala during sad expression processing. During fearful expression processing patients showed reduced inhibition from the vmPFC to amygdala and reduced excitation from the amygdala to dlPFC. Response was associated with connectivity from the amygdala to dlPFC during sad expression processing and amygdala to vmPFC connectivity during fearful expression processing. Our study clarifies the nature of face processing network alterations in adolescents and young adults with depression, highlighting key interactions between the amygdala and prefrontal cortex. Moreover, these findings highlight the potential utility of these interactions in predicting treatment response.

Developmental prosopagnosics and super recognizers rely on the same facial features used by individuals with normal face recognition abilities for face identification

Face recognition depends on the ability of the face processing system to extract facial features that define the identity of a face. In a recent study we discovered that altering a subset of facial features changed the identity of the face, indicating that they are critical for face identification. Changing another set of features did not change the identity of a face, indicating that they are not critical for face identification. In the current study, we assessed whether developmental prosopagnosics (DPs) and super recognizers (SRs) also rely more heavily on these critical features than non-critical features for face identification. To that end, we presented to DPs and SRs faces in which either the critical or the non-critical features were manipulated. In Study 1, we presented SRs with a famous face recognition task. We found that overall SRs recognized famous faces that differ in either critical or non-critical features better than controls. Similar to controls, changes in critical features had a larger effect on SRs’ face recognition than changes in non-critical features. In Study 2, we presented an identity matching task to DPs and SRs. Similar to controls, DPs and SRs perceived faces that differed in critical features as more different than faces that differed in non-critical features. Taken together, our results indicate that SRs and DPs use the same critical features for face identification as normal individuals. These findings emphasize the fundamental role of this subset of features for face identification.

Spatiotemporal Dynamics of Covert Versus Overt Processing of Happy, Fearful and Sad Facial Expressions.

Behavioral and electrophysiological correlates of the influence of task demands on the processing of happy, sad, and fearful expressions were investigated in a within-subjects study that compared a perceptual distraction condition with task-irrelevant faces (e.g., covert emotion task) to an emotion task-relevant categorization condition (e.g., overt emotion task). A state-of-the-art non-parametric mass univariate analysis method was used to address the limitations of previous studies. Behaviorally, participants responded faster to overtly categorized happy faces and were slower and less accurate to categorize sad and fearful faces; there were no behavioral differences in the covert task. Event-related potential (ERP) responses to the emotional expressions included the N170 (140–180 ms), which was enhanced by emotion irrespective of task, with happy and sad expressions eliciting greater amplitudes than neutral expressions. EPN (200–400 ms) amplitude was modulated by task, with greater voltages in the overt condition, and by emotion, however, there was no interaction of emotion and task. ERP activity was modulated by emotion as a function of task only at a late processing stage, which included the LPP (500–800 ms), with fearful and sad faces showing greater amplitude enhancements than happy faces. This study reveals that affective content does not necessarily require attention in the early stages of face processing, supporting recent evidence that the core and extended parts of the face processing system act in parallel, rather than serially. The role of voluntary attention starts at an intermediate stage, and fully modulates the response to emotional content in the final stage of processing.

Facial expression is retained in deep networks trained for face identification.

Facial expressions distort visual cues for identification in two-dimensional images. Face processing systems in the brain must decouple image-based information from multiple sources to operate in the social world. Deep convolutional neural networks (DCNN) trained for face identification retain identity-irrelevant, image-based information (e.g., viewpoint). We asked whether a DCNN trained for identity also retains expression information that generalizes over viewpoint change. DCNN representations were generated for a controlled dataset containing images of 70 actors posing 7 facial expressions (happy, sad, angry, surprised, fearful, disgusted, neutral), from 5 viewpoints (frontal, 90° and 45° left and right profiles). Two-dimensional visualizations of the DCNN representations revealed hierarchical groupings by identity, followed by viewpoint, and then by facial expression. Linear discriminant analysis of full-dimensional representations predicted expressions accurately, mean 76.8% correct for happiness, followed by surprise, disgust, anger, neutral, sad, and fearful at 42.0%; chance 14.3%. Expression classification was stable across viewpoints. Representational similarity heatmaps indicated that image similarities within identities varied more by viewpoint than by expression. We conclude that an identity-trained, deep network retains shape-deformable information about expression and viewpoint, along with identity, in a unified form—consistent with a recent hypothesis for ventral visual stream processing.

Differential Modulation of Effective Connectivity in the Brain's Extended Face Processing System by Fearful and Sad Facial Expressions.

The processing of emotional facial expressions is underpinned by the integration of information from a distributed network of brain regions. Despite investigations into how different emotional expressions alter the functional relationships within this network, there remains limited research examining which regions drive these interactions. This study investigated effective connectivity during the processing of sad and fearful facial expressions to better understand how these stimuli differentially modulate emotional face processing circuitry. Ninety-eight healthy human adolescents and young adults, aged between 15 and 25 years, underwent an implicit emotional face processing fMRI task. Using dynamic causal modeling (DCM), we examined five brain regions implicated in face processing. These were restricted to the right hemisphere and included the occipital and fusiform face areas, amygdala, and dorsolateral prefrontal cortex (dlPFC) and ventromedial prefrontal cortex (vmPFC). Processing sad and fearful facial expressions were associated with greater positive connectivity from the amygdala to dlPFC. Only the processing of fearful facial expressions was associated with greater negative connectivity from the vmPFC to amygdala. Compared with processing sad faces, processing fearful faces was associated with significantly greater connectivity from the amygdala to dlPFC. No difference was found between the processing of these expressions and the connectivity from the vmPFC to amygdala. Overall, our findings indicate that connectivity from the amygdala and dlPFC appears to be responding to dimensional features which differ between these expressions, likely those relating to arousal. Further research is necessary to examine whether this relationship is also observable for positively valenced emotions.

Face recognition in beginning readers: Investigating the potential relationship between reading and face recognition during the first year of school

ABSTRACT Reading acquisition has been suggested to drive the lateralization of the face processing system to the right cerebral hemisphere. To investigate whether this developmental co-dependency has a behavioural cost, at least in the short run, we tested whether learning to read reduced face recognition ability. In a longitudinal design, 82 children (5–7 years) were tested twice, at the beginning and end of their first year in school. In both sessions, visual letter recognition, word recognition, sentence reading, and immediate face recognition were tested. Reliable increases were found across all four measures, suggesting no absolute cost of literacy acquisition on face processing efficiency. In addition, we found no evidence suggesting a negative relationship between literacy acquisition and face recognition performance. Indeed, our results were most supportive of the null hypothesis suggesting independence in the development of literacy and face processing skills on the behavioural level.

Cultural influences on face scanning are consistent across infancy and adulthood

The emergence of cultural differences in face scanning is thought to be shaped by social experience. However, previous studies mainly investigated eye movements of adults and little is known about early development. The current study recorded eye movements of British and Japanese infants (aged 10 and 16 months) and adults, who were presented with static and dynamic faces on screen. Cultural differences were observed across all age groups, with British participants exhibiting more mouth scanning, and Japanese individuals showing increased central face (nose) scanning for dynamic stimuli. Age-related influences independent of culture were also revealed, with a shift from eye to mouth scanning between 10 and 16 months, while adults distributed their gaze more flexibly. Against our prediction, no age-related increases in cultural differences were observed, suggesting the possibility that cultural differences are largely manifest by 10 months of age. Overall, the findings suggest that individuals adopt visual strategies in line with their cultural background from early in infancy, pointing to the development of a highly adaptive face processing system that is shaped by early sociocultural experience.