Human Face Processing Research Articles

Multimodal imaging of the amygdala in non-clinical subjects with high vs. low autistic-like social skills traits

Recent clinical and theoretical frameworks suggest that social skills and theory of mind impairments characteristic of autism spectrum disorder (ASD) are distributed in the general population on a continuum between healthy individuals and patients. The present multimodal study aimed at investigating the amygdala's function, perfusion, and volume in 56 non-clinical subjects from the general population with high (n=28 High-SOC) or low (n=28 Low-SOC) autistic-like social skills traits. Participants underwent magnetic resonance imaging to evaluate the amygdala's functional connectivity at rest, blood perfusion by means of arterial spin labelling, its activation during a face evaluation task and lastly grey matter volumes. The High-SOC group was characterised by higher blood perfusion in both amygdalae, lower volume of the left amygdala and higher activations of the right amygdala during processing of human faces with fearful value. Resting state analyses did not reveal any significant difference between the two groups. Overall, our results highlight the presence of overlapping morpho-functional alterations of the amygdala between healthy individuals and ASD patients confirming the importance of the amygdala in this disorder and in social and emotional processing. Our findings may help disentangle the neurobiological facets of ASD elucidating aetiology and the relationship between clinical symptomatology and neurobiology.

You're Beautiful When You Smile: Event-Related Brain Potential (ERP) Evidence of Early Opposite-Gender Bias in Happy Faces.

Studies of social cognition have shown gender differences regarding human face processing. One interesting finding is the enhanced processing of opposite-gender faces at different time stages, as revealed by event-related brain potentials. Crucially, from an evolutionary perspective, such a bias might interact with the emotional expression of the face. To investigate this, 100 participants (50 female, 50 male) completed an expression-detection task while their EEG was recorded. In three blocks, fearful, happy and neutral faces (female and male) were randomly presented, with participants instructed to respond to only one predefined target expression level in each block. Using linear mixed models, we observed both faster reaction times as well as larger P1 and late positive potential (LPP) amplitudes for women compared to men, supporting a generally greater female interest in faces. Highly interestingly, the analysis revealed an opposite-gender bias at P1 for happy target faces. This suggests that participants' attentional templates may include more opposite-gender facial features when selectively attending to happy faces. While N170 was influenced by neither the face nor the participant gender, LPP was modulated by the face gender and specific combinations of the target status, face gender and expression, which is interpreted in the context of gender-emotion stereotypes. Future research should further investigate this expression and attention dependency of early opposite-gender biases.

A familiar face and person processing area in the human temporal pole

How does the brain process the faces of familiar people? Neuropsychological studies have argued for an area of the temporal pole (TP) linking faces with person identities, but magnetic susceptibility artifacts in this region have hampered its study with fMRI. Using data acquisition and analysis methods optimized to overcome this artifact, we identify a familiar face response in TP, reliably observed in individual brains. This area responds strongly to visual images of familiar faces over unfamiliar faces, objects, and scenes. However, TP did not just respond to images of faces, but also to a variety of high-level social cognitive tasks, including semantic, episodic, and theory of mind tasks. The response profile of TP contrasted with a nearby region of the perirhinal cortex that responded specifically to faces, but not to social cognition tasks. TP was functionally connected with a distributed network in the association cortex associated with social cognition, while PR was functionally connected with face-preferring areas of the ventral visual cortex. This work identifies a missing link in the human face processing system that specifically processes familiar faces, and is well placed to integrate visual information about faces with higher-order conceptual information about other people. The results suggest that separate streams for person and face processing reach anterior temporal areas positioned at the top of the cortical hierarchy.

Disentangled deep generative models reveal coding principles of the human face processing network.

Despite decades of research, much is still unknown about the computations carried out in the human face processing network. Recently, deep networks have been proposed as a computational account of human visual processing, but while they provide a good match to neural data throughout visual cortex, they lack interpretability. We introduce a method for interpreting brain activity using a new class of deep generative models, disentangled representation learning models, which learn a low-dimensional latent space that "disentangles" different semantically meaningful dimensions of faces, such as rotation, lighting, or hairstyle, in an unsupervised manner by enforcing statistical independence between dimensions. We find that the majority of our model's learned latent dimensions are interpretable by human raters. Further, these latent dimensions serve as a good encoding model for human fMRI data. We next investigate the representation of different latent dimensions across face-selective voxels. We find that low- and high-level face features are represented in posterior and anterior face-selective regions, respectively, corroborating prior models of human face recognition. Interestingly, though, we find identity-relevant and irrelevant face features across the face processing network. Finally, we provide new insight into the few "entangled" (uninterpretable) dimensions in our model by showing that they match responses in the ventral stream and carry information about facial identity. Disentangled face encoding models provide an exciting alternative to standard "black box" deep learning approaches for modeling and interpreting human brain data.

Separable processes for live “in-person” and live “zoom-like” faces

Abstract It has long been understood that the ventral visual stream of the human brain processes features of simulated human faces. Recently, specificity for real and interactive faces has been reported in lateral and dorsal visual streams, raising new questions regarding neural coding of interactive faces and lateral and dorsal face-processing mechanisms. We compare neural activity during two live interactive face-to-face conditions where facial features and tasks remain constant while the social contexts (in-person or on-line conditions) are varied. Current models of face processing do not predict differences in these two conditions as features do not vary. However, behavioral eye-tracking measures showed longer visual dwell times on the real face and also increased arousal as indicated by pupil diameters for the real face condition. Consistent with the behavioral findings, signal increases with functional near infrared spectroscopy, fNIRS, were observed in dorsal-parietal regions for the real faces and increased cross-brain synchrony was also found within these dorsal-parietal regions for the real In-person Face condition. Simultaneously, acquired electroencephalography, EEG, also showed increased theta power in real conditions. These neural and behavioral differences highlight the importance of natural, in-person, paradigms and social context for understanding live and interactive face processing in humans.

Modeling naturalistic face processing in humans with deep convolutional neural networks.

Deep convolutional neural networks (DCNNs) trained for face identification can rival and even exceed human-level performance. The ways in which the internal face representations in DCNNs relate to human cognitive representations and brain activity are not well understood. Nearly all previous studies focused on static face image processing with rapid display times and ignored the processing of naturalistic, dynamic information. To address this gap, we developed the largest naturalistic dynamic face stimulus set in human neuroimaging research (700+ naturalistic video clips of unfamiliar faces). We used this naturalistic dataset to compare representational geometries estimated from DCNNs, behavioral responses, and brain responses. We found that DCNN representational geometries were consistent across architectures, cognitive representational geometries were consistent across raters in a behavioral arrangement task, and neural representational geometries in face areas were consistent across brains. Representational geometries in late, fully connected DCNN layers, which are optimized for individuation, were much more weakly correlated with cognitive and neural geometries than were geometries in late-intermediate layers. The late-intermediate face-DCNN layers successfully matched cognitive representational geometries, as measured with a behavioral arrangement task that primarily reflected categorical attributes, and correlated with neural representational geometries in known face-selective topographies. Our study suggests that current DCNNs successfully capture neural cognitive processes for categorical attributes of faces but less accurately capture individuation and dynamic features.

Looking at the upper facial half enlarges the range of holistic face processing

Previous studies suggested that upper and lower facial halves might be involved in the human holistic face processing differently. In this study, we replicated and extended the finding above. In Experiment 1, we used the standard composite-face task to measure holistic face processing when participants made judgements on the upper and lower facial halves separately. Results showed that the composite-face effect was stronger for the upper facial half compared to the lower half. In Experiment 2, we investigated how facial information was integrated when participants focused on different features, using the perceptual field paradigm. Results showed that: (1) more “peripheral faces” were chosen when participants fixated at the eyes than when they fixated at the mouth; (2) less “peripheral faces” were chosen for inverted faces regardless of the fixated features. Findings from both experiments together indicate that more peripheral facial information were integrated when participants focused on the upper facial half, highlighting the significance of focusing on the upper facial half in face processing.

Not so fast: Limited validity of deep convolutional neural networks as in silico models for human naturalistic face processing

Not so fast: Limited validity of deep convolutional neural networks as in silico models for human naturalistic face processing

An Exploratory Analysis of the Neural Correlates of Human-Robot Interactions With Functional Near Infrared Spectroscopy.

Functional near infrared spectroscopy (fNIRS) has been gaining increasing interest as a practical mobile functional brain imaging technology for understanding the neural correlates of social cognition and emotional processing in the human prefrontal cortex (PFC). Considering the cognitive complexity of human-robot interactions, the aim of this study was to explore the neural correlates of emotional processing of congruent and incongruent pairs of human and robot audio-visual stimuli in the human PFC with fNIRS methodology. Hemodynamic responses from the PFC region of 29 subjects were recorded with fNIRS during an experimental paradigm which consisted of auditory and visual presentation of human and robot stimuli. Distinct neural responses to human and robot stimuli were detected at the dorsolateral prefrontal cortex (DLPFC) and orbitofrontal cortex (OFC) regions. Presentation of robot voice elicited significantly less hemodynamic response than presentation of human voice in a left OFC channel. Meanwhile, processing of human faces elicited significantly higher hemodynamic activity when compared to processing of robot faces in two left DLPFC channels and a left OFC channel. Significant correlation between the hemodynamic and behavioral responses for the face-voice mismatch effect was found in the left OFC. Our results highlight the potential of fNIRS for unraveling the neural processing of human and robot audio-visual stimuli, which might enable optimization of social robot designs and contribute to elucidation of the neural processing of human and robot stimuli in the PFC in naturalistic conditions.

Emotion Facial Processing in Children With Autism Spectrum Disorder: A Pilot Study of the Impact of Service Dogs.

Processing and recognizing facial expressions are key factors in human social interaction. Past research suggests that individuals with autism spectrum disorder (ASD) present difficulties to decode facial expressions. Those difficulties are notably attributed to altered strategies in the visual scanning of expressive faces. Numerous studies have demonstrated the multiple benefits of exposure to pet dogs and service dogs on the interaction skills and psychosocial development of children with ASD. However, no study has investigated if those benefits also extend to the processing of facial expressions. The aim of this study was to investigate if having a service dog had an influence on facial expression processing skills of children with ASD. Two groups of 15 children with ASD, with and without a service dog, were compared using a facial expression recognition computer task while their ocular movements were measured using an eye-tracker. While the two groups did not differ in their accuracy and reaction time, results highlighted that children with ASD owning a service dog directed less attention toward areas that were not relevant to facial expression processing. They also displayed a more differentiated scanning of relevant facial features according to the displayed emotion (i.e., they spent more time on the mouth for joy than for anger, and vice versa for the eyes area). Results from the present study suggest that having a service dog and interacting with it on a daily basis may promote the development of specific visual exploration strategies for the processing of human faces.

Chapter 11 - Face processing in the temporal lobe

Face perception is a socially important but complex process with many stages and many facets. There is substantial evidence from many sources that it involves a large extent of the temporal lobe, from the ventral occipitotemporal cortex and superior temporal sulci to anterior temporal regions. While early human neuroimaging work suggested a core face network consisting of the occipital face area, fusiform face area, and posterior superior temporal sulcus, studies in both humans and monkeys show a system of face patches stretching from posterior to anterior in both the superior temporal sulcus and inferotemporal cortex. Sophisticated techniques such as fMRI adaptation have shown that these face-activated regions show responses that have many of the attributes of human face processing. Lesions of some of these regions in humans lead to variants of prosopagnosia, the inability to recognize the identity of a face. Lesion, imaging, and electrophysiologic data all suggest that there is a segregation between identity and expression processing, though some suggest this may be better characterized as a distinction between static and dynamic facial information.

Single-Unit Recordings Reveal the Selectivity of a Human Face Area.

The exquisite capacity of primates to detect and recognize faces is crucial for social interactions. Although disentangling the neural basis of human face recognition remains a key goal in neuroscience, direct evidence at the single-neuron level is limited. We recorded from face-selective neurons in human visual cortex in a region characterized by functional magnetic resonance imaging (fMRI) activations for faces compared with objects. The majority of visually responsive neurons in this fMRI activation showed strong selectivity at short latencies for faces compared with objects. Feature-scrambled faces and face-like objects could also drive these neurons, suggesting that this region is not tightly tuned to the visual attributes that typically define whole human faces. These single-cell recordings within the human face processing system provide vital experimental evidence linking previous imaging studies in humans and invasive studies in animal models.SIGNIFICANCE STATEMENT We present the first recordings of face-selective neurons in or near an fMRI-defined patch in human visual cortex. Our unbiased multielectrode array recordings (i.e., no selection of neurons based on a search strategy) confirmed the validity of the BOLD contrast (faces-objects) in humans, a finding with implications for all human imaging studies. By presenting faces, feature-scrambled faces, and face-pareidolia (perceiving faces in inanimate objects) stimuli, we demonstrate that neurons at this level of the visual hierarchy are broadly tuned to the features of a face, independent of spatial configuration and low-level visual attributes.

Face Recognition by Humans and Machines: Three Fundamental Advances from Deep Learning.

Deep learning models currently achieve human levels of performance on real-world face recognition tasks. We review scientific progress in understanding human face processing using computational approaches based on deep learning. This review is organized around three fundamental advances. First, deep networks trained for face identification generate a representation that retains structured information about the face (e.g., identity, demographics, appearance, social traits, expression) and the input image (e.g., viewpoint, illumination). This forces us to rethink the universe of possible solutions to the problem of inverse optics in vision. Second, deep learning models indicate that high-level visual representations of faces cannot be understood in terms of interpretable features. This has implications for understanding neural tuning and population coding in the high-level visual cortex. Third, learning in deep networks is a multistep process that forces theoretical consideration of diverse categories of learning that can overlap, accumulate over time, and interact. Diverse learning types are needed to model the development of human face processing skills, cross-race effects, and familiarity with individual faces.

Robot face memorability is affected by uncanny appearance

Human face recognition is influenced by various social and environmental constructs. For example, both age and race can affect the likelihood that a human face will be correctly recalled. General face appearance (i.e. friendly or untrustworthy faces) can also influence memorability. As human-robot interaction (HRI) becomes more commonplace, understanding what factors influence face recognition for non-human social agents is increasingly important. In particular, while there is a growing literature comparing the recognition of real human faces to computer-generated face images, comparisons between human face processing and robot face processing are largely unexplored. Here, we examined how the uncanny/eeriness of robot-faces affects memorability by using a 2AFC old/new task with various robot faces. Participants rated robot and human faces on perceived uncanniness during a study phase and were subsequently given a surprise memory task with only a fraction of the previously-encountered robot faces. Our results demonstrate that robots who are rated as more uncanny are more memorable than those that do not elicit the eerie feelings that correspond with uncanny faces: The more uncanny the robot face, the more accurately and quickly they were recalled. We discuss these results in the context of the design of social agents for HRI and also vis-a-vis theories of human face recognition and memory.

A shared mechanism for facial expression in human faces and face pareidolia.

Facial expressions are vital for social communication, yet the underlying mechanisms are still being discovered. Illusory faces perceived in objects (face pareidolia) are errors of face detection that share some neural mechanisms with human face processing. However, it is unknown whether expression in illusory faces engages the same mechanisms as human faces. Here, using a serial dependence paradigm, we investigated whether illusory and human faces share a common expression mechanism. First, we found that images of face pareidolia are reliably rated for expression, within and between observers, despite varying greatly in visual features. Second, they exhibit positive serial dependence for perceived facial expression, meaning an illusory face (happy or angry) is perceived as more similar in expression to the preceding one, just as seen for human faces. This suggests illusory and human faces engage similar mechanisms of temporal continuity. Third, we found robust cross-domain serial dependence of perceived expression between illusory and human faces when they were interleaved, with serial effects larger when illusory faces preceded human faces than the reverse. Together, the results support a shared mechanism for facial expression between human faces and illusory faces and suggest that expression processing is not tightly bound to human facial features.

Chinese Aesthetic Mask: Three Forehead and Five Eyes-Holistic Processing and Facial Attractiveness.

Human face processing has been attributed to holistic processing. Here, we ask whether humans are sensitive to configural information when perceiving facial attractiveness. By referring to a traditional Chinese aesthetic theory-Three Forehead and Five Eyes-we generated a series of faces that differed in spacing between facial features. We adopted a two-alternative forced-choice task in Experiment 1 and a rating task in Experiment 2 to assess attractiveness. Both tasks showed a consistent result: The faces which fit the Chinese aesthetic theory were chosen or rated as most attractive. This effect of configural information on facial attractiveness was larger for faces with highly attractive features than for faces with low attractive features. These findings provide experimental evidence for the traditional Chinese aesthetic theory. This issue can be further explored from the perspective of culture in the future.

LIFE Child Depression - a prospective longitudinal cohort study on the origin of depressive disorders between childhood and early adulthood

LIFE Child Depression - a prospective longitudinal cohort study on the origin of depressive disorders between childhood and early adulthood LIFE Child Depression is a prospective longitudinal study on the origin and course of depressive symptoms and disorders between child- and adulthood. The aim of the study is to identify patterns of developmental courses of symptoms and disorders and to investigate the interplay of psychosocial, biological and genetic risk and protective factors in the development of depressive disorders. The present paper gives an overview on results of the study. The sample was already assessed three times. A clinical sample was recruited from two local child psychiatric in- and outpatient services in Leipzig, a control sample was recruited from a children's health check program at our medical faculty (LIFE Child Health) and from the local registration office. We found some important context- and parent-associated risk factors for depressive disorders, such as negative life events, low socioeconomic status and depression in mothers (but not in fathers). Moreover, we found some characteristic biological and cognitive-emotional characteristics of children with depressive disorders, such as low stress-related cortisol, low evaluation of own performance, and more negative cognitions in dealing with stressful situations, low self-esteem and a general impairment of emotional processing of human faces. Only some of the risk factors were found to be specific to depression. Instead, most of them can be regarded as general risk factors for psychological disorders in childhood. It is also noteworthy, that some of the risk associations were gender-specific and need to be looked at from a differential point of view. Our study gives important indications for prevention for children at risk for depressive disorders as well as for therapeutic approaches.

General and own-species attentional face biases.

Humans demonstrate enhanced processing of human faces compared with animal faces, known as own-species bias. This bias is important for identifying people who may cause harm, as well as for recognizing friends and kin. However, growing evidence also indicates a more general face bias. Faces have high evolutionary importance beyond conspecific interactions, as they aid in detecting predators and prey. Few studies have explored the interaction of these biases together. In three experiments, we explored processing of human and animal faces, compared with each other and to nonface objects, which allowed us to examine both own-species and broader face biases. We used a dot-probe paradigm to examine human adults' covert attentional biases for task-irrelevant human faces, animal faces, and objects. We replicated the own-species attentional bias for human faces relative to animal faces. We also found an attentional bias for animal faces relative to objects, consistent with the proposal that faces broadly receive privileged processing. Our findings suggest that humans may be attracted to a broad class of faces. Further, we found that while participants rapidly attended to human faces across all cue display durations, they attended to animal faces only when they had sufficient time to process them. Our findings reveal that the dot-probe paradigm is sensitive for capturing both own-species and more general face biases, and that each has a different attentional signature, possibly reflecting their unique but overlapping evolutionary importance.

The effect of intranasal oxytocin on visual processing and salience of human faces

The mechanisms underlying the role of oxytocin (OT) as a regulator of social behavior in mammals are only partly understood. Recently, it has been proposed that OT increases the salience of social stimuli. We carried out a randomized, double-blind, cross-over study of the effects of OT on binocular rivalry, a visual phenomenon underpinned by the interplay of excitation and inhibition in the cortex. A final sample of 45 participants viewed images of social stimuli (faces with different emotional expressions) and non-social stimuli (houses and Gabor patches). We demonstrate a robust effect that intranasal OT increases the salience of human faces in binocular rivalry, such that dominance durations of faces are longer—this effect is not modulated by the facial expression. We tentatively show that OT treatment increases dominance durations for non-social stimuli. Our results lend support to the social salience hypothesis of OT, and in addition offer provisional support for the role of OT in influencing excitation-inhibition balance in the brain.

From concepts to percepts in human and machine face recognition: A reply to Blauch, Behrmann & Plaut

Intact recognition of familiar faces is critical for appropriate social interactions. Thus, the human face processing system should be optimized for familiar face recognition. Blauch et al. (2020) used face recognition deep convolutional neural networks (DCNNs) that are trained to maximize recognition of the trained (familiar) identities, to model human unfamiliar and familiar face recognition. In line with this model, we discuss behavioral, neuroimaging and computational findings that indicate that human face recognition develops from the generation of identity-specific concepts of familiar faces that are learned in a supervised manner, to the generation of view-invariant identity-general perceptual representations. Face-trained DCNNs seem to share some fundamental similarities with this framework.