System For Face Perception Research Articles

Shared neural codes for visual and semantic information about familiar faces in a common representational space

Processes evoked by seeing a personally familiar face encompass recognition of visual appearance and activation of social and person knowledge. Whereas visual appearance is the same for all viewers, social and person knowledge may be more idiosyncratic. Using between-subject multivariate decoding of hyperaligned functional magnetic resonance imaging data, we investigated whether representations of personally familiar faces in different parts of the distributed neural system for face perception are shared across individuals who know the same people. We found that the identities of both personally familiar and merely visually familiar faces were decoded accurately across brains in the core system for visual processing, but only the identities of personally familiar faces could be decoded across brains in the extended system for processing nonvisual information associated with faces. Our results show that personal interactions with the same individuals lead to shared neural representations of both the seen and unseen features that distinguish their identities.

The neural representation of personally familiar and unfamiliar faces in the distributed system for face perception

Personally familiar faces are processed more robustly and efficiently than unfamiliar faces. The human face processing system comprises a core system that analyzes the visual appearance of faces and an extended system for the retrieval of person-knowledge and other nonvisual information. We applied multivariate pattern analysis to fMRI data to investigate aspects of familiarity that are shared by all familiar identities and information that distinguishes specific face identities from each other. Both identity-independent familiarity information and face identity could be decoded in an overlapping set of areas in the core and extended systems. Representational similarity analysis revealed a clear distinction between the two systems and a subdivision of the core system into ventral, dorsal and anterior components. This study provides evidence that activity in the extended system carries information about both individual identities and personal familiarity, while clarifying and extending the organization of the core system for face perception.

Temporal dynamics of the core and extended face perception system with fMRI

Temporal dynamics of the core and extended face perception system with fMRI

Plastic reorganization of neural systems for perception of others in the congenitally blind

Recent evidence suggests that the function of the core system for face perception might extend beyond visual face-perception to a broader role in person perception. To critically test the broader role of core face-system in person perception, we examined the role of the core system during the perception of others in 7 congenitally blind individuals and 15 sighted subjects by measuring their neural responses using fMRI while they listened to voices and performed identity and emotion recognition tasks. We hypothesised that in people who have had no visual experience of faces, core face-system areas may assume a role in the perception of others via voices. Results showed that emotions conveyed by voices can be decoded in homologues of the core face system only in the blind. Moreover, there was a specific enhancement of response to verbal as compared to non-verbal stimuli in bilateral fusiform face areas and the right posterior superior temporal sulcus showing that the core system also assumes some language-related functions in the blind. These results indicate that, in individuals with no history of visual experience, areas of the core system for face perception may assume a role in aspects of voice perception that are relevant to social cognition and perception of others' emotions.

Watching the brain recalibrate: Neural correlates of renormalization during face adaptation

The face perception system flexibly adjusts its neural responses to current face exposure, inducing aftereffects in the perception of subsequent faces. For instance, adaptation to expanded faces makes undistorted faces appear compressed, and adaptation to compressed faces makes undistorted faces appear expanded. Such distortion aftereffects have been proposed to result from renormalization, in which the visual system constantly updates a prototype according to the adaptors’ characteristics and evaluates subsequent faces relative to that. However, although consequences of adaptation are easily observed in behavioral aftereffects, it has proven difficult to observe renormalization during adaptation itself. Here we directly measured brain responses during adaptation to establish a neural correlate of renormalization. Given that the face-evoked occipito-temporal P2 event-related brain potential has been found to increase with face prototypicality, we reasoned that the adaptor-elicited P2 could serve as an electrophysiological indicator for renormalization. Participants adapted to sequences of four distorted (compressed or expanded) or undistorted faces, followed by a slightly distorted test face, which they had to classify as undistorted or distorted. We analysed ERPs evoked by each of the adaptors and found that P2 (but not N170) amplitudes evoked by consecutive adaptor faces exhibited an electrophysiological pattern of renormalization during adaptation to distorted faces: P2 amplitudes evoked by both compressed and expanded adaptors significantly increased towards asymptotic levels as adaptation proceeded. P2 amplitudes were smallest for the first adaptor, significantly larger for the second, and yet larger for the third adaptor. We conclude that the sensitivity of the occipito-temporal P2 to the perceived deviation of a face from the norm makes this component an excellent tool to study adaptation-induced renormalization.

Disentangling the Representation of Identity from Head View Along the Human Face Processing Pathway

Neural models of a distributed system for face perception implicate a network of regions in the ventral visual stream for recognition of identity. Here, we report a functional magnetic resonance imaging (fMRI) neural decoding study in humans that shows that this pathway culminates in the right inferior frontal cortex face area (rIFFA) with a representation of individual identities that has been disentangled from variable visual features in different images of the same person. At earlier stages in the pathway, processing begins in early visual cortex and the occipital face area with representations of head view that are invariant across identities, and proceeds to an intermediate level of representation in the fusiform face area in which identity is emerging but still entangled with head view. Three-dimensional, view-invariant representation of identities in the rIFFA may be the critical link to the extended system for face perception, affording activation of person knowledge and emotional responses to familiar faces.

Consecutive TMS-fMRI reveals remote effects of neural noise to the “occipital face area”

The human cortical system for face perception comprises a network of connected regions including the middle fusiform gyrus (“fusiform face area” or FFA), the inferior occipital gyrus (“occipital face area” or OFA), and the posterior superior temporal sulcus (pSTS). Here, we sought to investigate how transcranial magnetic stimulation (TMS) to the OFA affects activity within the face processing network. We used offline repetitive TMS to temporarily introduce neural noise in the right OFA in healthy subjects. We then immediately performed functional magnetic resonance imaging (fMRI) to measure changes in blood oxygenation level dependent (BOLD) signal across the face network using an fMR-adaptation (fMR-A) paradigm. We hypothesized that TMS to the right OFA would induce abnormal face identity coding throughout the face processing network in regions to which it has direct or indirect connections. Indeed, BOLD signal for face identity, but not non-face (butterfly) identity, decreased in the right OFA and FFA following TMS to the right OFA compared to both sham TMS and TMS to a control site, the nearby object-related lateral occipital area (LO). Further, TMS to the right OFA decreased face-related activation in the left FFA, without any effect in the left OFA. Our findings indicate that TMS to the right OFA selectively disrupts face coding at both the stimulation site and bilateral FFA. TMS to the right OFA also decreased BOLD signal for different identity stimuli in the right pSTS. Together with mounting evidence from patient studies, we demonstrate connectivity of the OFA within the face network and that its activity modulates face processing in bilateral FFA as well as the right pSTS. Moreover, this study shows that deep regions within the face network can be remotely probed by stimulating structures closer to the cortical surface.

Absence of Sex-Contingent Gaze Direction Aftereffects Suggests a Limit to Contingencies in Face Aftereffects.

Face aftereffects (e.g., expression aftereffects) can be simultaneously induced in opposite directions for different face categories (e.g., male and female faces). Such aftereffects are typically interpreted as indicating that distinct neural populations code the categories on which adaptation is contingent, e.g., male and female faces. Moreover, they suggest that these distinct populations selectively respond to variations in the secondary stimulus dimension, e.g., emotional expression. However, contingent aftereffects have now been reported for so many different combinations of face characteristics, that one might question this interpretation. Instead, the selectivity might be generated during the adaptation procedure, for instance as a result of associative learning, and not indicate pre-existing response selectivity in the face perception system. To alleviate this concern, one would need to demonstrate some limit to contingent aftereffects. Here, we report a clear limit, showing that gaze direction aftereffects are not contingent on face sex. We tested 36 young Caucasian adults in a gaze adaptation paradigm. We initially established their ability to discriminate the gaze direction of male and female test faces in a pre-adaptation phase. Afterwards, half of the participants adapted to female faces looking left and male faces looking right, and half adapted to the reverse pairing. We established the effects of this adaptation on the perception of gaze direction in subsequently presented male and female test faces. We found that adaptation induced pronounced gaze direction aftereffects, i.e., participants were biased to perceive small gaze deviations to both the left and right as direct. Importantly, however, aftereffects were identical for male and female test faces, showing that the contingency of face sex and gaze direction participants experienced during the adaptation procedure had no effect.

Reverse engineering the face perception system: insights from congenital prosopagnosia

Reverse engineering the face perception system: insights from congenital prosopagnosia

Gaze direction affects the magnitude of face identity aftereffects.

The face perception system partly owes its efficiency to adaptive mechanisms that constantly recalibrate face coding to our current diet of faces. Moreover, faces that are better attended produce more adaptation. Here, we investigated whether the social cues conveyed by a face can influence the amount of adaptation that face induces. We compared the magnitude of face identity aftereffects induced by adaptors with direct and averted gazes. We reasoned that faces conveying direct gaze may be more engaging and better attended and thus produce larger aftereffects than those with averted gaze. Using an adaptation duration of 5 s, we found that aftereffects for adaptors with direct and averted gazes did not differ (Experiment 1). However, when processing demands were increased by reducing adaptation duration to 1 s, we found that gaze direction did affect the magnitude of the aftereffect, but in an unexpected direction: Aftereffects were larger for adaptors with averted rather than direct gaze (Experiment 2). Eye tracking revealed that differences in looking time to the faces between the two gaze directions could not account for these findings. Subsequent ratings of the stimuli (Experiment 3) showed that adaptors with averted gaze were actually perceived as more expressive and interesting than adaptors with direct gaze. Therefore it appears that the averted-gaze faces were more engaging and better attended, leading to larger aftereffects. Overall, our results suggest that naturally occurring facial signals can modulate the adaptive impact a face exerts on our perceptual system. Specifically, the faces that we perceive as most interesting also appear to calibrate the organization of our perceptual system most strongly.

On the other side of the fence: effects of social categorization and spatial grouping on memory and attention for own-race and other-race faces.

The term “own-race bias” refers to the phenomenon that humans are typically better at recognizing faces from their own than a different race. The perceptual expertise account assumes that our face perception system has adapted to the faces we are typically exposed to, equipping it poorly for the processing of other-race faces. Sociocognitive theories assume that other-race faces are initially categorized as out-group, decreasing motivation to individuate them. Supporting sociocognitive accounts, a recent study has reported improved recognition for other-race faces when these were categorized as belonging to the participants' in-group on a second social dimension, i.e., their university affiliation. Faces were studied in groups, containing both own-race and other-race faces, half of each labeled as in-group and out-group, respectively. When study faces were spatially grouped by race, participants showed a clear own-race bias. When faces were grouped by university affiliation, recognition of other-race faces from the social in-group was indistinguishable from own-race face recognition. The present study aimed at extending this singular finding to other races of faces and participants. Forty Asian and 40 European Australian participants studied Asian and European faces for a recognition test. Faces were presented in groups, containing an equal number of own-university and other-university Asian and European faces. Between participants, faces were grouped either according to race or university affiliation. Eye tracking was used to study the distribution of spatial attention to individual faces in the display. The race of the study faces significantly affected participants' memory, with better recognition of own-race than other-race faces. However, memory was unaffected by the university affiliation of the faces and by the criterion for their spatial grouping on the display. Eye tracking revealed strong looking biases towards both own-race and own-university faces. Results are discussed in light of the theoretical accounts of the own-race bias.

TMS to the “occipital face area” affects recognition but not categorization of faces

The human cortical system for face perception is comprised of a network of connected regions including the middle fusiform gyrus (“fusiform face area” or FFA), the inferior occipital cortex (“occipital face area” or OFA), and the superior temporal sulcus. The traditional hierarchical feedforward model of visual processing suggests information flows from early visual cortex to the OFA for initial face feature analysis to higher order regions including the FFA for identity recognition. However, patient data suggest an alternative model. Patients with acquired prosopagnosia, an inability to visually recognize faces, have been documented with lesions to the OFA but who nevertheless show face-selective activation in the FFA. Moreover, their ability to categorize faces remains intact. This suggests that the FFA is not solely responsible for face recognition and the network is not strictly hierarchical, but may be organized in a reverse hierarchical fashion. We used transcranial magnetic stimulation (TMS) to temporarily disrupt processing in the OFA in neurologically-intact individuals and found participants’ ability to categorize intact versus scrambled faces was unaffected, however face identity discrimination was significantly impaired. This suggests that face categorization but not recognition can occur without the “earlier” OFA being online and indicates that “lower level” face category processing may be assumed by other intact face network regions such as the FFA. These results are consistent with the patient data and support a non-hierarchical, global-to-local model with re-entrant connections between the OFA and other face processing areas.

The face perception system becomes species-specific at 3 months: An eye-tracking study

The current study aimed at investigating own- vs. other-species preferences in 3-month-old infants. The infants’ eye movements were recorded during a visual preference paradigm to assess whether they show a preference for own-species faces when contrasted with other-species faces. Human and monkey faces, equated for all low-level perceptual characteristics, were used. Our results demonstrated that 3-month-old infants preferred the human face, suggesting that the face perception system becomes species-specific after 3 months of visual experience with a specific class of faces. The eye tracking results are also showing that fixations were more focused on the eye area of human faces, supporting the notion of their importance in holding visual attention.

Multivoxel patterns in face-sensitive temporal regions reveal an encoding schema based on detecting life in a face

More than a decade of research has demonstrated that faces evoke prioritized processing in a 'core face network' of three brain regions. However, whether these regions prioritize the detection of global facial form (shared by humans and mannequins) or the detection of life in a face has remained unclear. Here, we dissociate form-based and animacy-based encoding of faces by using animate and inanimate faces with human form (humans, mannequins) and dog form (real dogs, toy dogs). We used multivariate pattern analysis of BOLD responses to uncover the representational similarity space for each area in the core face network. Here, we show that only responses in the inferior occipital gyrus are organized by global facial form alone (human vs dog) while animacy becomes an additional organizational priority in later face-processing regions: the lateral fusiform gyri (latFG) and right superior temporal sulcus. Additionally, patterns evoked by human faces were maximally distinct from all other face categories in the latFG and parts of the extended face perception system. These results suggest that once a face configuration is perceived, faces are further scrutinized for whether the face is alive and worthy of social cognitive resources.

Dynamic Facial Expressions Evoke Distinct Activation in the Face Perception Network: A Connectivity Analysis Study

Very little is known about the neural structures involved in the perception of realistic dynamic facial expressions. In the present study, a unique set of naturalistic dynamic facial emotional expressions was created. Through fMRI and connectivity analysis, a dynamic face perception network was identified, which is demonstrated to extend Haxby et al.'s [Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. The distributed human neural system for face perception. Trends in Cognitive Science, 4, 223-233, 2000] distributed neural system for face perception. This network includes early visual regions, such as the inferior occipital gyrus, which is identified as insensitive to motion or affect but sensitive to the visual stimulus, the STS, identified as specifically sensitive to motion, and the amygdala, recruited to process affect. Measures of effective connectivity between these regions revealed that dynamic facial stimuli were associated with specific increases in connectivity between early visual regions, such as the inferior occipital gyrus and the STS, along with coupling between the STS and the amygdala, as well as the inferior frontal gyrus. These findings support the presence of a distributed network of cortical regions that mediate the perception of different dynamic facial expressions.

Is the face-perception system human-specific at birth?

The present study investigates the human-specificity of the orienting system that allows neonates to look preferentially at faces. Three experiments were carried out to determine whether the face-perception system that is present at birth is broad enough to include both human and nonhuman primate faces. The results demonstrate that the newborns did not show any spontaneous visual preference for the human face when presented simultaneously with a monkey face that shared the same features, configuration, and low-level perceptual properties (Experiment 1). The newborns were, however, able to discriminate between the 2 faces belonging to the 2 different species (Experiment 2). In Experiment 3, the newborns were found to prefer looking at an upright, compared with an inverted, monkey face, as they do for human faces. Overall, the results demonstrate that newborns perceive monkey and human faces in a similar way. These findings are consistent with the hypothesis that the system underlying face preference at birth is broad enough to bias newborns' attention toward both human and nonhuman primate faces.

Asymmetrical interactions in the perception of face identity and emotional expression are not unique to the primate visual system

The human visual system appears to process the identity of faces separately from their emotional expression, whereas the human visual system does not appear to process emotional expression separately from identity. All current explanations of this visual processing asymmetry implicitly assume that it arises because of the organization of a specialized human face perception system. A second possibility is that this finding reflects general principles of perceptual processing. Studying animals that are unlikely to have evolved a specialized face perception system may shed fresh light on this issue. We report two experiments that investigated the interaction of identity and emotional expression in pigeons' perception of human faces. Experiment 1 found that pigeons perceive the similarity among faces sharing identity and emotion, and that these two dimensions are integral according to a spatial model of generalization. Experiment 2 found that pigeons' discrimination of emotion was reliably affected by irrelevant variations in identity, whereas pigeons' discrimination of identity was not reliably affected by irrelevant variations in emotion. Thus, the asymmetry previously reported in human studies was reproduced in our pigeon study. These results challenge the view that a specialized human face perception system must underlie this effect.

Deficits in long-term recognition memory reveal dissociated subtypes in congenital prosopagnosia.

The study investigates long-term recognition memory in congenital prosopagnosia (CP), a lifelong impairment in face identification that is present from birth. Previous investigations of processing deficits in CP have mostly relied on short-term recognition tests to estimate the scope and severity of individual deficits. We firstly report on a controlled test of long-term (one year) recognition memory for faces and objects conducted with a large group of participants with CP. Long-term recognition memory is significantly impaired in eight CP participants (CPs). In all but one case, this deficit was selective to faces and didn't extend to intra-class recognition of object stimuli. In a test of famous face recognition, long-term recognition deficits were less pronounced, even after accounting for differences in media consumption between controls and CPs. Secondly, we combined test results on long-term and short-term recognition of faces and objects, and found a large heterogeneity in severity and scope of individual deficits. Analysis of the observed heterogeneity revealed a dissociation of CP into subtypes with a homogeneous phenotypical profile. Thirdly, we found that among CPs self-assessment of real-life difficulties, based on a standardized questionnaire, and experimentally assessed face recognition deficits are strongly correlated. Our results demonstrate that controlled tests of long-term recognition memory are needed to fully assess face recognition deficits in CP. Based on controlled and comprehensive experimental testing, CP can be dissociated into subtypes with a homogeneous phenotypical profile. The CP subtypes identified align with those found in prosopagnosia caused by cortical lesions; they can be interpreted with respect to a hierarchical neural system for face perception.

Recognizing moving faces: A psychological and neural synthesis

Information for identifying a human face can be found both in the invariant structure of features and in idiosyncratic movements and gestures. When both kinds of information are available, psychological evidence indicates that: (1) dynamic information contributes more to recognition under non-optimal viewing conditions, e.g. poor illumination, low image resolution, recognition from a distance; (2) dynamic information contributes more as a viewer's experience with the face increases; and (3) a structure-from-motion analysis can make a perceptually based contribution to face recognition. A recently proposed distributed neural system for face perception, with minor modifications, can accommodate the psychological findings with moving faces.

Is Social Phobia a “Mis-Communication” Disorder? Brain Functional Connectivity during Face Perception Differs between Patients with Social Phobia and Healthy Control Subjects

Recently, a differential recruitment of brain areas throughout the distributed neural system for face perception has been found in social phobic patients as compared to healthy control subjects. These functional abnormalities in social phobic patients extend beyond emotion-related brain areas, such as the amygdala, to include cortical networks that modulate attention and process other facial features, and they are also associated with an alteration of the task-related activation/deactivation trade-off. Functional connectivity is becoming a powerful tool to examine how components of large-scale distributed neural systems are coupled together while performing a specific function. This study was designed to determine whether functional connectivity networks among brain regions within the distributed system for face perception also would differ between social phobic patients and healthy controls. Data were obtained from eight social phobic patients and seven healthy controls by using functional magnetic resonance imaging. Our findings indicated that social phobic patients and healthy controls have different patterns of functional connectivity across brain regions within both the core and the extended systems for face perception and the default mode network. To our knowledge, this is the first study that shows that functional connectivity during brain response to socially relevant stimuli differs between social phobic patients and healthy controls. These results expand our previous findings and indicate that brain functional changes in social phobic patients are not restricted to a single specific brain structure, but rather involve a mis-communication among different sensory and emotional processing brain areas.