Neural Representation Of Faces Research Articles

Face identity and facial expression representations with adaptation paradigms: New directions for potential applications.

Adaptation and aftereffect are well-known procedures for exploring our neural representation of visual stimuli. It has been reported that they occur in face identity, facial expressions, and low-level visual features. This method has two primary advantages. One is to reveal the common or shared process of faces, that is, the overlapped or discrete representation of face identities or facial expressions. The other is to investigate the coding system or theory of face processing that underlies the ability to recognize faces. This study aims to organize recent research to guide the reader into the field of face adaptation and its aftereffect and to suggest possible future expansions in the use of this paradigm. To achieve this, we reviewed the behavioral short-term aftereffect studies on face identity (i.e., who it is) and facial expressions (i.e., what expressions such as happiness and anger are expressed), and summarized their findings about the neural representation of faces. First, we summarize the basic characteristics of face aftereffects compared to simple visual features to clarify that facial aftereffects occur at a different stage and are not inherited or combinations of low-level visual features. Next, we introduce the norm-based coding hypothesis, which is one of the theories used to represent face identity and facial expressions, and adaptation is a commonly used procedure to examine this. Subsequently, we reviewed studies that applied this paradigm to immature or impaired face recognition (i.e., children and individuals with autism spectrum disorder or prosopagnosia) and examined the relationships between their poor recognition performance and representations. Moreover, we reviewed studies dealing with the representation of non-presented faces and social signals conveyed via faces and discussed that the face adaptation paradigm is also appropriate for these types of examinations. Finally, we summarize the research conducted to date and propose a new direction for the face adaptation paradigm.

Neural Representations of Faces Are Tuned to Eye Movements.

Eye movements provide a functional signature of how human vision is achieved. Many recent studies have consistently reported robust idiosyncratic visual sampling strategies during face recognition. Whether these interindividual differences are mirrored by idiosyncratic neural responses remains unknown. To this aim, we first tracked eye movements of male and female observers during face recognition. Additionally, for every observer we obtained an objective index of neural face discrimination through EEG that was recorded while they fixated different facial information. We found that foveation of facial features fixated longer during face recognition elicited stronger neural face discrimination responses across all observers. This relationship occurred independently of interindividual differences in preferential facial information sampling (e.g., eye vs mouth lookers), and started as early as the first fixation. Our data show that eye movements play a functional role during face processing by providing the neural system with the information that is diagnostic to a specific observer. The effective processing of identity involves idiosyncratic, rather than universal face representations.SIGNIFICANCE STATEMENT When engaging in face recognition, observers deploy idiosyncratic fixation patterns to sample facial information. Whether these individual differences concur with idiosyncratic face-sensitive neural responses remains unclear. To address this issue, we recorded observers' fixation patterns, as well as their neural face discrimination responses elicited during fixation of 10 different locations on the face, corresponding to different types of facial information. Our data reveal a clear interplay between individuals' face-sensitive neural responses and their idiosyncratic eye-movement patterns during identity processing, which emerges as early as the first fixation. Collectively, our findings favor the existence of idiosyncratic, rather than universal face representations.

The Relative Role of Viewpoint and Identity in the Neural Representation of Faces in Fusiform Gyrus

The Relative Role of Viewpoint and Identity in the Neural Representation of Faces in Fusiform Gyrus

The neural representation of faces and bodies in motion and at rest

The neural organization of person processing relies on brain regions functionally selective for faces or bodies, with a subset of these regions preferring moving stimuli. Although the response properties of the individual areas are well established, less is known about the neural response to a whole person in a natural environment. Targeting an area of cortex that spans multiple functionally-selective face and body regions, we examined the relationship among neural activity patterns elicited in response to faces, bodies, and people in static and moving displays. When both stimuli were static or moving, pattern classification analyses indicated highly discriminable responses to faces, bodies, and whole people. Neural discrimination transferred in both directions between representations created from moving or static stimuli. It transferred also to stimuli experienced across static and dynamic presentations (one static and the other dynamic). In both transfer cases, however, discrimination accuracy decreased relative to the case where the representations were both created and tested with static or moving forms. Next, we examined the relative contribution of activity pattern and response magnitude to discrimination by comparing classifiers that operated with magnitude-normalized scans with classifiers that retained pattern and magnitude information. When both stimuli were moving or static, response magnitude contributed to classification, but the spatially distributed activity pattern accounted for most of the discrimination. Across static and moving presentations, activity pattern accounted completely for the discriminability of neural responses to faces, bodies, and people, with no contribution from response magnitude. Combined, the results indicate redundant and flexible access to person-based shape codes from moving and static presentations. The transfer of shape information across presentation types that preferentially access dorsal and ventral visual processing streams indicates that a common shape code may ground functional divisions in the processing of face and body information.

Neural representations of faces and limbs neighbor in human high-level visual cortex: evidence for a new organization principle

Neurophysiology and optical imaging studies in monkeys and functional magnetic resonance imaging (fMRI) studies in both monkeys and humans have localized clustered neural responses in inferotemporal cortex selective for images of biologically relevant categories, such as faces and limbs. Using higher resolution (1.5 mm voxels) fMRI scanning methods than past studies (3–5 mm voxels), we recently reported a network of multiple face- and limb-selective regions that neighbor one another in human ventral temporal cortex (Weiner and Grill-Spector, Neuroimage, 52(4):1559–1573, 2010) and lateral occipitotemporal cortex (Weiner and Grill-Spector, Neuroimage, 56(4):2183–2199, 2011). Here, we expand on three basic organization principles of high-level visual cortex revealed by these findings: (1) consistency in the anatomical location of functional regions, (2) preserved spatial relationship among functional regions, and (3) a topographic organization of face- and limb-selective regions in adjacent and alternating clusters. We highlight the implications of this structure in comparing functional brain organization between typical and atypical populations. We conclude with a new model of high-level visual cortex consisting of ventral, lateral, and dorsal components, where multimodal processing related to vision, action, haptics, and language converges in the lateral pathway.Electronic supplementary materialThe online version of this article (doi:10.1007/s00426-011-0392-x) contains supplementary material, which is available to authorized users.

Neural representations of personally familiar and unfamiliar faces in the anterior inferior temporal cortex of monkeys

To investigate the neural representations of faces in primates, particularly in relation to their personal familiarity or unfamiliarity, neuronal activities were chronically recorded from the ventral portion of the anterior inferior temporal cortex (AITv) of macaque monkeys during the performance of a facial identification task using either personally familiar or unfamiliar faces as stimuli. By calculating the correlation coefficients between neuronal responses to the faces for all possible pairs of faces given in the task and then using the coefficients as neuronal population-based similarity measures between the faces in pairs, we analyzed the similarity/dissimilarity relationship between the faces, which were potentially represented by the activities of a population of the face-responsive neurons recorded in the area AITv. The results showed that, for personally familiar faces, different identities were represented by different patterns of activities of the population of AITv neurons irrespective of the view (e.g., front, 90° left, etc.), while different views were not represented independently of their facial identities, which was consistent with our previous report. In the case of personally unfamiliar faces, the faces possessing different identities but presented in the same frontal view were represented as similar, which contrasts with the results for personally familiar faces. These results, taken together, outline the neuronal representations of personally familiar and unfamiliar faces in the AITv neuronal population.

Gender in Facial Representations: A Contrast-Based Study of Adaptation within and between the Sexes

Face aftereffects are proving to be an effective means of examining the properties of face-specific processes in the human visual system. We examined the role of gender in the neural representation of faces using a contrast-based adaptation method. If faces of different genders share the same representational face space, then adaptation to a face of one gender should affect both same- and different-gender faces. Further, if these aftereffects differ in magnitude, this may indicate distinct gender-related factors in the organization of this face space. To control for a potential confound between physical similarity and gender, we used a Bayesian ideal observer and human discrimination data to construct a stimulus set in which pairs of different-gender faces were equally dissimilar as same-gender pairs. We found that the recognition of both same-gender and different-gender faces was suppressed following a brief exposure of 100ms. Moreover, recognition was more suppressed for test faces of a different-gender than those of the same-gender as the adaptor, despite the equivalence in physical and psychophysical similarity. Our results suggest that male and female faces likely occupy the same face space, allowing transfer of aftereffects between the genders, but that there are special properties that emerge along gender-defining dimensions of this space.

Neural Representations of Faces and Body Parts in Macaque and Human Cortex: A Comparative fMRI Study

Single-cell studies in the macaque have reported selective neural responses evoked by visual presentations of faces and bodies. Consistent with these findings, functional magnetic resonance imaging studies in humans and monkeys indicate that regions in temporal cortex respond preferentially to faces and bodies. However, it is not clear how these areas correspond across the two species. Here, we directly compared category-selective areas in macaques and humans using virtually identical techniques. In the macaque, several face- and body part-selective areas were found located along the superior temporal sulcus (STS) and middle temporal gyrus (MTG). In the human, similar to previous studies, face-selective areas were found in ventral occipital and temporal cortex and an additional face-selective area was found in the anterior temporal cortex. Face-selective areas were also found in lateral temporal cortex, including the previously reported posterior STS area. Body part-selective areas were identified in the human fusiform gyrus and lateral occipitotemporal cortex. In a first experiment, both monkey and human subjects were presented with pictures of faces, body parts, foods, scenes, and man-made objects, to examine the response profiles of each category-selective area to the five stimulus types. In a second experiment, face processing was examined by presenting upright and inverted faces. By comparing the responses and spatial relationships of the areas, we propose potential correspondences across species. Adjacent and overlapping areas in the macaque anterior STS/MTG responded strongly to both faces and body parts, similar to areas in the human fusiform gyrus and posterior STS. Furthermore, face-selective areas on the ventral bank of the STS/MTG discriminated both upright and inverted faces from objects, similar to areas in the human ventral temporal cortex. Overall, our findings demonstrate commonalities and differences in the wide-scale brain organization between the two species and provide an initial step toward establishing functionally homologous category-selective areas.

Differential sensitivity for viewpoint between familiar and unfamiliar faces in human visual cortex

People are extremely proficient at recognizing faces that are familiar to them, but are poor at identifying unfamiliar faces. We used fMR-adaptation to ask whether this difference in recognition might be reflected in the relative viewpoint-dependence of face-selective regions in the brain. A reduced response (adaptation) to repeated images of unfamiliar or familiar faces was found in the fusiform face area (FFA), but not in the superior temporal sulcus (STS) face-selective region. To establish if the neural representation of faces was invariant to changes in viewpoint, we parametrically varied the viewing angle of successive images using 3-dimensional models of unfamiliar and familiar faces. We found adaptation to familiar faces across all changes in viewpoint in the FFA. In contrast, a release from adaptation was apparent in the FFA when unfamiliar faces were viewed at increasing viewing angles. These results provide a neural basis for differences in the recognition of familiar and unfamiliar faces.

The M170 Reflects a Viewpoint-Dependent Representation for Both Familiar and Unfamiliar Faces

The aim of this study was to determine the extent to which the neural representation of faces in visual cortex is viewpoint dependent or viewpoint invariant. Magnetoencephalography was used to measure evoked responses to faces during an adaptation paradigm. Using familiar and unfamiliar faces, we compared the amplitude of the M170 response to repeated images of the same face with images of different faces. We found a reduction in the M170 amplitude to repeated presentations of the same face image compared with images of different faces when shown from the same viewpoint. To establish if this adaptation to the identity of a face was invariant to changes in viewpoint, we varied the viewing angle of the face within a block. We found a reduction in response was no longer evident when images of the same face were shown from different viewpoints. This viewpoint-dependent pattern of results was the same for both familiar and unfamiliar faces. These results imply that either the face-selective M170 response reflects an early stage of face processing or that the computations underlying face recognition depend on a viewpoint-dependent neuronal representation.

Long-term Expertise with Artificial Objects Increases Visual Competition with Early Face Categorization Processes

The degree of commonality between the perceptual mechanisms involved in processing faces and objects of expertise is intensely debated. To clarify this issue, we recorded occipito-temporal event-related potentials in response to faces when concurrently processing visual objects of expertise. In car experts fixating pictures of cars, we observed a large decrease of an evoked potential elicited by face stimuli between 130 and 200 msec, the N170. This sensory suppression was much lower when the car and face stimuli were separated by a 200-msec blank interval. With and without this delay, there was a strong correlation between the face-evoked N170 amplitude decrease and the subject's level of car expertise as measured in an independent behavioral task. Together, these results show that neural representations of faces and nonface objects in a domain of expertise compete for visual processes in the occipito-temporal cortex as early as 130-200 msec following stimulus onset.

Neural response to the visual familiarity of faces

Recognizing personally familiar faces is the result of a spatially distributed process that involves visual perceptual areas and areas that play a role in other cognitive and social functions, such as the anterior paracingulate cortex, the precuneus and the amygdala [M.I. Gobbini, E. Leibenluft, N. Santiago, J.V. Haxby, Social and emotional attachment in the neural representation of faces, Neuroimage 22 (2004) 1628–1635; M.I. Gobbini, J.V. Haxby, Neural systems for recognition of familiar faces, Neuropsychologia, in press; E. Leibenluft, M.I. Gobbini, T. Harrison, J.V. Haxby, Mothers’ neural activation in response to pictures of their, and other, children, Biol. Psychiatry 56 (2004) 225–232]. In order to isolate the role of visual familiarity in face recognition, we used fMRI to measure the response to faces characterized by experimentally induced visual familiarity that carried no biographical information or emotional content. The fMRI results showed a stronger response in the precuneus to the visually familiar faces consistent with studies that implicate this region in the retrieval of information from long-term memory and imagery. Moreover, this finding supports the hypothesis of a key role for the precuneus in the acquisition of familiarity with faces [H. Kosaka, M. Omori, T. Iidaka, T. Murata, T. Shimoyama, T. Okada, N. Sadato, Y. Yonekura, Y. Wada, Neural substrates participating in acquisition of facial familiarity: an fMRI study, Neuroimage 20 (2003) 1734–1742]. By contrast, the visually familiar faces evoked a weaker response in the fusiform gyrus, which may reflect the development of a sparser encoding or a reduced attentional load when processing stimuli that are familiar. The visually familiar faces also evoked a weaker response in the amygdala, supporting the proposed role of this structure in mediating the guarded attitude when meeting someone new.

Social and emotional attachment in the neural representation of faces

To dissociate the role of visual familiarity from the role of social and emotional factors in recognizing familiar individuals, we measured neural activity using functional magnetic resonance imaging (fMRI) while subjects viewed (1) faces of personally familiar individuals (i.e. friends and family), (2) faces of famous individuals, and (3) faces of strangers. Personally familiar faces evoked a stronger response than did famous familiar faces and unfamiliar faces in areas that have been associated with ‘theory of mind’, and a weaker response in the amygdala. These response modulations may reflect the spontaneous activation of social knowledge about the personality and attitudes of close friends and relatives and the less guarded attitude one has around these people. These results suggest that familiarity causes changes in neural response that extend beyond a visual memory for a face.

Neural responses to Mooney images reveal a modular representation of faces in human visual cortex

The way in which information about objects is represented in visual cortex remains controversial. One model of human object recognition poses that information is processed in modules, highly specialised for different categories of objects; an opposing model appeals to a distributed representation across a large network of visual areas. We addressed this debate by monitoring activity in face- and object-selective areas while human subjects viewed ambiguous face stimuli (Mooney faces). The measured neural response in the face-selective region of the fusiform gyrus was greater when subjects reported seeing a face than when they perceived the image as a collection of blobs. In contrast, there was no difference in magnetic resonance response between face and no-face perceived events in either the face-selective voxels of the superior temporal sulcus or the object-selective voxels of the parahippocampal gyrus and lateral occipital complex. These results challenge the concept that neural representation of faces is distributed and overlapping and suggest that the fusiform gyrus is tightly linked to the awareness of faces.