"Hearing faces and seeing voices": Amodal coding of person identity in the human brain.

Bashar Awwad Shiekh Hasan,Pascal Belin,Joachim Gross,Mitchell Valdes-Sosa

doi:10.1038/srep37494

Abstract

Recognizing familiar individuals is achieved by the brain by combining cues from several sensory modalities, including the face of a person and her voice. Here we used functional magnetic resonance (fMRI) and a whole-brain, searchlight multi-voxel pattern analysis (MVPA) to search for areas in which local fMRI patterns could result in identity classification as a function of sensory modality. We found several areas supporting face or voice stimulus classification based on fMRI responses, consistent with previous reports; the classification maps overlapped across modalities in a single area of right posterior superior temporal sulcus (pSTS). Remarkably, we also found several cortical areas, mostly located along the middle temporal gyrus, in which local fMRI patterns resulted in identity “cross-classification”: vocal identity could be classified based on fMRI responses to the faces, or the reverse, or both. These findings are suggestive of a series of cortical identity representations increasingly abstracted from the input modality.

Highlights

Local patterns of cerebral activity measured with functional magnetic resonance imaging (fMRI) can classify familiar faces or voices. Overlap of face- and voice-classifying areas in right posterior STS. Cross-classification of facial and vocal identity in several temporal lobe areas
Whether the person identity nodes” (PINs) have a neuronal counterpart in the human brain remains unclear, in part owing to the fact that most studies of person recognition—either using neuropsychological assessment of patients with brain lesions, or neuroimaging techniques such as functional magnetic resonance imaging in healthy volunteers—have focused on single modality, mostly face, far second, voice; only few studies have investigated the cerebral bases of person recognition based on more than one sensory modality[1,4]
For the first time, local areas resulting in identity cross-classification, that is, successful classification of identities in one modality based on fMRI responses in the other modality

Summary

Introduction

Local patterns of cerebral activity measured with fMRI can classify familiar faces or voices. Overlap of face- and voice-classifying areas in right posterior STS. Cross-classification of facial and vocal identity in several temporal lobe areas. The contribution of different sensory modalities to person perception is acknowledged in particular by cognitive models such as Bruce and Young (1986)’s model of face perception. They proposed the notion of “person identity nodes” (PINs): a portion of associative memory holding identity-specific semantic codes that can be accessed via the face, the voice or other modalities: it is the point at which person recognition, as opposed to face recognition, is achieved[6,7]. A PIN, as defined in Bruce & Young (1986), would correspond to a patient with a brain

Methods

Results

Conclusion