Audiovisual integration in macaque face patch neurons

Amit P Khandhadia,Aidan P Murphy,Lizabeth M Romanski,Jennifer K Bizley,David A Leopold

doi:10.1016/j.cub.2021.01.102

Abstract

SummaryPrimate social communication depends on the perceptual integration of visual and auditory cues, reflected in the multimodal mixing of sensory signals in certain cortical areas. The macaque cortical face patch network, identified through visual, face-selective responses measured with fMRI, is assumed to contribute to visual social interactions. However, whether face patch neurons are also influenced by acoustic information, such as the auditory component of a natural vocalization, remains unknown. Here, we recorded single-unit activity in the anterior fundus (AF) face patch, in the superior temporal sulcus, and anterior medial (AM) face patch, on the undersurface of the temporal lobe, in macaques presented with audiovisual, visual-only, and auditory-only renditions of natural movies of macaques vocalizing. The results revealed that 76% of neurons in face patch AF were significantly influenced by the auditory component of the movie, most often through enhancement of visual responses but sometimes in response to the auditory stimulus alone. By contrast, few neurons in face patch AM exhibited significant auditory responses or modulation. Control experiments in AF used an animated macaque avatar to demonstrate, first, that the structural elements of the face were often essential for audiovisual modulation and, second, that the temporal modulation of the acoustic stimulus was more important than its frequency spectrum. Together, these results identify a striking contrast between two face patches and specifically identify AF as playing a potential role in the integration of audiovisual cues during natural modes of social communication.

Highlights

Each macaque received a single implant into a recorded face patch
We have previously demonstrated that this recording method supports longitudinal, stable recordings from the same cells over multiple sessions.[29,30]
We recorded from 295 neurons in face patches of four monkey subjects: 240 from anterior fundus (AF) (125 from monkey SP, 115 from monkey SR) and 55 neurons from anterior medial (AM) (49 from monkey W, 6 from monkey M)

Summary

Introduction

In humans and other primate species, audiovisual integration plays an important role in social communication, for example, during the perception of a conspecific’s vocalization and concomitant facial behavior.[1,2] The temporal cortex, and the superior temporal sulcus (STS), contain zones of convergence for high-level sensory signals.[3,4,5,6] In the macaque, the STS fundus borders high-level visual and auditory cortex[7,8,9,10] and exchanges connections with other multisensory areas, including ventrolateral prefrontal cortex (VLPFC) and intraparietal cortex.[8,11,12] At the single-cell level, neurons within portions of the STS respond to visual and auditory stimuli, as well as their combination.[5,13,14,15,16] Functional MRI (fMRI) investigation of the macaque temporal cortex has revealed a number of operationally defined regions named according to their visual category selectivity, such as face and body patches.[17,18,19,20,21] In macaques, face patches are replete with cells that respond more strongly to faces than to other categories of images[22,23,24] and form an interconnected network.[25,26] A subset of these patches lies along the STS and is coextensive with known multisensory regions in the fundus.[5,15] despite intensive study of neurons within the visually defined face patches, it is presently unknown whether or not they participate in multisensory integration

Methods

Results

Discussion

Conclusion