Abstract
Auditory and visual signals often occur together, and the two sensory channels are known to influence each other to facilitate perception. The neural basis of this integration is not well understood, although other forms of multisensory influences have been shown to occur at surprisingly early stages of processing in cortex. Primary visual cortex neurons can show frequency-tuning to auditory stimuli, and auditory cortex responds selectively to certain somatosensory stimuli, supporting the possibility that complex visual signals may modulate early stages of auditory processing. To elucidate which auditory regions, if any, are responsive to complex visual stimuli, we recorded from auditory cortex and the superior temporal sulcus while presenting visual stimuli consisting of various objects, neutral faces, and facial expressions generated during vocalization. Both objects and conspecific faces elicited robust field potential responses in auditory cortex sites, but the responses varied by category: both neutral and vocalizing faces had a highly consistent negative component (N100) followed by a broader positive component (P180) whereas object responses were more variable in time and shape, but could be discriminated consistently from the responses to faces. The face response did not vary within the face category, i.e., for expressive vs. neutral face stimuli. The presence of responses for both objects and neutral faces suggests that auditory cortex receives highly informative visual input that is not restricted to those stimuli associated with auditory components. These results reveal selectivity for complex visual stimuli in a brain region conventionally described as non-visual “unisensory” cortex.
Highlights
The perception of communication signals is one example of multisensory integration that occurs in the daily life of social primates: both visual and auditory channels provide information through facial expressions and vocalizations, respectively (Ghazanfar and Logothetis, 2003; Izumi and Kojima, 2004; Jordan et al, 2005; Sumby and Pollack, 1954)
Activity from a total of 127 electrode sites in auditory cortex was analyzed for responsiveness to visual stimuli
All 127 electrode sites showed a significant local field potential (LFP) deviation from baseline in response to at least one of the two categories (t-test, p < 0.01), and 98% of sites (124/127) were category specific (t-test, p < 0.01)
Summary
The perception of communication signals is one example of multisensory integration that occurs in the daily life of social primates: both visual and auditory channels provide information through facial expressions and vocalizations, respectively (Ghazanfar and Logothetis, 2003; Izumi and Kojima, 2004; Jordan et al, 2005; Sumby and Pollack, 1954). Previous work has demonstrated that the auditory cortex responds to visual and/or somatosensory stimuli (Bizley et al, 2007; Schroeder and Foxe, 2002), consistent with anatomical connectivity (Bizley et al, 2007; Lewis and Van Essen, 2000) and fMRI results of visual stimulation in monkeys (Kayser et al, 2007) and humans (Martuzzi et al, 2007). The selectivity of neurons in primary and secondary auditory cortex to complex visual stimuli has been unexplored This absence is all the more striking, given the discrepant fMRI results of multisensory integration of speech signals in auditory cortex, mentioned above, and the recent exploration of multisensory effects for communication signals in monkey auditory cortex; namely, that auditory cortical sites integrate the auditory and visual components of species-typical vocalizations (Ghazanfar et al, 2005)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
