A neural basis for the synthesis of sound, sight, and touch

Abstract

That visual cues affect speech perception has been known for some time, but is only one part of the much broader phenomenon of cross‐modal integration. This fundamental characteristic of the CNS has a profound influence on perception and behavior and is evident not only in speech perception, but in the facilitated detection, identification, and reaction to combinations of concordant cues from different modalities, and in the striking perceptual anomalies that can occur when these cues are discordant. Understanding the mechanisms by which these functions are achieved requires a fundamental knowledge of the neural bases of multisensory integration; specifically, the circuits involved and the principles by which such ‘‘multisensory’’ neurons synthesize their convergent sensory information. Perhaps the best known site of multisensory convergence is the superior colliculus, a midbrain structure involved in attentive and orientation functions. The spatial, temporal, and receptive field characteristics of its constituent multisensory neurons serve as an excellent model for understanding the neural principles of multisensory integration throughout the CNS and for predicting overt behavior. It will be these multisensory circuits, the neural principles by which they affect multisensory integration, and the impact of this integration on overt behavior which will be emphasized in this discussion.