Utilizing the ventriloquism-effect to investigate audio-visual binding

Abstract

Audio-visual binding – as subset of crossmodal integration – describes the combination of information across both these senses to the subjective unified perception of a bound object. We investigated audio-visual binding by using the ventriloquism-effect (localization of a sound is biased towards and by a simultaneous visual stimulus) to act as an indicator for perceived binding. Simple visual and auditory stimuli were presented synchronously or asynchronously. fMRI was recorded during task performance ( n = 19 subjects) in order to reveal activation in areas discussed to be involved in multisensory processing in the literature. Contrasting trials with reported ventriloquism-effect versus the no-binding condition revealed activation in the insula, superior temporal sulcus and parieto-occipital sulcus. Implementing the ventriloquism-effect allows us to relate these activations to consciousness-related processes, which probably are different from stimulus-driven multisensory integration in subcortical areas.