Tactile capture of auditory localization: an event‐related potential study

Abstract

The well-known ventriloquist illusion arises when sounds are mislocalized towards a synchronous but spatially discrepant visual stimulus, and a similar effect of touch on audition has also been reported. By manipulating hand position, we recently demonstrated that this audiotactile ventriloquism effect predominantly operates in an external coordinate system. Using event-related potentials, the present study investigated the neural correlates of this audiotactile ventriloquism effect. Participants reported the perceived location of brief auditory stimuli that were presented either alone or with concurrent tactile stimuli to the fingertips, which were situated at the left and right side of the speaker array. Concurrent electroencephalogram recordings suggested a biasing of cortical activity by the tactile stimuli, which was only observed for trials in which a ventriloquist illusion was elicited. Irrespective of the physical location of the sound source in the bimodal trials, centrally perceived sounds elicited an enhanced centrally distributed negativity, compared to laterally perceived sounds, between 260 and 400 ms following stimulus onset, similar to unimodal auditory stimuli that were actually presented from central and lateral positions, respectively. Moreover, this effect was modulated by hand posture. When external and anatomically centred reference frames were in conflict, the event-related potential effect was reduced for small audiotactile spatial discrepancies, which corresponded to the behavioural finding of a reduced audiotactile ventriloquism effect compared to a parallel hand posture. The present data suggest that the cortical representation of auditory space is adjusted to coincide with spatially disparate tactile input.