Visually-guided auditory adaptation and reference frame of the ventriloquism after effect

Abstract

Ventriloquism aftereffect (VA) is observed as a shift in the perceived locations of auditory stimuli, induced by repeated presentation of audiovisual signals with incongruent locations of auditory and visual components. Since the two modalities use a different reference frame (RF), audition is head-centered (HC) while vision is eye-centered (EC), the representations have to be aligned. A previous study examining RF of VA found inconsistent results: the RF was a mixture of HC and EC for VA induced in the center of the audiovisual field, while it was predominantly HC for VA induced in the periphery [Lin et al., JASA 121, 3095, 2007]. In addition, the study found an adaptation in the auditory space representation even for congruent AV stimuli in the periphery. Here, a computational model examines the origins of these effects. The model assumes that multiple stages of processing interact: (1) the stage of auditory spatial representation (HC), (2) the stage of saccadic eye responses (EC), and (3) some stage at which the representation is mixed (HC + EC). Observed results are most consistent with a suggestion that the neural representation underlying spatial auditory plasticity incorporates both HC and EC auditory information, possibly at different processing stages. [Work supported by VEGA-1/1011/16.]