The effects of reverberation on the spatio-temporal synchrony of moving audio-visual stimuli

Abstract

The current study investigated the temporal dynamics of audio-visual moving targets, examining the effects of velocity and acoustical environmental cues on the sensitivity of audio-visual cohesion. The acoustical stimuli were generated using Virtual Auditory Space, and two types of moving sounds were presented: (1) an immersive stimulus using Binaural Room Impulse Responses (BRIRs), that were recorded for each subject in situ in the testing room; thereby containing the combined spectral and temporal acoustic transformation due to both head-related filtering and the room’s characteristic response; (2) an environmentally impoverished stimulus using individualised Head-Related Impulse Responses (HRIRs) recorded in an anechoic chamber that lacked any reverberant cues. Apparent visual motion was generated by the sequential activation of LED’s along the same radial path as the auditory motion, 1 m away from the subjects. Moving stimuli were presented with a random temporal offset between each stimuli ranging from [−400 ms, 400 ms] (auditory or visual leading). Using a 2AFC paradigm, subjects reported which modality they perceived as leading for velocities of 25, 50 and 100°/s. At 25°/s the stimuli were perceived to be coincident when the visual stimulus was actually leading but, counter intuitively, the offset decreased as a function of increased velocity. The offset at PSE was greater for HRIRs than for BRIRs but the psychometric functions were significantly steeper for the HRIRs suggesting greater spatio-temporal resolution. This may reflect the degradation of the auditory cues produced by reverberation.