Visual intensity-dependent response latencies predict perceived audio–visual simultaneity

Abstract

To form a coherent presentation of the world, the brain needs to combine multiple sensory modalities accurately together in the temporal domain. Judgements on the relative timing of audio–visual stimuli are complex, due to the differing propagation speeds of light and sound through the environment and the nervous system, and the dependence of processing latencies on stimulus intensity (Piéron, 1913). Simultaneity judgement (SJ) and temporal order judgement (TOJ) tasks are often used to assess the temporal mechanisms underlying this binding process. However, these tasks consistently produce measures of perceived simultaneity that are uncorrelated with each other, leading to the suggestion that SJ and TOJ tasks could depend on separate neural mechanisms.Parise and Ernst’s (2016) multisensory correlation detector (MCD) model predicts this lack of correlation by assuming two internal processing stages, a lag computation and a correlation. Here we include and empirically evaluate an intensity-dependent processing delay in the MCD model.We estimate the points of subjective simultaneity (PSSs) using both SJ and TOJ tasks for four different visual intensities and a fixed auditory sound level. Evaluation of four variants of the intensity-dependent MCD model shows that the introduction of an early processing delay can predict the different PSS values obtained in the two respective tasks, without the need for later intensity-dependent multisensorial processing stages. Crucially, this early processing delay can be estimated from simple reaction times.