Effects of within-modal congruency, cross-modal congruency and temporal asynchrony on the perception of perceived audio–visual distance

Abstract

The factors we use to determine whether information from separate modalities should be assigned to the same source include task demands, the spatial and temporal coincidence of the composite signals, and, whether the signals are congruent with one another. In a series of experiments, we examined how temporal asynchrony and congruency interact in a competitive binding situation. Across a series of experiments, participants assigned a temporally roving auditory stimulus to competing primary or secondary visual anchors (VAV), or, a temporally roving visual stimulus to competing primary or secondary auditory anchors (AVA), based on causality. Congruency was defined in terms of simulated distance both within- and between-modalities (visual: small, auditory: quiet = far; visual: large, auditory: loud = near). Strong temporal effects were revealed, with differences between VAV and AVA conditions reflecting natural auditory lag tolerance for binding. During VAV conditions, binding was influenced only by visual congruency. During AVA conditions, binding was influenced by audio–visual congruency. These differences did not seem to be due to the relative discriminability between visual and auditory magnitude. The data reiterate the dominance of audition in the time domain (showing stronger temporal effects), the dominance of vision in the spatial domain (showing stronger congruency effects), and, the assistance of domain-inappropriate modalities by domain-appropriate modalities. A special case of congruency in terms of visual looming will also be discussed, along with the potential alerting properties of high magnitude stimuli.