Synchronized audio-visual transients drive efficient visual search for motion-in-depth

Abstract

In natural audio-visual environments, a change in depth is usually correlated with a change in loudness. In the present study, we investigated whether correlating disparity and loudness provides a functional advantage in binding disparity and sound amplitude in a visual search paradigm. To test this assumption, we used a method similar to that used by Van der Burg et al. (2008) to show that non-spatial modulations of loudness can drastically improve spatial visual search for a correlated luminance modulation. Subsequently (2010), they varied the shape of temporal modulation and demonstrated that transient events (square modulations) are required for this search efficiency, and that sinusoidal audiovisual modulations do not support efficient search. We used dynamic random-dot stereogram displays to produce pure disparity modulations. Target and distractors were 0.35 × 0.35 degrees disparity-defined squares (either 6 or 10 in total) presented on a ring at 2.5 deg eccentricity. Each square moved back and forth in depth from zero to +12 arcmin (crossed) disparity at different phases. The target's depth modulation was synchronized with an amplitude-modulated 500 Hz tone. Visual and auditory modulations were always congruent (both sinewave or squarewave). Four observers were asked to give speeded responses in a discrimination task on the target. Because binocular matching processes are known to favor smooth over abrupt changes of disparity across space and time, we expected the sine modulation condition to be at least as efficient as the square modulation in supporting efficient search. However, results show a significant improvement in visual search in the square condition compared to the sine condition, suggesting that transient auditory information can efficiently drive visual search in the disparity domain. In a second experiment, correlating sound with a distractor led to longer search times, indicating that the correlation is not easily ignored.