Footstep sounds increase sensitivity to point-light walking when visual cues are weak

Abstract

Multisensory neurons in superior colliculus display inverse effectiveness (IE); responses to multimodal stimuli are greater when the most effective unisensory stimulus produces a weak response (e.g. Meredith & Stein, 1983). Recently, fMRI evidence for inverse effectiveness has been found in superior temporal sulcus (Stevenson & James, 2009), a region implicated in multisensory integration (e.g. Barraclough et al., 2009) and human action perception (e.g. Blake & Shiffrar, 2008). While the relationship between neural IE and multisensory perception and performance remains unclear, and evidence suggests that not all multimodal processes adhere strictly to the IE rule (Holmes, 2007), behavioral studies have shown that the effect of auditory cues on visual processing is greater when the visual signal is relatively weak (e.g., Collignon et al., 2008). Previously, we found that sensitivity to point-light walkers improves when paired with footstep sounds. Because sounds' influence on vision depends upon the strength of the visual signal, we hypothesized that heard footsteps should increase detection sensitivity only when the visual stimulus is weak. To test this hypothesis, we analyzed data from Thomas & Shiffrar (2010) to see whether the detectability of the visual signal determined the degree to which footstep sounds improved detection of point-light walkers. We obtained percent correct for each movie presented in silence (range 33–100%). We then split all movies into two groups based on difficulty. A 3 × 2 mixed measures ANOVA with sound condition as the between-subjects factor (silence, tones, footsteps) and difficulty as the within-subjects factor (easy vs. hard) revealed a main effect of sound, difficulty and a sound X difficulty interaction. Subsequent analyses revealed that footsteps improved visual sensitivity only for “hard” movies, supporting our hypothesis. A follow-up experiment replicated and extended these results.