When the brain fails to recalibrate audiovisual simultaneity: Hysteresis in synchrony perception

Abstract

To accurately perceive the synchrony of visual and auditory events, the brain has to compensate for the speed differences between light and sound. This compensation may be accomplished by the mechanism of temporal recalibration (Fujisaki et al., 2004; Vroomen et al., 2004) a mechanism through which the subjects become more tolerant to constant audiovisual (AV) asynchrony by virtue of adaptation. However, a constant AV lag means that both the source and the observer are fixed in space; in an ecological setting, if a source is moving backward or away from the observer it will induce a progressive change in the AV delay. The present study shows that synchrony perception of slowly synchronizing AV stimuli is driven by a persistence effect or hysteresis (Hock et al., 1997). Specifically, when AV stimuli are progressively synchronizing, participants do not compensate for the initial asynchrony but rather persist in perceiving asynchrony; surprisingly however, slowly desynchronizing stimuli do not alter the perception of synchrony. Our results show that synchrony perception strongly depends on the dynamics of past AV stimulation. Whereas temporal recalibration effects suggest that the brain has a natural tendency to compensate for AV time lags emitted from a distant stationary source, our results suggest that compensation of AV lags for a moving source may not occur and perceived asynchrony persists.