Mechanisms of Multisensory Integration in the Time Domain

Abstract

Information from different senses has to be combined in order to form a meaningful and coherent representation of our environment (e.g., Calvert, Spence, & Stein, 2004; Lalanne & Lorenceau, 2004). Because there is no dedicated sense organ for the perception of duration, researchers have been especially interested in understanding which mechanisms govern such a multimodal integration in the domain of time perception (e.g., Chen & Yeh, 2009; Klink, Montijn, & van Wezel, 2011; Romei, De Haas, Mok, & Driver, 2011). In a series of experiments, we investigated to what extent visual and auditory temporal information contribute to a combined temporal percept, and we show that this mutual influence is presumably determined by low-level influences on the perceptual processing stage rather than controlled, decisional processes (Bausenhart, De la Rosa, & Ulrich, in press). Furthermore, we demonstrated that incongruent audiovisual durations seem to be integrated through the occurrence of a temporal ventriloquism effect (De la Rosa & Bausenhart, 2013). In terms of the pacemaker-accumulator model of time perception, this points to an involvement of the switch component rather than to a modulation of the pacemaker rate by incongruent audiovisual information. Moreover, not only temporally incongruent, but also spatially incongruent multimodal information seems to have an influence on perceived duration. Specifically, visuo-spatial information can evoke a Kappa effect (i.e., prolonged perceived duration for intervals defined by spatially separated signals, cf. Cohen, Hansel, & Sylvester, 1953) in the perception of auditory temporal intervals. Taken together, perceived duration is affected by temporal and spatial information from different modalities, which is effectively, flexibly, and rather automatically combined, in order to deliver a coherent and stable perceptual impression.