Recently, we have shown that perceived duration of visual intervals can be biased by the duration of concurrently presented auditory intervals with conflicting duration (De la Rosa & Bausenhart, 2013). We suggested that this multimodal biasing effect might be due to temporal ventriloquism (that is, the onset/offset of the visual interval gets ‘pulled’ towards the onset/offset of the auditory interval), which affects the switch component of a pacemaker-accumulator model (Gibbon, 1977). On the other hand, for filled audiovisual intervals with congruent duration, it has been suggested that multimodal integration leads to an increased pacemaker rate of an internal clock (Chen & Yeh, 2009; Walker & Scott, 1981). Importantly, both suggested mechanisms, temporal ventriloquism and pacemaker rate effect, are based on early, perceptual processes. Alternatively, however, it is possible that participants deliberately judge the visual interval duration according to the irrelevant auditory information, and, thus, the observed multimodal integration effects would emerge as the result of a decision process on a relatively late stage of processing. Therefore, in order to distinguish these two mechanisms, we propose an even-related brain potential (ERP) study with the aim to measure the mismatch negativity component (MMN; Näätänen & Alho, 1995). This component mirrors the detection of deviance at an early stage of processing and, thus, indicates underlying pre-attentive and automatic mechanisms. Interestingly, previous studies have already investigated the multimodal integration underlying the spatial ventriloquism effect by assessing the MMN (Stekelenburg, Vroomen, & de Gelder, 2004). It has been found that illusory auditory location shifts evoked by real visual location shifts produced an auditory MMN in the auditory cortex. Therefore, it was concluded that the multimodal integration underlying spatial ventriloquism takes place at a pre-attentive level of processing. Similarly, we will investigate the MMN response for conflicting audiovisual interval durations. To this end, our study consists of repeatedly presenting simultaneous audiovisual intervals with a standard duration, among which rare stimuli of deviant duration are randomly interspersed. Deviants can then occur in: (a) only the auditory modality, (b) only the visual modality, or (c) in both modalities. Additionally, unimodal visual and auditory standards and deviants will be presented as control conditions. We expect that the unimodal auditory and visual deviants evoke an auditory MMN in the auditory cortex and a visual MMN in occipital areas, respectively. Similarly, audiovisual deviants should simultaneously elicit a visual and an auditory MMN in their corresponding areas, probably with an increased magnitude compared to the unimodal conditions. Most interesting are the predictions for conflicting audiovisual intervals: If the multimodal integration of duration is an early perceptual process, auditory-only deviants in bimodal trials should bias perceived duration of the visual stimulation, and, thus, a visual MMN in occipital areas should be observed. Because auditory temporal information mostly dominates over visual temporal information, visual-only deviants in bimodal trials should be perceived as similar to the auditory (standard) duration and thus, the visual MMN in occipital areas should even be reduced compared to the unimodal visual MMN. However, if the multimodal integration of perceived duration results from a decision process on a relatively late stage of processing, no MMN for biased perceived duration would be expected, and it should just be evoked by deviant modalities with a real duration shift.
Read full abstract