Locating the sources for cross-modal interactions and decision making during judging the visual-affected auditory intensity change

Abstract

Audiovisual interaction has been one of the most important topics in cognitive neurosciences. Visual stimuli could significantly impact the auditory perception, and vice versa. Nevertheless, how much the change in visual stimuli would influence the perception of auditory change remains to be investigated. In this paper, we designed an audiovisual experiment in which subjects were required to judge whether there is a change in the intensities of two sounds with 150 ms interval, while there are two simultaneously presented size-changed visual stimuli. Behavioral results demonstrated that incongruent audiovisual change could result in the illusory perception of the change in sound intensity. For the correctly judged trials, source analysis showed two characteristic windows post the first auditory stimulus, i.e., (i) the 160-200 ms window including the auditory P200 and visual N100 wave, which was related to audiovisual interaction and working memory of the first stimulus with localized sources in insula and agranular retrolimbic area; and (ii) the 300-400 ms window for P300 with sources in premotor cortex and caudate nucleus, which were related to later audiovisual interaction, change discrimination and working memory. These preliminary results implied two stages in the audiovisual change perception task, with the involvement of insula, agranular retrolimbic, premotor cortex and caudate nucleus.