Visual cues release the temporal coherence of auditory objects in auditory scene analysis

Abstract

Auditory scene analysis can arrange alternating tones of high and low pitch in a single, integrated melody, or in two parallel, segregated melodies, depending on the presentation rate and pitch contrast of the tones. We conducted an electrophysiological experiment to determine whether an inherently stable sound organization can be altered by a synchronous presentation of visual cues. To this end, two tones with different frequencies were presented in alternation. Frequency distance was selected as narrow or wide, inducing an inherently stable integrated or segregated organization, respectively. To modulate the integration or segregation organization, visual stimuli were synchronized to either the within-set frequency pattern or with a superimposed intensity pattern. Occasional deviations of the regular frequency pattern were introduced. Elicitation of the mismatch negativity (MMN) component of event-related brain potentials by these deviants indexed the presence of a segregated organization. MMN was elicited by tone sequences with wide frequency distance irrespective of the presence of visual cues. At a narrow frequency distance, however, an MMN was elicited when the visual pattern promoted segregation of the sounds showing a release of the inherently stable integrated organization due to visual stimulation. The results demonstrate cross-modal effects on auditory object perceptual organization even on an inherently stable auditory organization.