Rapid discrimination of visual and multisensory memories revealed by electrical neuroimaging

Abstract

Though commonly held that multisensory experiences enrich our memories and that memories influence ongoing sensory processes, their neural mechanisms remain unresolved. Here, electrical neuroimaging shows that auditory–visual multisensory experiences alter subsequent processing of unisensory visual stimuli during the same block of trials at early stages poststimulus onset and within visual object recognition areas. We show this with a stepwise analysis of scalp-recorded event-related potentials (ERPs) that statistically tested (1) ERP morphology and amplitude, (2) global electric field power, (3) topographic stability of and changes in the electric field configuration, and (4) intracranial distributed linear source estimations. Subjects performed a continuous recognition task, discriminating repeated vs. initial image presentations. Corresponding, but task-irrelevant, sounds accompanied half of the initial presentations during a given block of trials. On repeated presentations within a block of trials, only images appeared, yielding two situations—the image's prior presentation was only visual or with a sound. Image repetitions that had been accompanied by sounds yielded improved memory performance accuracy (old or new discrimination) and were differentiated as early as ∼ 60–136 ms from images that had not been accompanied by sounds through generator changes in areas of the right lateral–occipital complex (LOC). It thus appears that unisensory percepts trigger multisensory representations associated with them. The collective data support the hypothesis that perceptual or memory traces for multisensory auditory–visual events involve a distinct cortical network that is rapidly activated by subsequent repetition of just the unisensory visual component.