Cross-modal visual–auditory–somatosensory integration in a multimodal object recognition task in humans

Abstract

EEG was recorded in a visual–auditory–somatosensory oddball reaction time task to study the relationship of cortical cross-modal processing and reaction time. Visual, auditory and somatosensory stimuli were presented alone and simultaneously in four experimental sessions. Target stimuli were applied in the visual modality to study cross-modal effects on object recognition. Subjects' task was to indicate the recognition of the target by pressing a button. EEG was recorded from 31 scalp electrodes. Significant decrease in reaction time confirmed that multisensory integration took place in multimodal stimulus condition. Recognition of the target was significantly improved in audiovisual and audio-somatosensory–visual conditions reflected by significantly decreased reaction time compared with unimodal visual and somatosensory–visual conditions. Analysis of event-related potentials revealed that P300 latency showed clear relationship to behavioral data. Results indicate that audiovisual cross-modal integration is more efficacious in visual object recognition task than somatosensory–visual integration.