The effects of temporal synchrony and visual material on audio-visual integration

Abstract

Event Abstract Back to Event The effects of temporal synchrony and visual material on audio-visual integration Maria Mittag1*, R. Takegata1 and T. Kujala1 1 Cognitive Brain Research Unit, University of Helsinki, Finland Associating letters with speech sounds is essential when learning to read. Recent studies on audio-visual integration suggested that 1) the integration of letter and speech sound takes place automatically and 2) that temporal synchrony seems to be critical for the integration. In the current study, we wished to determine the neural networks associated with an automatic integration of written and heard syllables in literate adults. The mismatch negativity (MMN) was recorded in a new multi-feature paradigm whereby the integration of several speech sounds with letters can be addressed within one experimental session. Subjects were presented with auditory stimuli (consonant-vowel syllables) together with visual stimuli, which differed between conditions. The auditory stimuli included vowel or consonant changes, or changes in syllable intensity, frequency, or vowel length. In the experimental condition the visual stimuli were written syllables, whereas in the control condition, they were scrambled images of the written syllables. The visual stimuli were presented in half of the blocks synchronously with the auditory stimuli and in the other half with a time delay. During the stimulation, the subjects pressed a button to designated visual targets. Changes in auditory stimuli elicited a MMN/N2 response, in all conditions. The topography of this response was different in the stimulus blocks including syllables than in those including scrambled images. Furthermore, topographic differences were observed between the conditions with and without the time delay. The results indicated that different neural networks are activated when speech stimuli are presented together with linguistic than with non-linguistic visual stimuli and that this effect seems to be influenced by temporal synchrony. Conference: MMN 09 Fifth Conference on Mismatch Negativity (MMN) and its Clinical and Scientific Applications, Budapest, Hungary, 4 Apr - 7 Apr, 2009. Presentation Type: Poster Presentation Topic: Poster Presentations Citation: Mittag M, Takegata R and Kujala T (2009). The effects of temporal synchrony and visual material on audio-visual integration. Conference Abstract: MMN 09 Fifth Conference on Mismatch Negativity (MMN) and its Clinical and Scientific Applications. doi: 10.3389/conf.neuro.09.2009.05.145 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 26 Mar 2009; Published Online: 26 Mar 2009. * Correspondence: Maria Mittag, Cognitive Brain Research Unit, University of Helsinki, Helsinki, Finland, maria.mittag@helsinki.fi Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Maria Mittag R. Takegata T. Kujala Google Maria Mittag R. Takegata T. Kujala Google Scholar Maria Mittag R. Takegata T. Kujala PubMed Maria Mittag R. Takegata T. Kujala Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.