Event Related Potentials Index Rapid Recalibration to Audiovisual Temporal Asynchrony.

David M Simon,Jean-Paul Noel,Mark T Wallace

doi:10.3389/fnint.2017.00008

Abstract

Asynchronous arrival of multisensory information at the periphery is a ubiquitous property of signals in the natural environment due to differences in the propagation time of light and sound. Rapid adaptation to these asynchronies is crucial for the appropriate integration of these multisensory signals, which in turn is a fundamental neurobiological process in creating a coherent perceptual representation of our dynamic world. Indeed, multisensory temporal recalibration has been shown to occur at the single trial level, yet the mechanistic basis of this rapid adaptation is unknown. Here, we investigated the neural basis of rapid recalibration to audiovisual temporal asynchrony in human participants using a combination of psychophysics and electroencephalography (EEG). Consistent with previous reports, participant’s perception of audiovisual temporal synchrony on a given trial (t) was influenced by the temporal structure of stimuli on the previous trial (t−1). When examined physiologically, event related potentials (ERPs) were found to be modulated by the temporal structure of the previous trial, manifesting as late differences (>125 ms post second-stimulus onset) in central and parietal positivity on trials with large stimulus onset asynchronies (SOAs). These findings indicate that single trial adaptation to audiovisual temporal asynchrony is reflected in modulations of late evoked components that have previously been linked to stimulus evaluation and decision-making.

Highlights

Events in the natural environment generate signals in different sensory modalities, and combining these signals through multisensory integration can confer substantial behavioral and perceptual benefits (Murray and Wallace, 2012)
We extended this procedure to individual stimulus onset asynchronies (SOAs) by fitting distributions according to the specific SOA on trial t−1
Previous work has indicated that recalibration to temporal asynchrony in audiovisual events occurs on both prolonged (Fujisaki et al, 2004; Vroomen et al, 2004) and rapid (Van der Burg et al, 2013) timescales, and that these processes are behaviorally dissociable (Van der Burg et al, 2015)

Summary

Introduction

Events in the natural environment generate signals in different sensory modalities, and combining these signals through multisensory integration can confer substantial behavioral and perceptual benefits (Murray and Wallace, 2012). An example of such facilitation can be found, for instance, for speech signals, in which the presence of visual cues (i.e., lip movements) substantially improves speech intelligibility (Sumby and Pollack, 1954; Ross et al, 2007). Previous work has demonstrated such recalibration after prolonged exposure to a fixed temporal asynchrony (Fujisaki et al, 2004; Vroomen et al, 2004). This phenomenon of temporal recalibration manifests as a shift of the point of subjective simultaneity (PSS) in the direction of the presented asynchrony

Methods

Results

Conclusion