Distinct mechanisms of temporal binding in generalized and cross-modal flash-lag effects

Ryusuke Hayashi,Ikuya Murakami

doi:10.1038/s41598-019-40370-7

Abstract

It remains unknown how the brain temporally binds sensory data across different modalities and attributes to create coherent perceptual experiences. To address this question, we measured what we see at the time we experience an event using a generalized version of the flash-lag effect (FLE) for combinations of visual attribute (bar orientation, face orientation, or face identity) and probe modality (visual or auditory). We asked participants to judge the content of rapidly and serially presented images seen at the same time as a briefly presented visual (flash) or auditory (click) probe and estimated the “time windows” contributing to decisions using reverse correlation analysis. We also used displays in which the visual attribute of a stimulus continuously changed and measured FLEs around abrupt flip in change direction and at the initiation and termination of a sequence. We consistently found clear latency-difference effects, which depended on visual attribute for the visual probe but did not for the auditory probe. The intra-modal FLE can be explained in terms of differential latency and temporal integration, but the cross-modal FLE is suggested to operate via a distinct mechanism; the content of a successive visual stream experienced after the awareness of a click is interpreted as simultaneous with the click.

Highlights

Our perceptual system is confronted with the difficult task of estimating how the external world develops in time
We used continuously changing visual stimuli to see whether the time window estimated via RSVP reflects a general property of the binding mechanism and explains perception in other situations in which visual attributes change smoothly and continuously, as in conventional flash-lag effect (FLE) studies
We examined whether the observed lag or lead effect was consistent with the observed peak variability in the RSVP paradigm

Summary

Introduction

Our perceptual system is confronted with the difficult task of estimating how the external world develops in time. As neurons respond faster to some stimuli than others, neural latency may influence the FLE (Krekelberg & Lappe[11]) It remains unclear how visual information is temporally integrated to construct the percept of a visual attribute at visual probe onset (intra-modal binding) or at auditory probe onset (cross-modal binding) despite differences in latency for processing. Reverse correlation analysis estimates the linear component (the first-order kernel) of this weight function by summing a randomly presented visual sequence conditional on an observer’s binary responses We refer to this temporal weight function as the “time window.”. We used continuously changing visual stimuli to see whether the time window estimated via RSVP reflects a general property of the binding mechanism and explains perception in other situations in which visual attributes change smoothly and continuously, as in conventional FLE studies. We examined differences between intra-modal and cross-modal FLEs using a visual flash and an auditory click

Methods

Results

Conclusion