Temporal Binding Research Articles

Investigating the Role of Leading Sensory Modality and Autistic Traits in the Visual-Tactile Temporal Binding Window.

Our ability to integrate multisensory information depends on processes occurring during the temporal binding window. There is limited research investigating the temporal binding window for visual-tactile integration and its relationship with autistic traits, sensory sensitivity, and unusual sensory experiences. We measured the temporal binding window for visual-tactile integration in 27 neurotypical participants who completed a simultaneity judgement task and three questionnaires: the Autism Quotient, the Glasgow Sensory Questionnaire, and the Multi-Modality Unusual Sensory Experiences Questionnaire. The average width of the visual-leading visual-tactile (VT) temporal binding window was 123ms, significantly narrower than the tactile-leading visual-tactile (TV) window (193ms). When comparing crossmodal (visual-tactile) stimuli with unimodal (visual-visual or tactile-tactile), the temporal binding window was significantly larger for crossmodal stimuli (VT: 123ms; TV: 193ms) than for unimodal pairs of stimuli (visual: 38ms; tactile 42ms). We did not find evidence to support a relationship between the size of the temporal binding window and autistic traits, sensory sensitivities, or unusual sensory perceptual experiences in this neurotypical population. Our results indicate that the leading sense presented in a multisensory pair influences the width of the temporal binding window. When tactile stimuli precede visual stimuli it may be difficult to determine the temporal boundaries of the stimuli, which leads to a delay in shifting attention from tactile to visual stimuli. This ambiguity in determining temporal boundaries of stimuli likely influences our ability to decide on whether stimuli are simultaneous or nonsimultaneous, which in turn leads to wider temporal binding windows.

Alpha-band sensory entrainment improves audiovisual temporal acuity.

Visual and auditory stimuli are transmitted from the environment to sensory cortices with different timing, requiring the brain to encode when sensory inputs must be segregated or integrated into a single percept. The probability that different audiovisual (AV) stimuli are integrated into a single percept even when presented asynchronously is reflected in the construct of temporal binding window (TBW). There is a strong interest in testing whether it is possible to broaden or shrink TBW by using different neuromodulatory approaches that can speed up or slow down ongoing alpha oscillations, which have been repeatedly hypothesized to be an important determinant of the TBWs size. Here, we employed a web-based sensory entrainment protocol combined with a simultaneity judgment task using simple flash-beep stimuli. The aim was to test whether AV temporal acuity could be modulated trial by trial by synchronizing ongoing neural oscillations in the prestimulus period to a rhythmic sensory stream presented in the upper (∼12 Hz) or lower (∼8.5 Hz) alpha range. As a control, we implemented a nonrhythmic condition where only the first and the last entrainers were employed. Results show that upper alpha entrainment shrinks AV TBW and improves AV temporal acuity when compared with lower alpha and control conditions. Our findings represent a proof of concept of the efficacy of sensory entrainment to improve AV temporal acuity in a trial-by-trial manner, and they strengthen the idea that alpha oscillations may reflect the temporal unit of AV temporal binding.

Distinct profiles of multisensory processing between professional goalkeepers and outfield football players

In association football (soccer), the position of goalkeeper is the most specialised position in the sport and has the primary objective of stopping the opposing team from scoring. While previous studies have highlighted differences in physiological and match performance profiles between goalkeepers and outfield players1, surprisingly little research has focused on whether goalkeepers differ in terms of their perceptual-cognitive abilities. Given that goalkeepers use multiple sensory cues and are often required to make rapid decisions based on incomplete multisensory information to fulfil their role2, we hypothesised that professional goalkeepers would display enhanced multisensory temporal processing relative to their outfield counterparts. To test this hypothesis, we measured the temporal binding windows - the time window within which signals from the different senses are integrated into a single percept - of professional goalkeepers, professional outfield players, and a control group with no professional football experience using the sound-induced flash illusion3. Our results indicated a marked difference in multisensory processing between the three groups. Specifically, we found that the goalkeepers displayed a narrower temporal binding window relative to both outfielders and control participants, indicating more precise audiovisual timing estimation. However, this enhanced multisensory temporal processing was accompanied by a general reduction in crossmodal interactions relative to the other two groups that could be attributed to an a priori tendency to segregate sensory signals. We propose that these differences stem from the idiosyncratic nature of the goalkeeping position that puts a premium on the ability of goalkeepers to make quick decisions, often based on partial or incomplete sensory information.

When 2 become 1: Autistic simultaneity judgements about asynchronous audiovisual speech.

It has been proposed that autistic people experience a temporal distortion whereby the temporal binding window of multisensory integration is extended. Research to date has focused on autistic children so whether these differences persist into adulthood remains unknown. In addition, the possibility that the previous observations have arisen from between-group differences in response bias, rather than perceptual differences, has not been addressed. Participants completed simultaneity judgements of audiovisual speech stimuli across a range of stimulus-onset asynchronies. Response times and accuracy data were fitted to a drift-diffusion model so that the drift rate (a measure of processing efficiency) and starting point (response bias) could be estimated. In Experiment 1, we tested a sample of non-autistic adults who completed the Autism Quotient questionnaire. Autism Quotient score was not correlated with either drift rate or response bias, nor were there between-group differences when splitting based on the first and third quantiles of scores. In Experiment 2, we compared the performance of autistic with a group of non-autistic adults. There were no between-group differences in either drift rate or starting point. The results of this study do not support the previous suggestion that autistic people have an extended temporal binding window for audiovisual speech. In addition, exploratory analysis revealed that operationalising the temporal binding window in different ways influenced whether a group difference was observed, which is an important consideration for future work.

Neural oscillations reflect the individual differences in the temporal perception of audiovisual speech.

Multisensory integration occurs within a limited time interval between multimodal stimuli. Multisensory temporal perception varies widely among individuals and involves perceptual synchrony and temporal sensitivity processes. Previous studies explored the neural mechanisms of individual differences for beep-flash stimuli, whereas there was no study for speech. In this study, 28 subjects (16 male) performed an audiovisual speech/ba/simultaneity judgment task while recording their electroencephalography. We examined the relationship between prestimulus neural oscillations (i.e. the pre-pronunciation movement-related oscillations) and temporal perception. The perceptual synchrony was quantified using the Point of Subjective Simultaneity and temporal sensitivity using the Temporal Binding Window. Our results revealed dissociated neural mechanisms for individual differences in Temporal Binding Window and Point of Subjective Simultaneity. The frontocentral delta power, reflecting top-down attention control, is positively related to the magnitude of individual auditory leading Temporal Binding Windows (auditory Temporal Binding Windows; LTBWs), whereas the parieto-occipital theta power, indexing bottom-up visual temporal attention specific to speech, is negatively associated with the magnitude of individual visual leading Temporal Binding Windows (visual Temporal Binding Windows; RTBWs). In addition, increased left frontal and bilateral temporoparietal occipital alpha power, reflecting general attentional states, is associated with increased Points of Subjective Simultaneity. Strengthening attention abilities might improve the audiovisual temporal perception of speech and further impact speech integration.

Increased temporal binding during voluntary motor task under local anesthesia

Temporal binding refers to a systemic bias in the perceived time interval between two related events, most frequently voluntary motor actions and a subsequent sensory effect. An inevitable component of most instrumental motor actions is tactile feedback. Yet, the role of tactile feedback within this phenomenon remains largely unexplored. Here, we used local anesthesia of the index finger to temporarily inhibit incoming sensory input from the finger itself, while participants performed an interval-estimation task in which they estimated the delay between a voluntary motor action (button press) and a second sensory event (click sound). Results were compared to a control condition with intact sensation. While clear binding was present in both conditions, the effect was significantly enhanced when tactile feedback was temporarily removed via local anesthesia. The results are discussed in light of current debates surrounding the underlying mechanisms and function of this temporal bias.

Age-related differences in temporal binding and the influence of action body parts.

If voluntary action is followed by an effect with a short time delay, the time interval between action and effect is often perceived to be shorter than it actually is. This perceptual time compression is termed intentional binding or temporal binding. We investigated age-related changes in adulthood considering temporal binding and its dependence on action body parts (i.e., hand vs. foot). This experiment included 17 young adults (mean age: 21.71 ± 3.14 years) and 27 older adults (mean age: 74.41 ± 3.38 years). Participants performed a button press task using their index fingers (hand condition) or toes (foot condition). The results showed that older participants exhibited a strong time compression comparable to young participants in the voluntary condition. Older participants also showed a strong time compression in involuntary action, which was induced by a mechanical device, differently from young participants. In line with previous research, the present age-related differences in time compression considering involuntary action suggest that causal belief significantly influences event perception rather than the associated intention of action or sensory afferents. The present results also suggest that the nature of action body parts has no significant influence on temporal binding, independent of age group.

Development of audio-tactile temporal binding with and without vision

Development of audio-tactile temporal binding with and without vision

Age-related changes to the attentional modulation of temporal binding.

During multisensory integration, the time range within which visual and auditory information can be perceived as synchronous and bound together is known as the temporal binding window (TBW). With increasing age, the TBW becomes wider, such that older adults erroneously, and often dangerously, integrate sensory inputs that are asynchronous. Recent research suggests that attentional cues can narrow the width of the TBW in younger adults, sharpening temporal perception and increasing the accuracy of integration. However, due to their age-related declines in attentional control, it is not yet known whether older adults can deploy attentional resources to narrow the TBW in the same way as younger adults. This study investigated the age-related changes to the attentional modulation of the TBW. Thirty younger and 30 older adults completed a cued-spatial-attention version of the stream-bounce illusion, assessing the extent to which the visual and auditory stimuli were integrated when presented at three different stimulus-onset asynchronies, and when attending to a validly cued or invalidly cued location. A 2 × 2 × 3 mixed ANOVA revealed that when participants attended to the validly cued location (i.e., when attention was present), susceptibility to the stream-bounce illusion decreased. However, crucially, this attentional manipulation significantly affected audiovisual integration in younger adults, but not in older adults. These findings suggest that older adults have multisensory integration-related attentional deficits. Directions for future research and practical applications surrounding treatments to improve the safety of older adults' perception and navigation through the environment are discussed.

Age-Related Changes to Multisensory Integration and Audiovisual Speech Perception.

Multisensory integration is essential for the quick and accurate perception of our environment, particularly in everyday tasks like speech perception. Research has highlighted the importance of investigating bottom-up and top-down contributions to multisensory integration and how these change as a function of ageing. Specifically, perceptual factors like the temporal binding window and cognitive factors like attention and inhibition appear to be fundamental in the integration of visual and auditory information-integration that may become less efficient as we age. These factors have been linked to brain areas like the superior temporal sulcus, with neural oscillations in the alpha-band frequency also being implicated in multisensory processing. Age-related changes in multisensory integration may have significant consequences for the well-being of our increasingly ageing population, affecting their ability to communicate with others and safely move through their environment; it is crucial that the evidence surrounding this subject continues to be carefully investigated. This review will discuss research into age-related changes in the perceptual and cognitive mechanisms of multisensory integration and the impact that these changes have on speech perception and fall risk. The role of oscillatory alpha activity is of particular interest, as it may be key in the modulation of multisensory integration.

When did it happen? Verbal information about causal relations affects time estimation

Usually, the closer two events occur, the more likely people infer a causal relationship between them. Recent studies have shown that this relationship between time and causality is bidirectional. Participants also tend to judge events closer in time if they assume that they are causally related. We present six experiments showing causal binding, but unlike other experiments, participants do not emit any motor action, and no physical feedback is given. Rather, all stimuli and causal information are provided verbally. After reading a list of events, participants were asked to estimate the time elapsed between two of them. Those participants who were informed that there was a causal relationship between the two events estimated them as occurring closer to each other. These results support causality- and heuristic-based explanations of temporal binding, as opposed to other explanations such as sensory integration or intentionality of action.

Change of rapid temporal recalibration magnitude for audiovisual asynchrony with modulation of temporal binding window width: A preliminary investigation.

The subjective synchrony perception for audiovisual stimuli is affected by previous temporal information. The point of subjective simultaneity is shifted toward the same asynchronous direction of audiovisual stimuli in a previous trial. This phenomenon is called "rapid temporal recalibration." The factors that modulate the magnitude of rapid temporal recalibration have not been fully investigated. Previously, a positive correlation has been found between the magnitude of rapid temporal recalibration and the width of the temporal binding window (TBW). This preliminary study examined the causal relationship between TBW size and rapid recalibration magnitude using a single experimental group comparison design. In this experiment, the magnitude of rapid recalibration was compared before and after perceptual training, which narrowed the TBW width. The results indicated that the magnitude of rapid recalibration was reduced by perceptual training. Therefore, it was speculated that TBW size determined the magnitude of rapid recalibration. This causal relationship helps elucidate the mechanisms of the adaptation for temporal lags between visual and auditory sensations.

Age-Related Changes in Temporal Binding Involving Auditory and Vestibular Inputs.

Maintaining balance involves the combination of sensory signals from the visual, vestibular, proprioceptive, and auditory systems. However, physical and biological constraints ensure that these signals are perceived slightly asynchronously. The brain only recognizes them as simultaneous when they occur within a period of time called the temporal binding window (TBW). Aging can prolong the TBW, leading to temporal uncertainty during multisensory integration. This effect might contribute to imbalance in the elderly but has not been examined with respect to vestibular inputs. Here, we compared the vestibular-related TBW in 13 younger and 12 older subjects undergoing 0.5 Hz sinusoidal rotations about the earth-vertical axis. An alternating dichotic auditory stimulus was presented at the same frequency but with the phase varied to determine the temporal range over which the two stimuli were perceived as simultaneous at least 75% of the time, defined as the TBW. The mean TBW among younger subjects was 286 ms (SEM ± 56 ms) and among older subjects was 560 ms (SEM ± 52 ms). TBW was related to vestibular sensitivity among younger but not older subjects, suggesting that a prolonged TBW could be a mechanism for imbalance in the elderly person independent of changes in peripheral vestibular function.

Incidental temporal binding in rats: A novel behavioral task.

We designed a behavioral task called One-Trial Trace Escape Reaction (OTTER), in which rats incidentally associate two temporally discontinuous stimuli: a neutral acoustic cue (CS) with an aversive stimulus (US) which occurs two seconds later (CS-2s-US sequence). Rats are first habituated to two similar environmental contexts (A and B), each consisting of an interconnected dark and light chamber. Next, rats experience the CS-2s-US sequence in the dark chamber of one of the contexts (either A or B); the US is terminated immediately after a rat escapes into the light chamber. The CS-2s-US sequence is presented only once to ensure the incidental acquisition of the association. The recall is tested 24 h later when rats are presented with only the CS in the alternate context (B or A), and their behavioral response is observed. Our results show that 59% of the rats responded to the CS by escaping to the light chamber, although they experienced only one CS-2s-US pairing. The OTTER task offers a flexible high throughput tool to study memory acquired incidentally after a single experience. Incidental one-trial acquisition of association between temporally discontinuous events may be one of the essential components of episodic memory formation.

Target detection does not influence temporal memory.

Target detection has been found to enhance memory for concurrently presented stimuli under dual-task conditions. This "attentional boost effect" is reminiscent of findings in the event memory literature, where conditions giving rise to event boundaries have been shown to enhance memory for boundary items. Target detection commonly requires a working memory update (e.g., adding to a covert mental target count), which is also thought to be a key contributor to creating event boundaries. However, whether target detection impacts temporal memory in similar ways as event boundaries remains unknown, because these two parallel literatures have used different types of memory tests, making direct comparisons difficult. In a preregistered experiment with sequential Bayes factor design, we examined whether target detection influences temporal binding between items by inserting target and nontarget stimuli during encoding of trial-unique object images, andthen comparing subsequent temporal order and distance memory for image pairs that span a target or nontarget. We found that target detection enhanced recognition memory for target trial images but had no effect on temporal binding between items. In a follow-up experiment, we showed that when the encoding task required updating of task set rather than target count, event segmentation-related temporal memory effects were observed. These results document that target detection as such does not disrupt inter-item associations in memory, and that attention orienting in the absence of updating task sets does not create event boundaries. This suggests a key distinction between declarative and procedural working memory updates in segmenting events in memory.

Older adults with slow sit to stand times show reduced temporal precision of audio–visual integration

Sustained integration of sensory inputs over increased temporal delays is associated with reduced cognitive and physical functioning in older adults and adverse outcomes such as falls. Here, we explored the relationship between multisensory integration and a clinically relevant measure of balance/postural control; Sit-to-Stand Time, the efficiency with which an older adult can transition between a seated and a standing posture. We investigated whether temporal multisensory integration was associated with performance on the Five-Times Sit-to-Stand Test (FTSST) in a large sample of 2556 older adults (mean age = 63.62 years, SD = 7.50; 55% female) drawn from The Irish Longitudinal Study on Ageing (TILDA). K-means clustering was applied to FTSST data, yielding three clusters characterised by fast (mean = 10.88 s; n = 1122), medium (mean = 14.34 s; n = 1133) and slow (mean = 18.97 s; n = 301) sit-to-stand times. At wave 3 of TILDA, older adults participated in the Sound Induced Flash Illusion (SIFI), a measure of the precision of temporal audio–visual integration, which included three audio–visual stimulus onset asynchronies (SOAs): 70, 150 and 230 ms. Older adults with the slowest sit-to-stand times were more susceptible to the SIFI at the longest SOA (230 ms) compared to the shortest SOA (70 ms) relative to those with the fastest times (p = 0.02). Older adults who take longer to repeatedly transition from a seated to a standing posture exhibit an expanded temporal binding window for audio–visual events, supporting a link between multisensory perception and balance/postural control in ageing.

Perception of audio-visual synchrony in infants at elevated likelihood of developing autism spectrum disorder.

Findings suggest that early deficits in multisensory integration may impair language development among infants with elevated likelihood of developing ASD. •Perceptual integration of auditory and visual cues within speech is important for language development. •Prior work suggests that children with ASD are less sensitive to the temporal synchrony within audio-visual speech. •In this study, infants at elevated likelihood of developing ASD showed a larger temporal binding window for adynamic social event (Speaking Face) than TD infants, suggesting less efficient multisensory integration.

Spatial and temporal (non)binding of audiovisual rhythms in sensorimotor synchronisation

Human movement synchronisation with moving objects strongly relies on visual input. However, auditory information also plays an important role, since real environments are intrinsically multimodal. We used electroencephalography (EEG) frequency tagging to investigate the selective neural processing and integration of visual and auditory information during motor tracking and tested the effects of spatial and temporal congruency between audiovisual modalities. EEG was recorded while participants tracked with their index finger a red flickering (rate fV = 15 Hz) dot oscillating horizontally on a screen. The simultaneous auditory stimulus was modulated in pitch (rate fA = 32 Hz) and lateralised between left and right audio channels to induce perception of a periodic displacement of the sound source. Audiovisual congruency was manipulated in terms of space in Experiment 1 (no motion, same direction or opposite direction), and timing in Experiment 2 (no delay, medium delay or large delay). For both experiments, significant EEG responses were elicited at fV and fA tagging frequencies. It was also hypothesised that intermodulation products corresponding to the nonlinear integration of visual and auditory stimuli at frequencies fV ± fA would be elicited, due to audiovisual integration, especially in Congruent conditions. However, these components were not observed. Moreover, synchronisation and EEG results were not influenced by congruency manipulations, which invites further exploration of the conditions which may modulate audiovisual processing and the motor tracking of moving objects.

Audiovisual illusion training improves multisensory temporal integration

When we perceive external physical stimuli from the environment, the brain must remain somewhat flexible to unaligned stimuli within a specific range, as multisensory signals are subject to different transmission and processing delays. Recent studies have shown that the width of the ‘temporal binding window (TBW)’ can be reduced by perceptual learning. However, to date, the vast majority of studies examining the mechanisms of perceptual learning have focused on experience-dependent effects, failing to reach a consensus on its relationship with the underlying perception influenced by audiovisual illusion. The sound-induced flash illusion (SiFI) training is a reliable function for improving perceptual sensitivity. The present study utilized the classic auditory-dominated SiFI paradigm with feedback training to investigate the effect of a 5-day SiFI training on multisensory temporal integration, as evaluated by a simultaneity judgment (SJ) task and temporal order judgment (TOJ) task. We demonstrate that audiovisual illusion training enhances multisensory temporal integration precision in the form of (i) the point of subjective simultaneity (PSS) shifts to reality (0 ms) and (ii) a narrowing TBW. The results are consistent with a Bayesian model of causal inference, suggesting that perception learning reduce the susceptibility to SiFI, whilst improving the precision of audiovisual temporal estimation.

The effect of prestimulus low-frequency neural oscillations on the temporal perception of audiovisual speech.

Perceptual integration and segregation are modulated by the phase of ongoing neural oscillation whose frequency period is broader than the size of the temporal binding window (TBW). Studies have shown that the abstract beep-flash stimuli with about 100 ms TBW were modulated by the alpha band phase. Therefore, we hypothesize that the temporal perception of speech with about hundreds of milliseconds of TBW might be affected by the delta-theta phase. Thus, we conducted a speech-stimuli-based audiovisual simultaneity judgment (SJ) experiment. Twenty human participants (12 females) attended this study, recording 62 channels of EEG. Behavioral results showed that the visual leading TBWs are broader than the auditory leading ones [273.37 ± 24.24 ms vs. 198.05 ± 19.28 ms, (mean ± sem)]. We used Phase Opposition Sum (POS) to quantify the differences in mean phase angles and phase concentrations between synchronous and asynchronous responses. The POS results indicated that the delta-theta phase was significantly different between synchronous and asynchronous responses in the A50V condition (50% synchronous responses in auditory leading SOA). However, in the V50A condition (50% synchronous responses in visual leading SOA), we only found the delta band effect. In the two conditions, we did not find a consistency of phases over subjects for both perceptual responses by the post hoc Rayleigh test (all ps > 0.05). The Rayleigh test results suggested that the phase might not reflect the neuronal excitability which assumed that the phases within a perceptual response across subjects concentrated on the same angle but were not uniformly distributed. But V-test showed the phase difference between synchronous and asynchronous responses across subjects had a significant phase opposition (all ps < 0.05) which is compatible with the POS result. These results indicate that the speech temporal perception depends on the alignment of stimulus onset with an optimal phase of the neural oscillation whose frequency period might be broader than the size of TBW. The role of the oscillatory phase might be encoding the temporal information which varies across subjects rather than neuronal excitability. Given the enriched temporal structures of spoken language stimuli, the conclusion that phase encodes temporal information is plausible and valuable for future research.