Multisensory Temporal Binding Window Research Articles

Transfer of Audio-Visual Temporal Training to Temporal and Spatial Audio-Visual Tasks.

Temporal and spatial characteristics of sensory inputs are fundamental to multisensory integration because they provide probabilistic information as to whether or not multiple sensory inputs belong to the same event. The multisensory temporal binding window defines the time range within which two stimuli of different sensory modalities are merged into one percept and has been shown to depend on training. The aim of the present study was to evaluate the role of the training procedure for improving multisensory temporal discrimination and to test for a possible transfer of training to other multisensory tasks. Participants were trained over five sessions in a two-alternative forced-choice simultaneity judgment task. The task difficulty of each trial was either at each participant's threshold (adaptive group) or randomly chosen (control group). A possible transfer of improved multisensory temporal discrimination on multisensory binding was tested with a redundant signal paradigm in which the temporal alignment of auditory and visual stimuli was systematically varied. Moreover, the size of the spatial audio-visual ventriloquist effect was assessed. Adaptive training resulted in faster improvements compared to the control condition. Transfer effects were found for both tasks: The processing speed of auditory inputs and the size of the ventriloquist effect increased in the adaptive group following the training. We suggest that the relative precision of the temporal and spatial features of a cross-modal stimulus is weighted during multisensory integration. Thus, changes in the precision of temporal processing are expected to enhance the likelihood of multisensory integration for temporally aligned cross-modal stimuli.

Altered multisensory temporal integration in obesity.

Eating is a multisensory behavior. The act of placing food in the mouth provides us with a variety of sensory information, including gustatory, olfactory, somatosensory, visual, and auditory. Evidence suggests altered eating behavior in obesity. Nonetheless, multisensory integration in obesity has been scantily investigated so far. Starting from this gap in the literature, we seek to provide the first comprehensive investigation of multisensory integration in obesity. Twenty male obese participants and twenty male healthy-weight participants took part in the study aimed at describing the multisensory temporal binding window (TBW). The TBW is defined as the range of stimulus onset asynchrony in which multiple sensory inputs have a high probability of being integrated. To investigate possible multisensory temporal processing deficits in obesity, we investigated performance in two multisensory audiovisual temporal tasks, namely simultaneity judgment and temporal order judgment. Results showed a wider TBW in obese participants as compared to healthy-weight controls. This holds true for both the simultaneity judgment and the temporal order judgment tasks. An explanatory hypothesis would regard the effect of metabolic alterations and low-grade inflammatory state, clinically observed in obesity, on the temporal organization of brain ongoing activity, which one of the neural mechanisms enabling multisensory integration.

Generalization of multisensory perceptual learning.

Life in a multisensory world requires the rapid and accurate integration of stimuli across the different senses. In this process, the temporal relationship between stimuli is critical in determining which stimuli share a common origin. Numerous studies have described a multisensory temporal binding window—the time window within which audiovisual stimuli are likely to be perceptually bound. In addition to characterizing this window’s size, recent work has shown it to be malleable, with the capacity for substantial narrowing following perceptual training. However, the generalization of these effects to other measures of perception is not known. This question was examined by characterizing the ability of training on a simultaneity judgment task to influence perception of the temporally-dependent sound-induced flash illusion (SIFI). Results do not demonstrate a change in performance on the SIFI itself following training. However, data do show an improved ability to discriminate rapidly-presented two-flash control conditions following training. Effects were specific to training and scaled with the degree of temporal window narrowing exhibited. Results do not support generalization of multisensory perceptual learning to other multisensory tasks. However, results do show that training results in improvements in visual temporal acuity, suggesting a generalization effect of multisensory training on unisensory abilities.

On the Temporal Precision of Thought: Individual Differences in the Multisensory Temporal Binding Window Predict Performance on Verbal and Nonverbal Problem Solving Tasks

Although psychology is greatly preoccupied by the tight link between the way that individuals perceive the world and their intelligent, creative behavior, there is little experimental work on the relationship between individual differences in perception and cognitive ability in healthy populations. Here, individual differences in problem solving ability were examined in relation to multisensory perception as measured by tolerance for temporal asynchrony between auditory and visual inputs, i.e., the multisensory temporal binding window. The results demonstrated that enhanced performance in both verbal and nonverbal problem solving tasks (the Remote Associates Test and Raven’s Advanced Progressive Matrices Task) is predicted by a narrower audio-visual temporal binding window, which reflects greater sensitivity to subtle discrepancies in sensory inputs. This suggests that the precision of individuals’ temporal window of multisensory integration might mirror their capacities for complex reasoning and thus the precision of their thoughts.

Altered multisensory temporal binding window in low-grade inflammation

Obesity has been associated with decreased performance on cognitive tests, including abstract reasoning, visuospatial organization, and attention. However, recent evidence suggests that alterations of unisensory and multisensory processing are also present. The aim of this study was to compare the temporal features of multisensory integration in twenty obese people to those of twenty healthy controls, by measuring the audio-visual temporal binding window, defined as the range of stimulus onset asynchrony in which multiple sensory inputs have a high probability of altering responses. Results revealed an extended temporal binding window in obese participants as compared to healthy controls, suggesting an alteration of multisensory binding. An explanatory hypothesis would regard the effect of the chronic low-grade inflammatory state, clinically observed in obesity, on neural activity. We speculate that pro-inflammatory cytokines, which are overexpressed in obesity, alter the temporal structure of the ongoing brain activity. This, in turn, impacts on the neural mechanisms enabling multisensory integration. We believe that our finding sheds new light on the link between the brain and the immune system, and it has potential implications for future experimental design and therapeutic implementation in obesity treatment.

Zapping the gap: Reducing the multisensory temporal binding window by means of transcranial direct current stimulation (tDCS).

Synchrony among the senses lies at the heart of our possession of a unified conscious perception of the world. However, due to discrepancies in physical and neural information processing from different senses, the brain accommodates a limited range of temporal asynchronies between sensory inputs, i.e. the multisensory temporal binding window (TBW). Using non-invasive brain stimulation, we sought to modulate the audio-visual TBW and to identify cortical areas implicated in the conscious perception of multisensory synchrony. Participants performed a simultaneity judgment task while experiencing anodal (Experiment 1) or cathodal (Experiment 2) transcranial direct current stimulation (tDCS) over parietal and frontal regions. The results demonstrate that stimulating the right posterior parietal cortex significantly reduces the audio-visual TBW by approximately 30%, thereby causally linking this region to the plasticity of the TBW. This highlights a potential interventional technique for populations with a wider TBW, such as in autism and dyslexia.

The effects of visual training on multisensory temporal processing.

The importance of multisensory integration for human behavior and perception is well documented, as is the impact that temporal synchrony has on driving such integration. Thus, the more temporally coincident two sensory inputs from different modalities are, the more likely they will be perceptually bound. This temporal integration process is captured by the construct of the temporal binding window-the range of temporal offsets within which an individual is able to perceptually bind inputs across sensory modalities. Recent work has shown that this window is malleable and can be narrowed via a multisensory perceptual feedback training process. In the current study, we seek to extend this by examining the malleability of the multisensory temporal binding window through changes in unisensory experience. Specifically, we measured the ability of visual perceptual feedback training to induce changes in the multisensory temporal binding window. Visual perceptual training with feedback successfully improved temporal visual processing, and more importantly, this visual training increased the temporal precision across modalities, which manifested as a narrowing of the multisensory temporal binding window. These results are the first to establish the ability of unisensory temporal training to modulate multisensory temporal processes, findings that can provide mechanistic insights into multisensory integration and which may have a host of practical applications.

Individual differences in the multisensory temporal binding window predict susceptibility to audiovisual illusions.

Human multisensory systems are known to bind inputs from the different sensory modalities into a unified percept, a process that leads to measurable behavioral benefits. This integrative process can be observed through multisensory illusions, including the McGurk effect and the sound-induced flash illusion, both of which demonstrate the ability of one sensory modality to modulate perception in a second modality. Such multisensory integration is highly dependent upon the temporal relationship of the different sensory inputs, with perceptual binding occurring within a limited range of asynchronies known as the temporal binding window (TBW). Previous studies have shown that this window is highly variable across individuals, but it is unclear how these variations in the TBW relate to an individual's ability to integrate multisensory cues. Here we provide evidence linking individual differences in multisensory temporal processes to differences in the individual's audiovisual integration of illusory stimuli. Our data provide strong evidence that the temporal processing of multiple sensory signals and the merging of multiple signals into a single, unified perception, are highly related. Specifically, the width of right side of an individuals' TBW, where the auditory stimulus follows the visual, is significantly correlated with the strength of illusory percepts, as indexed via both an increase in the strength of binding synchronous sensory signals and in an improvement in correctly dissociating asynchronous signals. These findings are discussed in terms of their possible neurobiological basis, relevance to the development of sensory integration, and possible importance for clinical conditions in which there is growing evidence that multisensory integration is compromised.

Developmental changes in the multisensory temporal binding window persist into adolescence

We live in a world rich in sensory information, and consequently the brain is challenged with deciphering which cues from the various sensory modalities belong together. Determinations regarding the relatedness of sensory information appear to be based, at least in part, on the spatial and temporal relationships between the stimuli. Stimuli that are presented in close spatial and temporal correspondence are more likely to be associated with one another and thus 'bound' into a single perceptual entity. While there is a robust literature delineating behavioral changes in perception induced by multisensory stimuli, maturational changes in multisensory processing, particularly in the temporal realm, are poorly understood. The current study examines the developmental progression of multisensory temporal function by analyzing responses on an audiovisual simultaneity judgment task in 6- to 23-year-old participants. The overarching hypothesis for the study was that multisensory temporal function will mature with increasing age, with the developmental trajectory for this change being the primary point of inquiry. Results indeed reveal an age-dependent decrease in the size of the 'multisensory temporal binding window', the temporal interval within which multisensory stimuli are likely to be perceptually bound, with changes occurring over a surprisingly protracted time course that extends into adolescence.

Neural Correlates of Multisensory Perceptual Learning

The brain's ability to bind incoming auditory and visual stimuli depends critically on the temporal structure of this information. Specifically, there exists a temporal window of audiovisual integration within which stimuli are highly likely to be perceived as part of the same environmental event. Several studies have described the temporal bounds of this window, but few have investigated its malleability. Recently, our laboratory has demonstrated that a perceptual training paradigm is capable of eliciting a 40% narrowing in the width of this window that is stable for at least 1 week after cessation of training. In the current study, we sought to reveal the neural substrates of these changes. Eleven human subjects completed an audiovisual simultaneity judgment training paradigm, immediately before and after which they performed the same task during an event-related 3T fMRI session. The posterior superior temporal sulcus (pSTS) and areas of auditory and visual cortex exhibited robust BOLD decreases following training, and resting state and effective connectivity analyses revealed significant increases in coupling among these cortices after training. These results provide the first evidence of the neural correlates underlying changes in multisensory temporal binding likely representing the substrate for a multisensory temporal binding window.

The Multisensory Temporal Binding Window: Perceptual Fusion, Training, and Autism

The process of integrating information across sensory modalities is highly dependent upon the temporal coincidence of the inputs. Audiovisual information is integrated within a range of temporal offsets, known as the temporal binding window (TBW), which varies between individuals. Three particular findings relating to TBW have led us to a novel approach to address sensory integration impairments in children with autism. The first is that autistic children have an atypically wide TBW, as measured through manipulations of audiovisual illusions. Second, an individual's TBW is related to their ability to perceptually fuse audiovisual inputs, particularly as seen in the McGurk effect; the narrower the right TBW, the stronger the McGurk effect. The third finding is that the TBW is plastic. Through perceptual feedback training, our lab showed that individuals' right TBW can be narrowed. These three findings, which we will present, lead to a study of perceptual feedback training in autistic children, who may have...

An extended multisensory temporal binding window in autism spectrum disorders.

Autism spectrum disorders (ASD) form a continuum of neurodevelopmental disorders, characterized by deficits in communication and reciprocal social interaction, as well as by repetitive behaviors and restricted interests. Sensory disturbances are also frequently reported in clinical and autobiographical accounts. However, surprisingly few empirical studies have characterized the fundamental features of sensory and multisensory processing in ASD. The current study is structured to test for potential differences in multisensory temporal function in ASD by making use of a temporally dependent, low-level multisensory illusion. In this illusion, the presentation of a single flash of light accompanied by multiple sounds often results in the illusory perception of multiple flashes. By systematically varying the temporal structure of the audiovisual stimuli, a “temporal window” within which these stimuli are likely to be bound into a single perceptual entity can be defined. The results of this study revealed that children with ASD report the flash-beep illusion over an extended range of stimulus onset asynchronies relative to children with typical development, suggesting that children with ASD have altered multisensory temporal function. These findings provide valuable new insights into our understanding of sensory processing in ASD and may hold promise for the development of more sensitive diagnostic measures and improved remediation strategies.