Double-flash Illusion Research Articles

Crossmodal Postdiction: Conscious Perception as Revisionist History.

Postdiction occurs when later stimuli influence the perception of earlier stimuli. As the multisensory science field has grown in recent decades, the investigation of crossmodal postdictive phenomena has also expanded. Crossmodal postdiction can be considered (in its simplest form) the phenomenon in which later stimuli in one modality influence earlier stimuli in another modality (e.g., Intermodal Apparent Motion). Crossmodal postdiction can also appear in more nuanced forms, such as unimodal postdictive illusions (e.g., Apparent Motion) that are influenced by concurrent crossmodal stimuli (e.g., Crossmodal Influence on Apparent Motion), or crossmodal illusions (e.g., the Double Flash Illusion) that are influenced postdictively by a stimulus in one or the other modality (e.g., a visual stimulus in the Illusory Audiovisual Rabbit Illusion). In this review, these and other varied forms of crossmodal postdiction will be discussed. Three neuropsychological models proposed for unimodal postdiction will be adapted to the unique aspects of processing and integrating multisensory stimuli. Crossmodal postdiction opens a new window into sensory integration, and could potentially be used to identify new mechanisms of crossmodal crosstalk in the brain.

Temporal binding window and sense of agency are related processes modifiable via occipital tACS.

The temporal binding window refers to the time frame within which temporal grouping of sensory information takes place. Sense of agency is the feeling of being in control of one’s actions, and their associated outcomes. While previous research has shown that temporal cues and multisensory integration play a role in sense of agency, no studies have directly assessed whether individual differences in the temporal binding window and sense of agency are associated. In all three experiments, to assess sense of agency, participants pressed a button triggering, after a varying delay, the appearance of the circle, and reported their sense of agency over the effect. To assess the temporal binding window a simultaneity judgment task (Experiment 1) and a double-flash illusion task (Experiment 2 and 3) was also performed. As expected, the temporal binding window correlated with the sense of agency window. In Experiment 3, these processes were modulated by applying occipital tACS at either 14Hz or 8Hz. We found 14Hz tACS stimulation was associated with narrower temporal biding window and sense of agency window. Our results suggest the temporal binding window and the time window of sense of agency are related. They also point towards a possible underlying neural mechanism (alpha peak frequency) for this association.

Audiovisual integration and the P2 component in adult Asperger’s syndrome: An ERP-study

BackgroundA widened temporal window of integration (TWI) in audiovisual processing has been detected for children with autism spectrum disorder. However, research indicates a narrowing of this TWI and an associated change in audiovisual integration in the course of development. MethodsTo further elucidate audiovisual integration processes in adulthood, we compared adult participants with Asperger’s Syndrome (AS) to healthy controls (HC) by using the sound-induced double flash illusion. For a better understanding of underlying neural mechanisms, event-related potentials were measured via electroencephalography (EEG). ResultsThe number of reported sound-induced flash illusions indicated audiovisual integration. A similar TWI size for both, participants with AS and HC, was found. Additionally, enhanced P2 amplitudes were detected for participants with AS compared to HC. ConclusionResults indicate an involvement of attentional processes in audiovisual perception in participants with AS.

The temporal sensitivity to the tactile-induced double flash illusion mediates the impact of beta oscillations on schizotypal personality traits

The coherent experience of the self and the world depends on the ability to integrate vs. segregate sensory information. Optimal temporal integration between the senses is mediated by oscillatory properties of neural activity. Previous research showed reduced temporal sensitivity to multisensory events in schizotypy, a personality trait linked to schizophrenia. Here we used the tactile-induced Double-Flash-Illusion (tDFI) to investigate the tactile-to-visual temporal sensitivity in schizotypy, as indexed by the temporal window of illusion (TWI) and its neural underpinnings. We measured EEG oscillations within the beta band, recently shown to correlate with the tDFI. We found individuals with higher schizotypal traits to have wider TWI and slower beta waves accounting for the temporal window within which they perceive the illusion. Our results indicate reduced tactile-to-visual temporal sensitivity to mediate the effect of slowed oscillatory beta activity on schizotypal personality traits. We conclude that slowed oscillatory patterns might constitute an early marker for psychosis proneness.

Sense of agency with illusory visual events.

Sense of agency (SoA) is the subjective feeling of being in control of one's own actions and that one's action causes changes in external circumstances. In the present study, we investigated how percepts influenced a production of SoA when effect-related signals were illusory. For this purpose, we adopted the cross-modal double-flash illusion (DFI). In DFI, when two brief auditory stimuli (i.e., beeps) are presented in quick succession and accompanied by a single visual flash, observers tend to report two visual flashes. In the experiments, we let participants press a given button once or twice. Three stimulus conditions were used: one beep sound (inducer) with a single flash (effect; 1B1F), two beeps with two flashes (2B2F), and two beeps with one flash (DFI condition). Then, participants reported how much they thought they had caused the events (i.e., visual flashes) by their actions. Through the series of experiments, we found even illusory visual events enhanced SoA in the DFI condition. However, SoA in the DFI condition was smaller than that in the 2B2F condition. These results indicate that both the congruency between action and apparent stimulus as well as the stimulus property modulate SoA. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Crossmodal Postdiction: Conscious Perception as Revisionist History

Postdiction occurs when later stimuli influence the perception of earlier stimuli. As the multisensory science field has grown in recent decades, the investigation of crossmodal postdictive phenomena has also expanded. Crossmodal postdiction can be considered (in its simplest form) the phenomenon in which later stimuli in one modality influence earlier stimuli in another modality (e.g., Intermodal Apparent Motion). Crossmodal postdiction can also appear in more nuanced forms, such as unimodal postdictive illusions (e.g., Apparent Motion) that are influenced by concurrent crossmodal stimuli (e.g., Crossmodal Influence on Apparent Motion), or crossmodal illusions (e.g., the Double Flash Illusion) that are influenced postdictively by a stimulus in one or the other modality (e.g., a visual stimulus in the Illusory Audiovisual Rabbit Illusion). In this review, these and other varied forms of crossmodal postdiction will be discussed. Three neuropsychological models proposed for unimodal postdiction will be adapted to the unique aspects of processing and integrating multisensory stimuli. Crossmodal postdiction opens a new window into sensory integration, and could potentially be used to identify new mechanisms of crossmodal crosstalk in the brain.

Crossmodal Postdiction: Conscious Perception as Revisionist History †

Postdiction occurs when later stimuli influence the perception of earlier stimuli. As the multisensory science field has grown in recent decades, the investigation of crossmodal postdictive phenomena has also expanded. Crossmodal postdiction can be considered (in its simplest form) the phenomenon in which later stimuli in one modality influence earlier stimuli in another modality (e.g., Intermodal Apparent Motion). Crossmodal postdiction can also appear in more nuanced forms, such as unimodal postdictive illusions (e.g., Apparent Motion) that are influenced by concurrent crossmodal stimuli (e.g., Crossmodal Influence on Apparent Motion), or crossmodal illusions (e.g., the Double Flash Illusion) that are influenced postdictively by a stimulus in one or the other modality (e.g., a visual stimulus in the Illusory Audiovisual Rabbit Illusion). In this review, these and other varied forms of crossmodal postdiction will be discussed. Three neuropsychological models proposed for unimodal postdiction will be adapted to the unique aspects of processing and integrating multisensory stimuli. Crossmodal postdiction opens a new window into sensory integration, and could potentially be used to identify new mechanisms of crossmodal crosstalk in the brain.

Autistic Communication and Imagination Sub-Traits Are Related to Audiovisual Temporal Integration in the Stream-Bounce Illusion.

Autism spectrum disorder (ASD) is characterized by atypical social communication and restricted and repetitive behaviors; such traits are continuously distributed across nonclinical and clinical populations. Recently, relationships between ASD traits and low-level multisensory processing have been investigated, because atypical sensory reactivity has been regarded as a diagnostic criterion of ASD. Studies regarding an audiovisual illusion (the double-flash illusion) reported that social communication difficulties are related to temporal aspects of audiovisual integration. This study investigated whether similar relationships exist in another audiovisual illusion (the stream-bounce effect). In this illusion, two visual objects move toward each other, coincide, and pass each other, and the presentation of a transient sound at their coincidence induces a dominant perception that they bounce away from each other. Typically developing adults were recruited to perform experimental trials involving the stream-bounce effect. We measured their ASD traits using the Autism-Spectrum Quotient. The total quotient score was not related to any behavioral measurements of the effect. In contrast, for participants with higher difficulty in communication, the greatest magnitude of the stream-bounce effect occurred when the presentation timing of the sound tended to follow the visual coincidence. Participants with higher difficulty in imagination also showed the greatest magnitude of the effect when the presentation timing of the sound preceded that of the visual coincidence. Our findings regarding the stream-bounce effect, along with previous findings regarding the double-flash illusion, suggest that atypical temporal audiovisual integration is uniquely related to ASD sub-traits, especially in social communication.

The Dynamic Double Flash Illusion: Auditory Triggered Replay of Illusory Visual Expansion.

In the original double flash illusion, a visual flash (e.g., a sharp-edged disk, or uniformly filled circle) presented with two short auditory tones (beeps) is often followed by an illusory flash. The illusory flash has been previously shown to be triggered by the second auditory beep. The current study extends the double flash illusion by showing that this paradigm can not only create the illusory repeat of an on-off flash, but also trigger an illusory expansion (and in some cases a subsequent contraction) that is induced by the flash of a circular brightness gradient (gradient disk) to replay as well. The perception of the dynamic double flash illusion further supports the interpretation of the illusory flash (in the double flash illusion) as similar in its spatial and temporal properties to the perception of the real visual flash, likely by replicating the neural processes underlying the illusory expansion of the real flash. We show further that if a gradient disk (generating an illusory expansion) and a sharp-edged disk are presented simultaneously side by side with two sequential beeps, often only one visual stimulus or the other will be perceived to double flash. This indicates selectivity in auditory-visual binding, suggesting the usefulness of this paradigm as a psychophysical tool for investigating crossmodal binding phenomena.

Double Flash Illusions: Current Findings and Future Directions.

Twenty years ago, the first report on the sound-induced double flash illusion, a visual illusion induced by sound, was published. In this paradigm, participants are presented with different numbers of auditory and visual stimuli. In case of an incongruent number of auditory and visual stimuli, the influence of auditory information on visual perception can lead to the perception of the illusion. Thus, combining two auditory stimuli with one visual stimulus can induce the perception of two visual stimuli, the so-called fission illusion. Alternatively, combining one auditory stimulus with two visual stimuli can induce the perception of one visual stimulus, the so-called fusion illusion. Overall, current research shows that the illusion is a reliable indicator of multisensory integration. It has also been replicated using different stimulus combinations, such as visual and tactile stimuli. Importantly, the robustness of the illusion allows the widespread use for assessing multisensory integration across different groups of healthy participants and clinical populations and in various task setting. This review will give an overview of the experimental evidence supporting the illusion, the current state of research concerning the influence of cognitive processes on the illusion, the neural mechanisms underlying the illusion, and future research directions. Moreover, an exemplary experimental setup will be described with different options to examine perception, alongside code to test and replicate the illusion online or in the laboratory.

Oscillatory Properties of Functional Connections Between Sensory Areas Mediate Cross-Modal Illusory Perception.

The presentation of simple auditory stimuli can significantly impact visual processing and even induce visual illusions, such as the auditory-induced double flash illusion (DFI). These cross-modal processes have been shown to be driven by occipital oscillatory activity within the alpha band. Whether this phenomenon is network specific or can be generalized to other sensory interactions remains unknown. The aim of the current study was to test whether cross-modal interactions between somatosensory-to-visual areas leading to the same (but tactile-induced) DFI share similar properties with the auditory DFI. We hypothesized that if the effects are mediated by the oscillatory properties of early visual areas per se, then the two versions of the illusion should be subtended by the same neurophysiological mechanism (i.e., the speed of the alpha frequency). Alternatively, if the oscillatory activity in visual areas predicting this phenomenon is dependent on the specific neural network involved, then it should reflect network-specific oscillatory properties. In line with the latter, results recorded in humans (both sexes) show a network-specific oscillatory profile linking the auditory DFI to occipital alpha oscillations, replicating previous findings, and tactile DFI to occipital beta oscillations, a rhythm typical of somatosensory processes. These frequency-specific effects are observed for visual (but not auditory or somatosensory) areas and account for auditory-visual connectivity in the alpha band and somatosensory-visual connectivity in the beta band. We conclude that task-dependent visual oscillations reflect network-specific oscillatory properties favoring optimal directional neural communication timing for sensory binding.SIGNIFICANCE STATEMENT We investigated the oscillatory correlates of the auditory- and tactile-induced double flash illusion (DFI), a phenomenon where two interleaved beeps (taps) set within 100 ms apart and paired with one visual flash induce the sensation of a second illusory flash. Results confirm previous evidence that the speed of individual occipital alpha oscillations predict the temporal window of the auditory-induced illusion. Importantly, they provide novel evidence that the tactile-induced DFI is instead mediated by the speed of individual occipital beta oscillations. These task-dependent occipital oscillations are shown to be mediated by the oscillatory properties of the neural network engaged in the task to favor optimal temporal integration between the senses.

What you saw is what you will hear: Two new illusions with audiovisual postdictive effects.

Neuroscience investigations are most often focused on the prediction of future perception or decisions based on prior brain states or stimulus presentations. However, the brain can also process information retroactively, such that later stimuli impact conscious percepts of the stimuli that have already occurred (called “postdiction”). Postdictive effects have thus far been mostly unimodal (such as apparent motion), and the models for postdiction have accordingly been limited to early sensory regions of one modality. We have discovered two related multimodal illusions in which audition instigates postdictive changes in visual perception. In the first illusion (called the “Illusory Audiovisual Rabbit”), the location of an illusory flash is influenced by an auditory beep-flash pair that follows the perceived illusory flash. In the second illusion (called the “Invisible Audiovisual Rabbit”), a beep-flash pair following a real flash suppresses the perception of the earlier flash. Thus, we showed experimentally that these two effects are influenced significantly by postdiction. The audiovisual rabbit illusions indicate that postdiction can bridge the senses, uncovering a relatively-neglected yet critical type of neural processing underlying perceptual awareness. Furthermore, these two new illusions broaden the Double Flash Illusion, in which a single real flash is doubled by two sounds. Whereas the double flash indicated that audition can create an illusory flash, these rabbit illusions expand audition’s influence on vision to the suppression of a real flash and the relocation of an illusory flash. These new additions to auditory-visual interactions indicate a spatio-temporally fine-tuned coupling of the senses to generate perception.

Higher proneness to multisensory illusions is driven by reduced temporal sensitivity in people with high schizotypal traits.

A coherent sense of self, typically altered in schizophrenia, is accompanied by a coherent ability to integrate sensory information. According to the idea of a psychosis continuum, high schizotypal traits in the general population may be associated to higher proneness to multisensory illusions, akin to schizophrenia. We directly tested this hypothesis by means of the double-flash illusion in participants with low and high schizotypal scores. We confirmed the higher proneness to illusions in the high-schizotypal group. Crucially, such higher proneness was fully explained by a significantly reduced temporal sensitivity to integrate sensory information. We conclude that reduced temporal sensitivity accounts for enhanced proneness to illusions in people at higher risk and represents an early marker of psychosis.

Enhanced temporal binding of audiovisual information in the bilingual brain

We asked whether bilinguals’ benefits reach beyond the auditory modality to benefit multisensory processing. We measured audiovisual integration of auditory and visual cues in monolinguals and bilinguals via the double-flash illusion where the presentation of multiple auditory stimuli concurrent with a single visual flash induces an illusory perception of multiple flashes. We varied stimulus onset asynchrony (SOA) between auditory and visual cues to measure the “temporal binding window” where listeners fuse a single percept. Bilinguals showed faster responses and were less susceptible to the double-flash illusion than monolinguals. Moreover, monolinguals showed poorer sensitivity in AV processing compared to bilinguals. The width of bilinguals’ AV temporal integration window was narrower than monolinguals’ for both leading and lagging SOAs (Biling.: -65–112 ms; Mono.: -193 – 112 ms). Our results suggest the plasticity afforded by speaking multiple languages enhances multisensory integration and audiovisual binding in the bilingual brain.

The Spatial Double Flash Illusion: Audition-Induced Spatial Displacement

The Spatial Double Flash Illusion: Audition-Induced Spatial Displacement

Enlarged temporal integration window in schizophrenia indicated by the double-flash illusion

ABSTRACTIntroduction: In the present study we were interested in the processing of audio-visual integration in schizophrenia compared to healthy controls. The amount of sound-induced double-flash illusions served as an indicator of audio-visual integration. We expected an altered integration as well as a different window of temporal integration for patients.Methods: Fifteen schizophrenia patients and 15 healthy volunteers matched for age and gender were included in this study. We used stimuli with eight different temporal delays (stimulus onset asynchronys (SOAs) 25, 50, 75, 100, 125, 150, 200 and 300 ms) to induce a double-flash illusion. Group differences and the widths of temporal integration windows were calculated on percentages of reported double-flash illusions.Results: Patients showed significantly more illusions (ca. 36–44% vs. 9–16% in control subjects) for SOAs 150–300. The temporal integration window for control participants went from SOAs 25 to 200 whereas for patients integration was found across all included temporal delays. We found no significant relationship between the amount of illusions and either illness severity, chlorpromazine equivalent doses or duration of illness in patients.Conclusions: Our results are interpreted in favour of an enlarged temporal integration window for audio-visual stimuli in schizophrenia patients, which is consistent with previous research.

Crossmodal binding rivalry: A “race” for integration between unequal sensory inputs

Exposure to multiple but unequal (in number) sensory inputs often leads to illusory percepts, which may be the product of a conflict between those inputs. To test this conflict, we utilized the classic sound induced visual fission and fusion illusions under various temporal configurations and timing presentations. This conflict between unequal numbers of sensory inputs (i.e., crossmodal binding rivalry) depends on the binding of the first audiovisual pair and its temporal proximity to the upcoming unisensory stimulus. We, therefore, expected that tight coupling of the first audiovisual pair would lead to higher rivalry with the upcoming unisensory stimulus and, thus, weaker illusory percepts. Loose coupling, on the other hand, would lead to lower rivalry and higher illusory percepts. Our data showed the emergence of two different participant groups, those with low discrimination performance and strong illusion reports (particularly for fusion) and those with the exact opposite pattern, thus extending previous findings on the effect of visual acuity in the strength of the illusion. Most importantly, our data revealed differential illusory strength across different temporal configurations for the fission illusion, while for the fusion illusion these effects were only noted for the largest stimulus onset asynchronies tested. These findings support that the optimal integration theory for the double flash illusion should be expanded so as to also take into account the multisensory temporal interactions of the stimuli presented (i.e., temporal sequence and configuration).

Musicians have enhanced audiovisual multisensory binding: experience-dependent effects in the double-flash illusion.

Musical training is associated with behavioral and neurophysiological enhancements in auditory processing for both musical and nonmusical sounds (e.g., speech). Yet, whether the benefits of musicianship extend beyond enhancements to auditory-specific skills and impact multisensory (e.g., audiovisual) processing has yet to be fully validated. Here, we investigated multisensory integration of auditory and visual information in musicians and nonmusicians using a double-flash illusion, whereby the presentation of multiple auditory stimuli (beeps) concurrent with a single visual object (flash) induces an illusory perception of multiple flashes. We parametrically varied the onset asynchrony between auditory and visual events (leads and lags of ±300ms) to quantify participants' "temporal window" of integration, i.e., stimuli in which auditory and visual cues were fused into a single percept. Results show that musically trained individuals were both faster and more accurate at processing concurrent audiovisual cues than their nonmusician peers; nonmusicians had a higher susceptibility for responding to audiovisual illusions and perceived double flashes over an extended range of onset asynchronies compared to trained musicians. Moreover, temporal window estimates indicated that musicians' windows (<100ms) were ~2-3× shorter than nonmusicians' (~200ms), suggesting more refined multisensory integration and audiovisual binding. Collectively, findings indicate a more refined binding of auditory and visual cues in musically trained individuals. We conclude that experience-dependent plasticity of intensive musical experience extends beyond simple listening skills, improving multimodal processing and the integration of multiple sensory systems in a domain-general manner.

Visual limitations shape audio-visual integration.

Recent studies have proposed that some cross-modal illusions might be expressed in what were previously thought of as sensory-specific brain areas. Therefore, one interesting question is whether auditory-driven visual illusory percepts respond to manipulations of low-level visual attributes (such as luminance or chromatic contrast) in the same way as their nonillusory analogs. Here, we addressed this question using the double flash illusion (DFI), whereby one brief flash can be perceived as two when combined with two beeps presented in rapid succession. Our results showed that the perception of two illusory flashes depended on luminance contrast, just as the temporal resolution for two real flashes did. Specifically we found that the higher the luminance contrast, the stronger the DFI. Such a pattern seems to contradict what would be predicted from a maximum likelihood estimation perspective, and can be explained by considering that low-level visual stimulus attributes similarly modulate the perception of sound-induced visual phenomena and "real" visual percepts. This finding provides psychophysical support for the involvement of sensory-specific brain areas in the expression of the DFI. On the other hand, the addition of chromatic contrast failed to produce a change in the strength of the DFI despite it improved visual sensitivity to real flashes. The null impact of chromaticity on the cross-modal illusion might suggest a weaker interaction of the parvocellular visual pathway with the auditory system for cross-modal illusions.

Audio–visual integration through the parallel visual pathways

Audio–visual integration has been shown to be present in a wide range of different conditions, some of which are processed through the dorsal, and others through the ventral visual pathway. Whereas neuroimaging studies have revealed integration-related activity in the brain, there has been no imaging study of the possible role of segregated visual streams in audio–visual integration.We set out to determine how the different visual pathways participate in this communication. We investigated how audio–visual integration can be supported through the dorsal and ventral visual pathways during the double flash illusion. Low-contrast and chromatic isoluminant stimuli were used to drive preferably the dorsal and ventral pathways, respectively. In order to identify the anatomical substrates of the audio–visual interaction in the two conditions, the psychophysical results were correlated with the white matter integrity as measured by diffusion tensor imaging.The psychophysiological data revealed a robust double flash illusion in both conditions. A correlation between the psychophysical results and local fractional anisotropy was found in the occipito-parietal white matter in the low-contrast condition, while a similar correlation was found in the infero-temporal white matter in the chromatic isoluminant condition. Our results indicate that both of the parallel visual pathways may play a role in the audio–visual interaction.