Illusory Flash Research Articles

Subjective Audibility Modulates the Susceptibility to Sound-Induced Flash Illusion: Effect of Loudness and Auditory Masking.

When a brief flash is presented along with two brief sounds, the single flash is often perceived as two flashes. This phenomenon is called a sound-induced flash illusion, in which the auditory sense, with its relatively higher reliability in providing temporal information, modifies the visual perception. Decline of audibility due to hearing impairment is known to make subjects less susceptible to the flash illusion. However, the effect of decline of audibility on susceptibility to the illusion has not been directly investigated in subjects with normal hearing. The present study investigates the relationship between audibility and susceptibility to the illusion by varying the sound pressure level of the stimulus. In the task for reporting the number of auditory stimuli, lowering the sound pressure level caused the rate of perceiving two sounds to decrease on account of forward masking. The occurrence of the illusory flash was reduced as the intensity of the second auditory stimulus decreased, and was significantly correlated with the rate of perceiving the two auditory stimuli. These results suggest that the susceptibility to sound-induced flash illusion depends on the subjective audibility of each sound.

Visually biased Perception in Cochlear Implant Users: A Study of the McGurk and Sound-Induced Flash Illusions.

The reduction in spectral resolution by cochlear implants oftentimes requires complementary visual speech cues to facilitate understanding. Despite substantial clinical characterization of auditory-only speech measures, relatively little is known about the audiovisual (AV) integrative abilities that most cochlear implant (CI) users rely on for daily speech comprehension. In this study, we tested AV integration in 63 CI users and 69 normal-hearing (NH) controls using the McGurk and sound-induced flash illusions. To our knowledge, this study is the largest to-date measuring the McGurk effect in this population and the first that tests the sound-induced flash illusion (SIFI). When presented with conflicting AV speech stimuli (i.e., the phoneme "ba" dubbed onto the viseme "ga"), we found that 55 CI users (87%) reported a fused percept of "da" or "tha" on at least one trial. After applying an error correction based on unisensory responses, we found that among those susceptible to the illusion, CI users experienced lower fusion than controls-a result that was concordant with results from the SIFI where the pairing of a single circle flashing on the screen with multiple beeps resulted in fewer illusory flashes for CI users. While illusion perception in these two tasks appears to be uncorrelated among CI users, we identified a negative correlation in the NH group. Because neither illusion appears to provide further explanation of variability in CI outcome measures, further research is needed to determine how these findings relate to CI users' speech understanding, particularly in ecological listening conditions that are naturally multisensory.

Exposure to first-person shooter videogames is associated with multisensory temporal precision and migraine incidence

Adaptive interactions with the environment require optimal integration and segregation of sensory information. Yet, temporal misalignments in the presentation of visual and auditory stimuli may generate illusory phenomena such as the sound-induced flash illusion, in which a single flash paired with multiple auditory stimuli induces the perception of multiple illusory flashes. This phenomenon has been shown to be robust and resistant to feedback training. According to a Bayesian account, this is due to a statistically optimal combination of the signals operated by the nervous system. From this perspective, individual susceptibility to the illusion might be moulded through prolonged experience. For example, repeated exposure to the illusion and prolonged training sessions partially impact on the reported illusion.Therefore, extensive and immersive audio-visual experience, such as first-person shooter videogames, should sharpen individual capacity to correctly integrate multisensory information over time, leading to more veridical perception. We tested this hypothesis by comparing the temporal profile of the sound-induced illusion in a group of expert first-person shooter gamers and a non-players group. In line with the hypotheses, gamers experience significantly narrower windows of illusion (~87 ms) relative to non-players (~105 ms), leading to higher veridical reports in gamers (~68%) relative to non-players (~59%). Moreover, according to recent literature, we tested whether audio-visual intensive training in gamers could be related to the incidence of migraine, and found that its severity may be directly proportioned to the time spent on videogames. Overall, these results suggest that continued training within audio-visual environments such as first-person shooter videogames improves temporal discrimination and sensory integration. This finding may pave the way for future therapeutic strategies based on self-administered multisensory training. On the other hand, the impact of intensive training on visual-related stress disorders, such as migraine incidence, should be taken into account as a risk factor during therapeutic planning.

The Dynamic Double Flash Illusion: Auditory Triggered Replay of Illusory Visual Expansion.

In the original double flash illusion, a visual flash (e.g., a sharp-edged disk, or uniformly filled circle) presented with two short auditory tones (beeps) is often followed by an illusory flash. The illusory flash has been previously shown to be triggered by the second auditory beep. The current study extends the double flash illusion by showing that this paradigm can not only create the illusory repeat of an on-off flash, but also trigger an illusory expansion (and in some cases a subsequent contraction) that is induced by the flash of a circular brightness gradient (gradient disk) to replay as well. The perception of the dynamic double flash illusion further supports the interpretation of the illusory flash (in the double flash illusion) as similar in its spatial and temporal properties to the perception of the real visual flash, likely by replicating the neural processes underlying the illusory expansion of the real flash. We show further that if a gradient disk (generating an illusory expansion) and a sharp-edged disk are presented simultaneously side by side with two sequential beeps, often only one visual stimulus or the other will be perceived to double flash. This indicates selectivity in auditory-visual binding, suggesting the usefulness of this paradigm as a psychophysical tool for investigating crossmodal binding phenomena.

Atypical Multisensory Integration and the Temporal Binding Window in Autism Spectrum Disorder

The present study examined the relationship between multisensory integration and the temporal binding window (TBW) for multisensory processing in adults with Autism spectrum disorder (ASD). The ASD group was less likely than the typically developing group to perceive an illusory flash induced by multisensory integration during a sound-induced flash illusion (SIFI) task. Although both groups showed comparable TBWs during the multisensory temporal order judgment task, correlation analyses and Bayes factors provided moderate evidence that the reduced SIFI susceptibility was associated with the narrow TBW in the ASD group. These results suggest that the individuals with ASD exhibited atypical multisensory integration and that individual differences in the efficacy of this process might be affected by the temporal processing of multisensory information.

Oscillatory Properties of Functional Connections Between Sensory Areas Mediate Cross-Modal Illusory Perception.

The presentation of simple auditory stimuli can significantly impact visual processing and even induce visual illusions, such as the auditory-induced double flash illusion (DFI). These cross-modal processes have been shown to be driven by occipital oscillatory activity within the alpha band. Whether this phenomenon is network specific or can be generalized to other sensory interactions remains unknown. The aim of the current study was to test whether cross-modal interactions between somatosensory-to-visual areas leading to the same (but tactile-induced) DFI share similar properties with the auditory DFI. We hypothesized that if the effects are mediated by the oscillatory properties of early visual areas per se, then the two versions of the illusion should be subtended by the same neurophysiological mechanism (i.e., the speed of the alpha frequency). Alternatively, if the oscillatory activity in visual areas predicting this phenomenon is dependent on the specific neural network involved, then it should reflect network-specific oscillatory properties. In line with the latter, results recorded in humans (both sexes) show a network-specific oscillatory profile linking the auditory DFI to occipital alpha oscillations, replicating previous findings, and tactile DFI to occipital beta oscillations, a rhythm typical of somatosensory processes. These frequency-specific effects are observed for visual (but not auditory or somatosensory) areas and account for auditory-visual connectivity in the alpha band and somatosensory-visual connectivity in the beta band. We conclude that task-dependent visual oscillations reflect network-specific oscillatory properties favoring optimal directional neural communication timing for sensory binding.SIGNIFICANCE STATEMENT We investigated the oscillatory correlates of the auditory- and tactile-induced double flash illusion (DFI), a phenomenon where two interleaved beeps (taps) set within 100 ms apart and paired with one visual flash induce the sensation of a second illusory flash. Results confirm previous evidence that the speed of individual occipital alpha oscillations predict the temporal window of the auditory-induced illusion. Importantly, they provide novel evidence that the tactile-induced DFI is instead mediated by the speed of individual occipital beta oscillations. These task-dependent occipital oscillations are shown to be mediated by the oscillatory properties of the neural network engaged in the task to favor optimal temporal integration between the senses.

What you saw is what you will hear: Two new illusions with audiovisual postdictive effects.

Neuroscience investigations are most often focused on the prediction of future perception or decisions based on prior brain states or stimulus presentations. However, the brain can also process information retroactively, such that later stimuli impact conscious percepts of the stimuli that have already occurred (called “postdiction”). Postdictive effects have thus far been mostly unimodal (such as apparent motion), and the models for postdiction have accordingly been limited to early sensory regions of one modality. We have discovered two related multimodal illusions in which audition instigates postdictive changes in visual perception. In the first illusion (called the “Illusory Audiovisual Rabbit”), the location of an illusory flash is influenced by an auditory beep-flash pair that follows the perceived illusory flash. In the second illusion (called the “Invisible Audiovisual Rabbit”), a beep-flash pair following a real flash suppresses the perception of the earlier flash. Thus, we showed experimentally that these two effects are influenced significantly by postdiction. The audiovisual rabbit illusions indicate that postdiction can bridge the senses, uncovering a relatively-neglected yet critical type of neural processing underlying perceptual awareness. Furthermore, these two new illusions broaden the Double Flash Illusion, in which a single real flash is doubled by two sounds. Whereas the double flash indicated that audition can create an illusory flash, these rabbit illusions expand audition’s influence on vision to the suppression of a real flash and the relocation of an illusory flash. These new additions to auditory-visual interactions indicate a spatio-temporally fine-tuned coupling of the senses to generate perception.

Temporal and spectral audiotactile interactions in musicians.

Previous investigations have revealed that the complex sensory exposure of musical training alters audiovisual interactions. As of yet, there has been little evidence on the effects of musical training on audiotactile interactions at a behavioural level. Here, we tested audiotactile interaction in musicians using the audiotactile illusory flash and the parchment-skin illusion. Significant differences were only found between musicians and non-musicians for the audiotactile illusory flash. Both groups had similar task-relevant unisensory abilities, but unlike non-musicians, the number of auditory stimulations did not have a statistically important influence on the number of perceived tactile stimulations for musicians. Musicians and non-musicians similarly perceived the parchment-skin illusion. Spectral alterations of self-generated palmar sounds similarly altered the perception of wetness and dryness for both groups. These results suggest that musical training does not seem to alter multisensory interactions at large. The specificity of the sensory enhancement suggests that musical training specifically alters processes underlying the interaction of temporal audiotactile stimuli and not the global interaction between these modalities. These results are consistent with previous unisensory and multisensory investigations on sensory abilities related to audiotactile processing in musicians.

Visual limitations shape audio-visual integration.

Recent studies have proposed that some cross-modal illusions might be expressed in what were previously thought of as sensory-specific brain areas. Therefore, one interesting question is whether auditory-driven visual illusory percepts respond to manipulations of low-level visual attributes (such as luminance or chromatic contrast) in the same way as their nonillusory analogs. Here, we addressed this question using the double flash illusion (DFI), whereby one brief flash can be perceived as two when combined with two beeps presented in rapid succession. Our results showed that the perception of two illusory flashes depended on luminance contrast, just as the temporal resolution for two real flashes did. Specifically we found that the higher the luminance contrast, the stronger the DFI. Such a pattern seems to contradict what would be predicted from a maximum likelihood estimation perspective, and can be explained by considering that low-level visual stimulus attributes similarly modulate the perception of sound-induced visual phenomena and "real" visual percepts. This finding provides psychophysical support for the involvement of sensory-specific brain areas in the expression of the DFI. On the other hand, the addition of chromatic contrast failed to produce a change in the strength of the DFI despite it improved visual sensitivity to real flashes. The null impact of chromaticity on the cross-modal illusion might suggest a weaker interaction of the parvocellular visual pathway with the auditory system for cross-modal illusions.

Individual Differences in Alpha Frequency Drive Crossmodal Illusory Perception

SummaryPerception routinely integrates inputs from different senses. Stimulus temporal proximity critically determines whether or not these inputs are bound together. Despite the temporal window of integration being a widely accepted notion, its neurophysiological substrate remains unclear. Many types of common audio-visual interactions occur within a time window of ∼100 ms [1–5]. For example, in the sound-induced double-flash illusion, when two beeps are presented within ∼100 ms together with one flash, a second illusory flash is often perceived [2]. Due to their intrinsic rhythmic nature, brain oscillations are one candidate mechanism for gating the temporal window of integration. Interestingly, occipital alpha band oscillations cycle on average every ∼100 ms, with peak frequencies ranging between 8 and 14 Hz (i.e., 120–60 ms cycle). Moreover, presenting a brief tone can phase-reset such oscillations in visual cortex [6, 7]. Based on these observations, we hypothesized that the duration of each alpha cycle might provide the temporal unit to bind audio-visual events. Here, we first recorded EEG while participants performed the sound-induced double-flash illusion task [4] and found positive correlation between individual alpha frequency (IAF) peak and the size of the temporal window of the illusion. Participants then performed the same task while receiving occipital transcranial alternating current stimulation (tACS), to modulate oscillatory activity [8] either at their IAF or at off-peak alpha frequencies (IAF±2 Hz). Compared to IAF tACS, IAF−2 Hz and IAF+2 Hz tACS, respectively, enlarged and shrunk the temporal window of illusion, suggesting that alpha oscillations might represent the temporal unit of visual processing that cyclically gates perception and the neurophysiological substrate promoting audio-visual interactions.

Illusory flashes and perception

Information from the environment can be based on a single or several modalities. The simultaneous processing of information separated in space and/or time depends on multiple factors. Visual illusions serve as a good tool with which to investigate the parallel processing of information and their interactions. This study was designed to gain information about a unimodal illusion: a target that flashes once seems to flash more as a result of a simultaneously presented inducer flashing several times nearby. The first aim of this work was to understand whether the number of perceived flashes is merely a result of a bias in the criterion level or whether it is based on a real percept. We then clarified how the illusion finds its way into the percept. The final step was designed to establish the logic of the processing in the background by determining whether the modality appropriateness hypothesis, the information reliability hypothesis, or the discontinuity theory best explains the predominant role of the inducer.

Pre- and post-stimulus EEG patterns associated with the touch-induced illusory flash

Pairing two brief auditory beeps with a single flash can evoke the percept of a second, illusory, flash. Investigations of the underlying neural mechanisms are limited to post-stimulus effects of this sound-induced illusory flash. We investigated whether touch modulates the visual evoked potential in a similar vein, and also looked at pre-stimulus activity. Electroencephalogram (EEG) was recorded over occipital and parieto-occipital areas of 12 observers. We compared bimodal EEG to its unimodal constituents (i.e., the difference waves) and found significant positive deflections around 110ms and 200ms and negative deflections around 330ms and 390ms from stimulus onset. These results are similar to those reported for the sound-induced illusion, albeit somewhat later. Furthermore, comparison of the EEG activity between those trials in which the illusion was perceived and those in which it was absent revealed that the phase of pre-stimulus alpha was linked to perceiving the illusion or not. We conclude that touch can modulate activity in the visual cortex and that similar neural mechanisms underlie perception of the sound- and touch-induced illusory flash and that the phase of the alpha wave at the moment of presentation that affects perception.

Audition influences color processing in the sound-induced visual flash illusion

Multisensory interactions can lead to illusory percepts, as exemplified by the sound-induced extra flash illusion (SIFI: Shams, Kamitani, & Shimojo, 2000, 2002). In this illusion, an audio-visual stimulus sequence consisting of two pulsed sounds and a light flash presented within a 100 ms time window generates the visual percept of two flashes. Here, we used colored visual stimuli to investigate whether concurrent auditory stimuli can affect the perceived features of the illusory flash. Zero, one or two pulsed sounds were presented concurrently with either a red or green flash or with two flashes of different colors (red followed by green) in rapid sequence. By querying both the number and color of the participants' visual percepts, we found that the double flash illusion is stimulus specific: i.e., two sounds paired with one red or one green flash generated the percept of two red or two green flashes, respectively. This implies that the illusory second flash is induced at a level of visual processing after perceived color has been encoded. In addition, we found that the presence of two sounds influenced the integration of color information from two successive flashes. In the absence of any sounds, a red and a green flash presented in rapid succession fused to form a single orange percept, but when accompanied by two sounds, this integrated orange percept was perceived to flash twice on a significant proportion of trials. In addition, the number of concurrent auditory stimuli modified the degree to which the successive flashes were integrated to an orange percept vs. maintained as separate red-green percepts. Overall, these findings show that concurrent auditory input can affect both the temporal and featural properties of visual percepts.

Observers can reliably identify illusory flashes in the illusory flash paradigm

In the illusory flash paradigm, a single flash may be experienced as two flashes when accompanied by two beeps or taps, and two flashes may be experienced as a single flash when accompanied by one beep or tap. The classic paradigm restricts responses to '1' and '2' (2-AFC), ignoring possible qualitative differences between real and illusory flashes and implicitly assuming that illusory flashes are indistinguishable from real flashes. We added a third response category 'different from that of either 1 or 2 flashes' (3-AFC). Eight naïve and 6 experienced observers responded to 160 real and 160 illusory flash trials. Experienced observers were exposed to 1,200 trials before the experiment but without receiving feedback on their performance. The third response category was used for only 4% of the real flash trials and for 44% of the illusory flash trials. Experienced observers did so more often (78%) than naïve observers (18%). This shows that observers can reliably identify illusory flashes and indicates that mere exposure to illusory flash trials (without feedback) is enough to detect and classify potential qualitative differences between real and illusory flashes.

Task dependent anatomical connections underlie multisensory processing

Our senses interact in daily life through multisensory integration, facilitating perceptual processes and behavioral responses. Numerous multisensory regions have been identified in humans and animals, raising the question of whether a single mechanism can support the dynamic range of experiences and behaviors multisensory processing engenders. The most common neural mechanisms proposed to underlie multisensory processing include anatomical connections directly linking early sensory areas, indirect connections to higher-order multisensory regions, and functional connectivity between cortical areas. Here we examine the relationship between white matter connectivity, as assessed with diffusion tensor imaging and individual differences in two divergent forms of multisensory processing: the redundant-target effect (RTE), in which subjects’ behavior is facilitated by congruent multisensory information, and the sound-induced illusory flash (SIIF) paradigm, in which incongruent multisensory information elicits a novel percept. Behavioral results demonstrated strong intra-subject reliability of the RTE and SIIF paradigms, but no correlation in performance between the two tasks. Consistent with this behavioral finding, we identified distinct anatomical networks underlying these two forms of multisensory processing. Using a whole-brain analysis and contrasting anatomical models of multisensory processing, increased behavioral performance on the RTE was associated with increased connectivity between the superior parietal lobe and early sensory regions. Conversely, increased incidence of illusion on the SIIF paradigm was associated with increased connectivity directly between early auditory and visual areas. These results implicate a broad network of anatomical connections involved in task-dependent multisensory processes.

Temporal Aspects of Sound Induced Visual Fission and Fusion in the Double-Flash Illusion Paradigm

The sound-induced flash illusion (SIFI) is thought to be due to early audiovisual connections which allow rapid interplay between early auditory and visual cortices. However, the temporal relationship between the perception of physically presented and additional illusory flashes has not yet been directly addressed. We presented participants with a stream of four brief auditory white noise bursts separated by gaps of 60ms (SOA=70ms) and one or two brief flashes which could appear in any of the gaps between noise bursts (positions 1–4) or before the auditory stream (position 0). All possible combinations were randomized and presented equally often. First, participants were asked to indicate the perceived number of flashes, and second, to indicate the perceived position of the flash(es). Thus we could compare the physical separation of the flashes with the perceived separation. We report the following results: 1) In illusory-fission trials the average perceived distance between the perceived flashes is ∼100ms. 2) SIFI is strongest when the flash is presented at position 1 and 2. 3) The extent of experienced fission-illusion depends on performance in aligning flashes with auditory gaps. 4) We observed auditory induced de-fusion which is strongest at position 1 and again correlated to overall task performance.

Illusory double flashes can speed up responses like physical ones: evidence from the sound-induced flash illusion

When a single brief flash is accompanied by two auditory beeps, participants often report perceiving two flashes. The present experiment examined whether the perception of illusory redundant flashes can result in faster responses as compared to the perception of a single flash, because previous research has shown such a redundancy gain for physical stimuli. To this end, participants were asked to respond as rapidly as possible to the onset of any flash. Following their response, they additionally indicated whether they perceived a single flash or a double flash. Most importantly, we observed significant shorter reaction times in response to redundant flashes, irrespective of whether they were physically presented or illusorily perceived. Taken together, our results suggest that an illusory percept can affect simple reaction time in much the same manner as the corresponding physical stimulation.

Accurate and rapid estimation of phosphene thresholds (REPT).

To calibrate the intensity of transcranial magnetic stimulation (TMS) at the occipital pole, the phosphene threshold is used as a measure of cortical excitability. The phosphene threshold (PT) refers to the intensity of magnetic stimulation that induces illusory flashes of light (phosphenes) on a proportion of trials. The existing PT estimation procedures lack the accuracy and mathematical rigour of modern threshold estimation methods. We present an improved and automatic procedure for estimating the PT which is based on the well-established Ψ Bayesian adaptive staircase approach. To validate the new procedure, we compared it with another commonly used procedure for estimating the PT. We found that our procedure is more accurate, reliable, and rapid when compared with an existing PT measurement procedure. The new procedure is implemented in Matlab and works automatically with the Magstim Rapid2 stimulator using a convenient graphical user interface. The Matlab program is freely available for download.

Multisensory Effects Differ for Counting Small and Large Pulse Numbers

In Illusory Flash (IF) experiments, congruent multisensory presentation has no effect on the mean estimate of the number of events, but decreases the variance in comparison with unisensory presentation. In contrast, congruent multisensory presentation in other Temporal Numerosity Judgement (TNJ) tasks affects the mean estimate (i.e., it often results in a reduction in underestimation) and increases the variance. In three experiments, we investigated the differences between both paradigms as possible causes of this discrepancy: the presence or absence of incongruent stimuli (Experiment 1), the instruction to the observer to either count flashes, beeps or multisensory events (Experiment 2), and the range of pulses presented (Experiment 3). We found significant differences between the mean numerosity estimate of multisensory and unisensory series in Experiment 3, but not in 1 and 2. This suggests that the difference in the range of pulses presented in IF (1-3 pulses) and TNJ (1-10 pulses) is the primary cause of the discrepancy. In the discussion we propose that this result may be explained by the use of two different strategies and their susceptibility to multisensory presentation. For small pulse numbers, observers can accurately count both unisensory and multisensory pulses. For larger numbers, observers can no longer count but will estimate the number based on the pulse series duration which is improved for multisensory stimuli.

Visual cortex as a site of cross-modal integration

It has been shown that visual perception can be strongly affected by auditory stimuli. It is unclear, however, what brain circuitry subserves these interactions. We used the sound-induced illusory flash effect (a single flash accompanied by two auditory beeps is perceived as two flashes (Nature 2000)) as a tool to investigate this question. This illusion is much stronger in the periphery than fovea. In a previous ERP study (NeuroReport 2001), the temporal onsets and patterns of modulation suggested that the modulation of activity by sound occurs within the visual cortex. In the present study we tried to localize the brain regions involved in the perception of the illusory flash more directly and accurately using event-related fMRI. We collected functional images of 3 participants in the following conditions. Unimodal conditions Vp and Vf consisted of visual stimulation: a small disk flashed once in the periphery or fovea, respectively. Bimodal conditions AVp and AVf consisted of auditory-visual stimulation: combination of 2 beeps with visual stimuli Vp and Vf, respectively. In another unimodal condition, Vp2, a physical double flash was presented in the periphery. Trials were randomized. Contrasting (the illusion) condition AVp against Vp resulted in activity in Brodmann's areas 17, 18, and 19. However, contrasting (no-illusion) condition AVf vs. Vf, revealed no differential activity in visual cortex. Thus, the auditory stimulation, per se, or attentional effects, could not account for the visual cortex activity associated with the illusory flash percept. Considering that the visual stimulus was identical in AVp and Vp, the enhanced activity of early visual areas in AVp can only be attributed to the perception of the illusory flash caused by sound. Similar brain areas were indicated when contrasting Vp2 against Vp, corresponding to a physical second flash. These results altogether suggest that the activity in the early visual cortical areas is modulated by sound.