Cross-modal Illusion Research Articles

How bodily perception parallels distal perception

Abstract Bodily perception is standardly contrasted with distal perception. In this paper, I argue that the standard view overlooks a significant respect in which bodily perception parallels distal perception. The parallel is motivated by a comparison between two cross-modal illusions. The ventriloquism effect manifests cross-modal relations between the distal senses of vision and audition and multi-sensory experiences jointly constituted by vision and audition. Thermal referral is a laboratory induced illusion that involves the bodily sensory systems underlying the perception of pressure and heat. I argue that thermal referral indicates the presence of cross-modal relations and multi-sensory experiences of a type sufficiently similar to those manifested by the ventriloquism effect to suggest that pressure and thermal perception parallel vision and audition at the level of bodily perception.

Inner speech and the body error theory.

Inner speech is commonly understood as the conscious experience of a voice within the mind. One recurrent theme in the scientific literature is that the phenomenon involves a representation of overt speech, for example, a representation of phonetic properties that result from a copy of speech instructions that were ultimately suppressed. I propose a larger picture that involves some embodied objects and their misperception. I call it "the Body Error Theory," or BET for short. BET is a form of illusionism, but the particular version I favor is a cross-modal illusion. Newly described here, my hypothesis is that the experience of inner speech arises from a mix of interoception and audition. Specifically, there is the detection of slight but well-confirmed activities in the speech musculature that occur during inner speech, which helps to transform representations of normal but quiet nonverbal sounds that inevitably occur during inner speech, from breathing to background noise, into a mistaken perception of inner speech. Simply put, activities in the speech musculature mix with sounds to create the appearance of speech sounds, which thus explains the "voice within the mind." I also show how BET's cross-modal system fits with standard information processing accounts for speech monitoring and how it accommodates the central insights of leading theories of inner speech. In addition, I show how BET is supported by data from experience-sampling surveys and how it can be empirically tested against its rivals.

Facilitation of imitative movement in patients with chronic hemiplegia triggered by illusory ownership

The sense of body ownership, the feeling that one’s body belongs to oneself, is a crucial subjective conscious experience of one’s body. Recent methodological advances regarding crossmodal illusions have provided novel insights into how multisensory interactions shape human perception and cognition, underpinning conscious experience, particularly alteration of body ownership. Moreover, in post-stroke rehabilitation, encouraging the use of the paretic limb in daily life is considered vital, as a settled sense of ownership and attentional engagement toward the paralyzed body part may promote increased frequency of its use and prevent learned nonuse. Therefore, in addition to traditional methods, novel interventions using neurorehabilitation techniques that induce self-body recognition are needed. This study investigated whether the illusory experience of a patient’s ownership alterations of their paretic hand facilitates the enhancement in the range of motion of succeeding imitation movements. An experiment combining a modified version of the rubber hand illusion with imitation training was conducted with chronic hemiplegia. A larger imitation movement of the paretic hand was observed in the illusion-induced condition, indicating that the feeling of ownership toward the observed limb promotes the induction of intrinsic potential for motor performance. This training, using subjective experience, may help develop new post-stroke rehabilitation interventions.

Crossmodal Postdiction: Conscious Perception as Revisionist History.

Postdiction occurs when later stimuli influence the perception of earlier stimuli. As the multisensory science field has grown in recent decades, the investigation of crossmodal postdictive phenomena has also expanded. Crossmodal postdiction can be considered (in its simplest form) the phenomenon in which later stimuli in one modality influence earlier stimuli in another modality (e.g., Intermodal Apparent Motion). Crossmodal postdiction can also appear in more nuanced forms, such as unimodal postdictive illusions (e.g., Apparent Motion) that are influenced by concurrent crossmodal stimuli (e.g., Crossmodal Influence on Apparent Motion), or crossmodal illusions (e.g., the Double Flash Illusion) that are influenced postdictively by a stimulus in one or the other modality (e.g., a visual stimulus in the Illusory Audiovisual Rabbit Illusion). In this review, these and other varied forms of crossmodal postdiction will be discussed. Three neuropsychological models proposed for unimodal postdiction will be adapted to the unique aspects of processing and integrating multisensory stimuli. Crossmodal postdiction opens a new window into sensory integration, and could potentially be used to identify new mechanisms of crossmodal crosstalk in the brain.

Tactile temporal offset cues reduce visual representational momentum

The perception of dynamic objects is sometimes biased. For example, localizing a moving object after it has disappeared results in a perceptual shift in the direction of motion, a bias known as representational momentum. We investigated whether the temporal characteristics of an irrelevant, spatially uninformative vibrotactile stimulus bias the perceived location of a visual target. In two visuotactile experiments, participants judged the final location of a dynamic, visual target. Simultaneously, a continuous (starting with the onset of the visual target, Experiments 1 and 2) or brief (33-ms stimulation, Experiment 2) vibrotactile stimulus (at the palm of participant’s hands) was presented, and the offset disparity between the visual target and tactile stimulation was systematically varied. The results indicate a cross-modal influence of tactile stimulation on the perceived final location of the visual target. Closer inspection of the nature of this cross-modal influence, observed here for the first time, reveals that the vibrotactile stimulus was likely just taken as a temporal cue regarding the offset of the visual target, but no strong interaction and combined processing of the two stimuli occurred. The present results are related to similar cross-modal temporal illusions and current accounts of multisensory perception, integration, and cross-modal facilitation.

Crossmodal Postdiction: Conscious Perception as Revisionist History

Postdiction occurs when later stimuli influence the perception of earlier stimuli. As the multisensory science field has grown in recent decades, the investigation of crossmodal postdictive phenomena has also expanded. Crossmodal postdiction can be considered (in its simplest form) the phenomenon in which later stimuli in one modality influence earlier stimuli in another modality (e.g., Intermodal Apparent Motion). Crossmodal postdiction can also appear in more nuanced forms, such as unimodal postdictive illusions (e.g., Apparent Motion) that are influenced by concurrent crossmodal stimuli (e.g., Crossmodal Influence on Apparent Motion), or crossmodal illusions (e.g., the Double Flash Illusion) that are influenced postdictively by a stimulus in one or the other modality (e.g., a visual stimulus in the Illusory Audiovisual Rabbit Illusion). In this review, these and other varied forms of crossmodal postdiction will be discussed. Three neuropsychological models proposed for unimodal postdiction will be adapted to the unique aspects of processing and integrating multisensory stimuli. Crossmodal postdiction opens a new window into sensory integration, and could potentially be used to identify new mechanisms of crossmodal crosstalk in the brain.

Crossmodal Postdiction: Conscious Perception as Revisionist History †

Postdiction occurs when later stimuli influence the perception of earlier stimuli. As the multisensory science field has grown in recent decades, the investigation of crossmodal postdictive phenomena has also expanded. Crossmodal postdiction can be considered (in its simplest form) the phenomenon in which later stimuli in one modality influence earlier stimuli in another modality (e.g., Intermodal Apparent Motion). Crossmodal postdiction can also appear in more nuanced forms, such as unimodal postdictive illusions (e.g., Apparent Motion) that are influenced by concurrent crossmodal stimuli (e.g., Crossmodal Influence on Apparent Motion), or crossmodal illusions (e.g., the Double Flash Illusion) that are influenced postdictively by a stimulus in one or the other modality (e.g., a visual stimulus in the Illusory Audiovisual Rabbit Illusion). In this review, these and other varied forms of crossmodal postdiction will be discussed. Three neuropsychological models proposed for unimodal postdiction will be adapted to the unique aspects of processing and integrating multisensory stimuli. Crossmodal postdiction opens a new window into sensory integration, and could potentially be used to identify new mechanisms of crossmodal crosstalk in the brain.

Changing Body Representation Through Full Body Ownership Illusions Might Foster Motor Rehabilitation Outcome in Patients With Stroke.

How our brain represents our body through the integration of internal and external sensory information so that we can interact with our surrounding environment has become a matter of interest especially in the field of neurorehabilitation. In this regard, there is an increasing interest in the use of multisensory integration techniques—such as the use of body ownership illusions—to modulate distorted body representations after brain damage. In particular, cross-modal illusions such as mirror visual feedback therapy (MVFT) have been widely used for motor rehabilitation. Despite the effectiveness of the MVFT for motor rehabilitation, there are some limitations to fully modify the distorted internal representation of the paretic limb in patients with stroke. A possible explanation for this relies on the physical limitations of the mirror in reproducing upper-limb distortions, which can result in a reduced sense of ownership of the mirrored limb. New digital technologies such as virtual reality (VR) and 360° videos allow researchers to create body ownership illusions by adapting virtual bodies so that they represent specific morphological characteristics including upper-limb distortions. In this manuscript, we present a new rehabilitation approach that employs full virtual body ownership illusions, using a 360° video system, for the assessment and modulation of the internal representation of the affected upper limb in stroke patients. We suggest modifying the internal representation of the upper limb to a normal position before starting motor rehabilitation training.

Multisensorial Perception in Chronic Migraine and the Role of Medication Overuse

Multisensory processing can be assessed by measuring susceptibility to crossmodal illusions such as the Sound-Induced Flash Illusion (SIFI). When a single flash is accompanied by 2 or more beeps, it is perceived as multiple flashes (fission illusion); conversely, a fusion illusion is experienced when more flashes are matched with a single beep, leading to the perception of a single flash. Such illusory perceptions are associated to crossmodal changes in visual cortical excitability. Indeed, increasing occipital cortical excitability, by means of transcranial electrical currents, disrupts the SIFI (ie, fission illusion). Similarly, a reduced fission illusion was shown in patients with episodic migraine, especially during the attack, in agreement with the pathophysiological model of cortical hyperexcitability of this disease. If episodic migraine patients present with reduced SIFI especially during the attack, we hypothesize that chronic migraine (CM) patients should consistently report less illusory effects than healthy controls; drugs intake could also affect SIFI. On such a basis, we studied the proneness to SIFI in CM patients (n = 63), including 52 patients with Medication Overuse Headache (MOH), compared to 24 healthy controls. All migraine patients showed reduced fission phenomena than controls (P < .0001). Triptan MOH patients (n = 23) presented significantly less fission effects than other CM groups (P = .008). This exploratory study suggests that CM – both with and without medication overuse – is associated to a higher visual cortical responsiveness which causes deficit of multisensorial processing, as assessed by the SIFI. PerspectiveThis observational study shows reduced susceptibility to the SIFI in CM, confirming and extending previous results in episodic migraine. MOH contributes to this phenomenon, especially in case of triptans.

Binding and differentiation in multisensory object perception

Cognitive scientists have long known that the modalities interact during perceptual processing. Cross-modal illusions like the ventriloquism effect show that the course of processing in one modality can alter the course of processing in another. But how do the modalities interact in the specific domain of object perception? This paper distinguishes and analyzes two kinds of multisensory interaction in object perception. First, the modalities may bind features to a single object or event. Second, the modalities may cooperate when differentiating an object or event from its surroundings. I critically evaluate evidence for various forms of multisensory binding. I then consider the case for multisensory differentiation. I argue that existing evidence for multisensory differentiation is inconclusive. I highlight ways that the issue might be empirically resolved.

Cathodal Occipital tDCS Is Unable to Modulate the Sound Induced Flash Illusion in Migraine.

Migraine is a highly disabling disease characterized by recurrent pain. Despite an intensive effort, mechanisms of migraine pathophysiology still represent an unsolved issue. Evidence from both animal and human studies suggests that migraine is characterized by hyperresponsivity or hyperexcitability of sensory cortices, especially the visual cortex. This phenomenon, in turn, may affect multisensory processing. Indeed, migraineurs present with an abnormal, reduced, perception of the Sound-induced Flash Illusion (SiFI), a crossmodal illusion that relies on optimal integration of visual and auditory stimuli by the occipital visual cortex. Decreasing visual cortical excitability with transcranial direct current stimulation (tDCS) can increase the SiFI in healthy subjects. Moving away from these issues, we applied cathodal tDCS over the visual cortex of migraineurs, with and without aura, in order to decrease cortical excitability and thus physiologically restoring the perception of a reliable SiFI. Differently from our expectations, tDCS was unable to reliably modulate SiFI in migraine. The chronic, relatively excessive, visual cortex hyperexcitability, featuring the migraineur brain, may render tDCS ineffective for restoring multisensory processing in this disease.

Psychophysiological effects of threatening a rubber hand that is perceptually embodied in healthy human subjects

In this issue of Scandinavian Journal of Pain, a study by Johnson nd colleagues investigated the roles of arousal and threatening n the peripersonal space during a rubber hand illusion [1]. The tudy is designed to evaluate whether or not arousal is selectively ssociated with threatening stimuli entering into the peripersonal pace. The study uses the paradigm of the rubber hand illusion, here participants are induced to behave as if a rubber hand is art of their own body. The illusion is usually induced by brushing person’s hand, hidden from view, while synchronously brushing visible rubber hand. In this study, the authors used brushing to imulate a non-threatening condition and a needle to modulate a hreatening condition. This paper outlines one of themost intriguing aspects: how selfody representation influences perception and cognition. Recent heories are suggesting that perception and cognition are influnced by internal representations of the body. This hypothesis is orroborated by research evidence showing for example that vision f the body increases spatial acuity of touch. Both experimental and clinical studies also highlight the imporance of body representation in the modulation of pain experience sing cross-modal illusions. These studies show that body selfepresentation and pain perception are related.

Visual limitations shape audio-visual integration.

Recent studies have proposed that some cross-modal illusions might be expressed in what were previously thought of as sensory-specific brain areas. Therefore, one interesting question is whether auditory-driven visual illusory percepts respond to manipulations of low-level visual attributes (such as luminance or chromatic contrast) in the same way as their nonillusory analogs. Here, we addressed this question using the double flash illusion (DFI), whereby one brief flash can be perceived as two when combined with two beeps presented in rapid succession. Our results showed that the perception of two illusory flashes depended on luminance contrast, just as the temporal resolution for two real flashes did. Specifically we found that the higher the luminance contrast, the stronger the DFI. Such a pattern seems to contradict what would be predicted from a maximum likelihood estimation perspective, and can be explained by considering that low-level visual stimulus attributes similarly modulate the perception of sound-induced visual phenomena and "real" visual percepts. This finding provides psychophysical support for the involvement of sensory-specific brain areas in the expression of the DFI. On the other hand, the addition of chromatic contrast failed to produce a change in the strength of the DFI despite it improved visual sensitivity to real flashes. The null impact of chromaticity on the cross-modal illusion might suggest a weaker interaction of the parvocellular visual pathway with the auditory system for cross-modal illusions.

Crossmodal illusions in neurorehabilitation.

In everyday life, many diverse bits of information, simultaneously derived from the different sensory channels, converge into discrete brain areas, and are ultimately synthetized into unified percepts. Such multisensory integration can dramatically alter the phenomenal experience of both environmental events and our own body. Crossmodal illusions are one intriguing product of multisensory integration. This review describes and discusses the main clinical applications of the most known crossmodal illusions in rehabilitation settings. We consider evidence highlighting the contribution of crossmodal illusions to restore, at least in part, defective mechanisms underlying a number of disorders of body representation related to pain, sensory, and motor impairments in neuropsychological and neurological diseases, and their use for improving neuroprosthetics. This line of research is enriching our understanding of the relationships between multisensory functions and the pathophysiological mechanisms at the basis of a number of brain disorders. The review illustrates the potential of crossmodal illusions for restoring disarranged spatial and body representations, and, in turn, different pathological symptoms.

Visual cortex hyperexcitability in migraine in response to sound-induced flash illusions.

Sound-induced flash illusions depend on visual cortical excitability. In this study, we explored whether sound-induced flash illusions are perceived differently in migraine, a condition associated with pathologic cortical hyperexcitability. Sound-induced flash illusions were examined in 59 migraine patients (mean age = 32 ± 16 years; 36 females), 32 without aura and 27 with aura, and in 24 healthy controls (mean age = 42 ± 17 years; 16 females). Patients were studied during attacks and interictally. Visual stimuli (flashes) accompanied by sounds (beeps) were presented in different combinations: a single flash with multiple beeps was given to induce the perception of multiple flashes ("fission" illusion), and multiple flashes with a single beep were used to reduce the number of perceived flashes ("fusion" illusion). For migraineurs, the fission illusion was reduced, especially during the attack, and almost abolished when a single flash was combined with 2 beeps (except for those without aura tested interictally); the fusion illusion was less consistently reported in both migraine groups, but not completely disrupted. Results from this study add novel clues to our understanding of visual cortex hyperexcitability in migraine, especially migraine with aura. Furthermore, these analyses underscore how pathologic changes in cortical excitability affect multisensory interactions. Cross-modal illusions represent a valid tool for exploration of functional connectivity between sensory areas, which likely has an important role in the pathophysiology of migraine.

1D4-2 体験記憶が微触感錯覚に与える影響の評価

1D4-2 体験記憶が微触感錯覚に与える影響の評価

The sound-induced phosphene illusion

Crossmodal illusions clearly show how perception, rather than being a modular and self-contained function, can be dramatically altered by interactions between senses. Here, we provide evidence for a novel crossmodal "physiological" illusion, showing that sounds can boost visual cortical responses in such a way to give rise to a striking illusory visual percept. In healthy participants, a single-pulse transcranial magnetic stimulation (sTMS) delivered to the occipital cortex evoked a visual percept, i.e., a phosphene. When sTMS is accompanied by two auditory beeps, the second beep induces in neurologically unimpaired participants the perception of an illusory second phosphene, namely the sound-induced phosphene illusion. This perceptual "fission" of a single phosphene, due to multiple beeps, is not matched by a "fusion" of double phosphenes due to a single beep, and it is characterized by an early auditory modulation of the TMS-induced visual responses (~80ms). Multiple beeps also induce an illusory feeling of multiple TMS pulses on the participants' scalp, consistent with an audio-tactile fission illusion. In conclusion, an auditory stimulation may bring about a phenomenological change in the conscious visual experience produced by the transcranial stimulation of the occipital cortex, which reveals crossmodal binding mechanisms within early stages of visual processing.

Illusory and Veridical Mapping of Tactile Objects in the Primary Somatosensory and Posterior Parietal Cortex

While several behavioral and neuroscience studies have explored visual, auditory, and cross-modal illusions, information about the phenomenology and neural correlates of somatosensory illusions is meager. By combining psychophysics and somatosensory evoked potentials, we explored in healthy humans the neural correlates of 2 compelling tactuo-proprioceptive illusions, namely Aristotle (1 object touching the contact area between 2 crossed fingers is perceived as 2 lateral objects) and Reverse illusions (2 lateral objects are perceived as 1 between crossed-fingers object). These illusions likely occur because of the tactuo-proprioceptive conflict induced by fingers being crossed in a non-natural posture. We found that different regions in the somatosensory stream exhibit different proneness to the illusions. Early electroencephalographic somatosensory activity (at 20 ms) originating in the primary somatosensory cortex (S1) reflects the phenomenal rather than the physical properties of the stimuli. Notably, later activity (around 200 ms) originating in the posterior parietal cortex is higher when subjects resist the illusions. Thus, while S1 activity is related to illusory perception, PPC acts as a conflict resolver that recodes tactile events from somatotopic to spatiotopic frames of reference and ultimately enables veridical perception.

Interactions between Senses: Updating on Neural Mechanisms and Behavioral Evidence

Interactions between Senses: Updating on Neural Mechanisms and Behavioral Evidence

Cross-modal Processing in the Occipito-temporal Cortex: A TMS Study of the Müller-Lyer Illusion

The Müller-Lyer illusion occurs both in vision and in touch, and transfers cross-modally from vision to haptics [Mancini, F., Bricolo, E., & Vallar, G. Multisensory integration in the Müller-Lyer illusion: From vision to haptics. Quarterly Journal of Experimental Psychology, 63, 818-830, 2010]. Recent evidence suggests that the neural underpinnings of the Müller-Lyer illusion in the visual modality involve the bilateral lateral occipital complex (LOC) and right superior parietal cortex (SPC). Conversely, the neural correlates of the haptic and cross-modal illusions have never been investigated previously. Here we used repetitive TMS (rTMS) to address the causal role of the regions activated by the visual illusion in the generation of the visual, haptic, and cross-modal visuo-haptic illusory effects, investigating putative modality-specific versus cross-modal underlying processes. rTMS was administered to the right and the left hemisphere, over occipito-temporal cortex or SPC. rTMS over left and right occipito-temporal cortex impaired both unisensory (visual, haptic) and cross-modal processing of the illusion in a similar fashion. Conversely, rTMS interference over left and right SPC did not affect the illusion in any modality. These results demonstrate the causal involvement of bilateral occipito-temporal cortex in the representation of the visual, haptic, and cross-modal Müller-Lyer illusion, in favor of the hypothesis of shared underlying processes. This indicates that occipito-temporal cortex plays a cross-modal role in perception both of illusory and nonillusory shapes.