Synchronous Multisensory Stimulation Research Articles

Enhancing bodily self-consciousness in the virtual world with synchronous multisensory stimulation, interoceptive feedback and peripersonal space expansion: A narrative review

The article explores the link between neuroscience and the design fields by discussing works on predictive brain, conscious mind, perception, and peripersonal space to increase bodily self-consciousness in the virtual world. Unlike the physical reality, the virtual world often lacks the intricate bodily feedback necessary for self-consciousness, thus resulting in less complete virtual representations. By incorporating findings from neuroscience and the virtual world, this research suggests a theoretical approach to enhance bodily self-consciousness in different virtual worlds, particularly augmented or diminished mixed realities and holistic virtual reality. This is achieved through synchronous multisensory stimulation (visuo-tactile), interoceptive feedback, and peripersonal space expansion (visuo-tactile and audio–tactile). As a narrative review, the article not only suggests a theoretical method but also highlights the potential for future practical experiments to apply these insights.

Look at me now! Enfacement illusion over computer-generated faces.

According to embodied cognition research, one's bodily self-perception can be illusory and temporarily shifted toward an external body. Similarly, the so-called "enfacement illusion" induced with a synchronous multisensory stimulation over the self-face and an external face can result in implicit and explicit changes in the bodily self. The present study aimed to verify (i) the possibility of eliciting an enfacement illusion over computer-generated faces and (ii) which multisensory stimulation condition was more effective. A total of 23 participants were asked to look at a gender-matched avatar in three synchronous experimental conditions and three asynchronous control conditions (one for each stimulation: visuotactile, visuomotor, and simple exposure). After each condition, participants were asked to complete a questionnaire assessing both the embodiment and the enfacement sensations to address different facets of the illusion. Results suggest a stronger effect of synchronous vs. asynchronous stimulation, and the difference was more pronounced for the embodiment items of the questionnaire. We also found a greater effect of visuotactile and visuomotor stimulations as compared to the simple exposure condition. These findings support the enfacement illusion as a new paradigm to investigate the ownership of different face identities and the specific role of visuotactile and visuomotor stimulations with virtual reality stimuli.

Multisensory integration induces body ownership of a handtool, but not any handtool.

Bodily boundaries are computed by integrating multisensory bodily signals and can be experimentally manipulated using bodily illusions. Research on tool use demonstrates that tools alter body representations motorically to account for changes in a user's action repertoire. The present experiment sought to unify perceptual and motoric accounts of tool embodiment using a modified Rubber Hand Illusion (RHI) that also addressed the skill and practice aspects of the tool use literature. In Experiment 1, synchronous multisensory stimulation induced perceptual embodiment of a tool, chopsticks. The embodiment of chopsticks was stronger for more skilled participants, and if the illusion was preceded by tool use. In Experiment 2, the illusion was not elicited with a different type of tool, a teacup, showing that not all objects can be incorporated. This experiment helps to clarify the role of perceptual and motoric embodiment and suggests future avenues for research into tools embodiment using this method.

Temporal factors affecting somatosensory-auditory interactions in speech processing.

Speech perception is known to rely on both auditory and visual information. However, sound-specific somatosensory input has been shown also to influence speech perceptual processing (Ito et al., 2009). In the present study, we addressed further the relationship between somatosensory information and speech perceptual processing by addressing the hypothesis that the temporal relationship between orofacial movement and sound processing contributes to somatosensory–auditory interaction in speech perception. We examined the changes in event-related potentials (ERPs) in response to multisensory synchronous (simultaneous) and asynchronous (90 ms lag and lead) somatosensory and auditory stimulation compared to individual unisensory auditory and somatosensory stimulation alone. We used a robotic device to apply facial skin somatosensory deformations that were similar in timing and duration to those experienced in speech production. Following synchronous multisensory stimulation the amplitude of the ERP was reliably different from the two unisensory potentials. More importantly, the magnitude of the ERP difference varied as a function of the relative timing of the somatosensory–auditory stimulation. Event-related activity change due to stimulus timing was seen between 160 and 220 ms following somatosensory onset, mostly around the parietal area. The results demonstrate a dynamic modulation of somatosensory–auditory convergence and suggest the contribution of somatosensory information for speech processing process is dependent on the specific temporal order of sensory inputs in speech production.

Virtual reality for assessment of patients suffering chronic pain: a case study

The study of body representation and ownership has been a very active research area in recent years. Synchronous multisensory stimulation has been used for the induction of the illusion of ownership over virtual body parts and even full bodies, and it has provided experimental paradigms for the understanding of the brain processing of body representation. However, the illusion of ownership of a virtual body has rarely been used for patient evaluation and diagnosis. Here we propose a method that exploits ownership of a virtual body in combination with a simple brain computer interface (BCI) and basic physiological measures to complement neurological assessment. A male patient presenting a fixed posture dystonia featuring a permanently closed left fist participated in this case study. The patient saw a virtual body that substituted his own after donning a head-mounted display and thereby entering the virtual reality. The left virtual hand had the same posture as his corresponding real hand. After inducing virtual hand ownership by correlated visuo-tactile stimulation and dynamic reflections in a virtual mirror, the virtual hand would open either automatically or through a cognitive task assessed through a BCI that required him to focus attention on the virtual hand. The results reveal that body ownership induced changes on electromyography and BCI performance in the patient that were different from those in five healthy controls. Overall, the case study shows that the induction of virtual body ownership combined with simple electrophysiological measures could be useful for the diagnosis of patients with neurological conditions.

Extending Body Space in Immersive Virtual Reality: A Very Long Arm Illusion

Recent studies have shown that a fake body part can be incorporated into human body representation through synchronous multisensory stimulation on the fake and corresponding real body part – the most famous example being the Rubber Hand Illusion. However, the extent to which gross asymmetries in the fake body can be assimilated remains unknown. Participants experienced, through a head-tracked stereo head-mounted display a virtual body coincident with their real body. There were 5 conditions in a between-groups experiment, with 10 participants per condition. In all conditions there was visuo-motor congruence between the real and virtual dominant arm. In an Incongruent condition (I), where the virtual arm length was equal to the real length, there was visuo-tactile incongruence. In four Congruent conditions there was visuo-tactile congruence, but the virtual arm lengths were either equal to (C1), double (C2), triple (C3) or quadruple (C4) the real ones. Questionnaire scores and defensive withdrawal movements in response to a threat showed that the overall level of ownership was high in both C1 and I, and there was no significant difference between these conditions. Additionally, participants experienced ownership over the virtual arm up to three times the length of the real one, and less strongly at four times the length. The illusion did decline, however, with the length of the virtual arm. In the C2–C4 conditions although a measure of proprioceptive drift positively correlated with virtual arm length, there was no correlation between the drift and ownership of the virtual arm, suggesting different underlying mechanisms between ownership and drift. Overall, these findings extend and enrich previous results that multisensory and sensorimotor information can reconstruct our perception of the body shape, size and symmetry even when this is not consistent with normal body proportions.

Transcending the Self – the Illusion of Body Ownership in Immersive Virtual Reality and its Impact on Behaviour

Virtual reality in various forms has been around for about 40 years. It has been considered mainly as a technology that can be used to generate the illusion of a transformation of place. However, it has recently been shown that it can successfully be used to transcend the self, through illusions of body ownership and transformation. Several papers have shown that it is possible to generate in people the illusory sense of ownership of a fake body using virtual reality techniques [1-5]. This can be achieved through synchronous multisensory stimulation with respect to the real and virtual body. For example, the participant sees his or her virtual body touched, while feeling the touch synchronously and in the same place on the real body. This can also lead to illusions of body transformation, such as a thin person having the illusion of being fat [6]. Our research suggests the prime importance of a first person perspective for the illusion of ownership, however, in [7] we also found that a representation of a...

Self-other bodily merging in the context of synchronous but arbitrary-related multisensory inputs

A debated issue in the multisensory literature concerns the relative contribution of bottom-up sensory components versus top-down cognitive elaborations in contributing to the rise and persistency of bodily illusion. Previous studies, for instance, have shown that simultaneity of sensory inputs and plausibility of the stimulated object play an important role in the rubber hand phenomenon, whereas violation of tactile expectancy does not disrupt the illusory feeling to own a fake hand. The present research examined this issue in the context of the "enfacement" phenomenon (i.e., self-other face-perception modification), using entirely arbitrary and non-ecological pairs of visual and tactile events. Visual and tactile stimulation was matched in terms of spatial location, but not linked by any previously learned associations, making temporal synchrony a critical binding factor. Participants received electro-tactile stimulations on their cheek, while they watched the face of a stranger illuminated on the cheek with a dot of white light. Synchronous (vs. asynchronous) stimulations yielded the enfacement effect. In addition, the stranger stimulated in synchrony was judged as more similar, physically and in terms of personality, and as closer to the self. These findings suggest that synchronous multisensory stimulation on the face can produce both perceptual and social binding, even in the absence of any previously learned associations between the stimulations.

Multisensory Stimulation Can Induce an Illusion of Larger Belly Size in Immersive Virtual Reality

BackgroundBody change illusions have been of great interest in recent years for the understanding of how the brain represents the body. Appropriate multisensory stimulation can induce an illusion of ownership over a rubber or virtual arm, simple types of out-of-the-body experiences, and even ownership with respect to an alternate whole body. Here we use immersive virtual reality to investigate whether the illusion of a dramatic increase in belly size can be induced in males through (a) first person perspective position (b) synchronous visual-motor correlation between real and virtual arm movements, and (c) self-induced synchronous visual-tactile stimulation in the stomach area.MethodologyTwenty two participants entered into a virtual reality (VR) delivered through a stereo head-tracked wide field-of-view head-mounted display. They saw from a first person perspective a virtual body substituting their own that had an inflated belly. For four minutes they repeatedly prodded their real belly with a rod that had a virtual counterpart that they saw in the VR. There was a synchronous condition where their prodding movements were synchronous with what they felt and saw and an asynchronous condition where this was not the case. The experiment was repeated twice for each participant in counter-balanced order. Responses were measured by questionnaire, and also a comparison of before and after self-estimates of belly size produced by direct visual manipulation of the virtual body seen from the first person perspective.ConclusionsThe results show that first person perspective of a virtual body that substitutes for the own body in virtual reality, together with synchronous multisensory stimulation can temporarily produce changes in body representation towards the larger belly size. This was demonstrated by (a) questionnaire results, (b) the difference between the self-estimated belly size, judged from a first person perspective, after and before the experimental manipulation, and (c) significant positive correlations between these two measures. We discuss this result in the general context of body ownership illusions, and suggest applications including treatment for body size distortion illnesses.

Synchronous Multisensory Stimulation Blurs Self-Other Boundaries

In a study that builds on recent cognitive neuroscience research on body perception and social psychology research on social relations, we tested the hypothesis that synchronous multisensory stimulation leads to self-other merging. We brushed the cheek of each study participant as he or she watched a stranger’s cheek being brushed in the same way, either in synchrony or in asynchrony. We found that this multisensory procedure had an effect on participants’ body perception as well as social perception. Study participants exposed to synchronous stimulation showed more merging of self and the other than participants exposed to asynchronous stimulation. The degree of self-other merging was determined by measuring participants’ body sensations and their perception of face resemblance, as well as participants’ judgment of the inner state of the other, closeness felt toward the other, and conformity behavior. The results of this study show how multisensory integration can affect social perception and create a sense of self-other similarity.

Hands only illusion: multisensory integration elicits sense of ownership for body parts but not for non-corporeal objects

The experience of body ownership can be successfully manipulated during the rubber hand illusion using synchronous multisensory stimulation. The hypothesis that multisensory integration is both a necessary and sufficient condition for body ownership is debated. We systematically varied the appearance of the object that was stimulated in synchrony or asynchrony with the participant's hand. A viewed object that was transformed in three stages from a plain wooden block to a wooden hand was compared to a realistic rubber hand. Introspective and behavioural results show that participants experience a sense of ownership only for the realistic prosthetic hand, suggesting that not all objects can be experienced as part of one's body. Instead, the viewed object must fit with a reference model of the body that contains important structural information about body parts. This body model can distinguish between corporeal and non-corporeal objects, and it therefore plays a critical role in maintaining a coherent sense of one's body.