Visuomotor Synchrony Research Articles

The role of visuomotor synchrony on virtual full-body illusions in children and adults.

The present study explored the effects of visuomotor synchrony in virtual reality during the embodiment of a full human avatar in children (aged 5-6 years) and adults. Participants viewed their virtual bodies from a first-person perspective while they moved the body during self-generated and structured movement. Embodiment was measured via questions and psychophysiological responses (skin conductance) to a virtual body-threat and during both movement conditions. Both children and adults had increased feelings of ownership and agency over a virtual body during synchronous visuomotor feedback (compared to asynchronous visuomotor feedback). Children had greater ownership compared to adults during synchronous movement but did not differ from adults on agency. There were no differences in SCRs (frequency or magnitude) between children and adults, between conditions (i.e., baseline or movement conditions) or visuomotor feedback. Collectively, the study highlights the importance of visuomotor synchrony for children's ratings of embodiment for a virtual avatar from at least 5 years old, and suggests adults and children are comparable in terms of psychophysiological arousal when moving (or receiving a threat to) a virtual body. This has important implications for our understanding of the development of embodied cognition and highlights the considerable promise of exploring visuomotor VR experiences in children.

Body ownership and kinaesthetic illusions: Dissociated bodily experiences for distinct levels of body consciousness?

Seeing an embodied humanoid avatar move its arms can induce in the observer the illusion that its own (static) arms are moving accordingly, the kinematic signals emanating from this avatar thus being considered like those from the biological body. Here, we investigated the causal relationship between these kinaesthetic illusions and the illusion of body ownership, manipulated through visuomotor synchronisation. The results of two experiments revealed that the sense of body ownership over an avatar seen from a first-person perspective was intimately linked to visuomotor synchrony. This was not the case for kinaesthetic illusions indicating that when superimposed on the biological body, the avatar is inevitably treated at the sensorimotor level as one’s own body, whether consciously considered as such or not. The question of whether these two bodily experiences (body ownership and kinaesthetic illusion) are underpinned by distinct representations, the body image, and the body schema, is discussed.

Temporal visuomotor synchrony induces embodiment towards an avatar with biomechanically impossible arm movements

Visuomotor synchrony in time and space induces a sense of embodiment towards virtual bodies experienced in first-person view using Virtual Reality (VR). Here, we investigated whether temporal visuomotor synchrony affects avatar embodiment even when the movements of the virtual arms are spatially altered from those of the user in a non-human-like manner. In a within-subjects design VR experiment, participants performed a reaching task controlling an avatar whose lower arms bent in inversed and biomechanically impossible directions from the elbow joints. They performed the reaching task using this “unnatural avatar” as well as a “natural avatar,” whose arm movements and positions spatially matched the user. The reaching tasks were performed with and without a one second delay between the real and virtual movements. While the senses of body ownership and agency towards the unnatural avatar were significantly lower compared to those towards the natural avatar, temporal visuomotor synchrony did significantly increase the sense of embodiment towards the unnatural avatar as well as the natural avatar. These results suggest that temporal visuomotor synchrony is crucial for inducing embodiment even when the spatial match between the real and virtual limbs is disrupted with movements outside the pre-existing cognitive representations of the human body.

Sense of body ownership and body agency in schizophrenia

Recent research suggests that embodiment sensations (sense of body ownership and sense of body agency) are altered in schizophrenia. Using a mirror box illusion setup, we tested if the anomalous embodiment experience depends on deficient processing of visuomotor synchrony, disrupted processing of movement mode, or both. The task required participants to press a lever with their index while looking at the image of the experimenter’s hand moving on a similar lever. The illusion of embodiment could arise because looking toward the direction of their own hand the participant saw the reflection of the experimenter’s hand visually superimposed to his own one through a mirror. During the illusion induction, we systematically varied visuomotor asynchrony (4 delays were imposed on the movement of the experimenter’s hand) and the mode of movement (the participant could perform active vs. passive movements). The strength of the illusion of embodiment of the external hand was assessed with explicit judgments of ownership and agency. Patients’ data showed an anomalous modulation of ownership with respect to visuomotor synchrony manipulation and an altered modulation of agency with respect to both visuomotor synchrony and movement mode manipulations. Results from the present study suggest that impairments affecting both the processing of temporal aspects of visuomotor signals and the processing of type of movement underlie anomalous embodiment sensations in schizophrenia. Hypotheses about potential deficits accounting for our results are proposed.

Virtual Reality Embodiment in Motor Imagery Brain–Computer Interface Training

This study investigates how the avatar embodiment in virtual reality (VR) influences training for operation of motor imagery brain–computer interfaces (MI-BCIs). In previous studies, we demonstrated the possibility to induce a BCI-based VR avatar embodiment (using mental commands instead of motion tracking) for purposes of training facilitation. This paper examines the relationship between BCI performance and subjective levels of embodiment, as well as differences in BCI performance achieved with training in two different VR environments. MI-BCI training variants with the following forms of feedback were studied: a) with feedback exploiting avatar embodiment in immersive VR (avatar movements indicating the user performance), b) with feedback exploiting avatar embodiment in gamified and progressively paced VR training, and c) with symbolic feedback in the standard training with Graz protocol. On-line performance from the BCI experiments and questionnaires on the sense of ownership and sense of agency toward the virtual avatar were studied. Questionnaire analysis showed that a robust sense of embodiment arose in the VR training environments, with a strong correlation between the reported ownership and agency toward the avatar. Interestingly, the achieved BCI performance was uncorrelated with both the ownership and the agency. Using gamification further increased the performance (but not the reported sense of ownership) in the training session. Embodiment in VR mediated by synchrony between mental commands and visual stimulation in VR arose under different conditions than embodiment based on visuo-motor synchrony. Consistency between the perceived sense of ownership and agency plays a more important role than the ability to issue MI-BCI commands correctly. These findings help to elucidate the positive effect of embodiment to initial steps in BCI training and can be leveraged in future MI-BCI training designs.

Follow Your Nose: Extended Arm Reach After Pinocchio Illusion in Virtual Reality

In this study, we recreate the Pinocchio Illusion—a bodily illusion whereby the perceived length of one’s nose is extended—in Virtual Reality. Participants (n = 38) self-administered tapping on the tip of the nose of a virtual avatar seen from the first-person perspective (using a hand-held controller) while the nose of the avatar slowly grew with each tap. The stimulating virtual arm and the virtual nose were linked such that while the nose grew the arm extended, and then also grew up to 50%. This produced an extension of the perceived reach of the stimulating arm, and an outward drift in the participants’ real arm. A positive correlation between the extent of the outward drift of the participants’ arm and the perceived reachability of distal objects was observed. These results were found both with synchronous tactile stimulation on the participants’ real nose, and without, but not for control conditions in which the visuomotor synchrony or body schema were violated. These findings open new avenues for hand grasp interactions with virtual objects out of arm’s-reach in immersive setups and are discussed in the context of theories of body ownership, body schema, and touch perception.

Social context facilitates visuomotor synchrony and bonding in children and adults

Interpersonal synchrony is a fundamental part of human social interaction, with known effects on facilitating social bonding. Moving in time with another person facilitates prosocial behaviour, however, it is unknown if the degree of synchronisation predicts the degree of social bonding. Similarly, while people readily fall in synchrony even without being instructed to do so, we do not know whether such spontaneous synchronisation elicits similar prosocial effects as instructed synchronisation. Across two studies, we investigated how context (social vs non-social stimulus) and instruction (instructed vs uninstructed) influenced synchronisation accuracy and bonding with the interaction partner in adults and children. The results revealed improved visuomotor synchrony within a social, compared to non-social, context in adults and children. Children, but not adults, synchronised more accurately when instructed to synchronise than when uninstructed. For both children and adults, synchronisation in a social context elicited stronger social bonding towards an interaction partner as compared to synchronisation in a non-social context. Finally, children’s, but not adults’, degree of synchrony with the partner was significantly associated with their feelings of social closeness. These findings illuminate the interaction of sensorimotor coupling and joint action in social contexts and how these mechanisms facilitate synchronisation ability and social bonding.

Virtual Mirror and Beyond: The Psychological Basis for Avatar Embodiment via a Mirror

In virtual reality (VR), a virtual mirror is often used to display the VR avatar to the user for enhancing the embodiment. The reflected image of the synchronization of the virtual body with the user’s movement is expected to be recognized as the user’s own reflection. In addition to the visuo-motor synchrony, there are some mirror reflection factors that are probably involved in avatar embodiment. This paper reviews literature on the psychological studies that involve mirror-specific self-identification and embodied perception to clarify how the reflected image of the virtual body is embodied. Furthermore, subjective misconceptions about mirror reflections reported in naïve optics have also been reviewed to discuss the potential of virtual mirror displays to modulate avatar embodiment.

Development of the bodily self: Effects of visuomotor synchrony and visual appearance on virtual embodiment in children and adults

The sense of a bodily self is thought to depend on adaptive weighting and integration of bodily afferents and prior beliefs. Evidence from studies using paradigms such as the rubber hand illusion and full body illusion suggests changes in the integration of visuotactile bodily signals throughout childhood. Here, we extended this line of research by assessing how bottom-up visuomotor synchrony and expectancy, modulated by visual appearance of virtual avatars, contribute to embodiment in children. We compared responses to a first-person perspective virtual full body illusion from 8- to 12-year-old children and adults while manipulating synchrony of the avatar’s movements (synchronous, 0.5-s delay, or 1-s delay compared with the participant’s movements) and appearance of the avatar (human or skeleton). We measured embodiment with both subjective questionnaires and objective skin conductance responses to virtual threat. Results showed that children experienced ownership for the virtual avatar in a similar way as adults, which was reduced with increasing asynchrony, and for the skeleton avatar as compared with the human avatar. This modulation of ownership was not reflected in the skin conductance responses, which were equally high in all experimental conditions and only showed a modulation of repetition by age. In contrast, in children the subjective experience of agency was less affected by the dampening effects of visuomotor asynchrony or reduced human likeness and was overall higher. These findings suggest that children can easily embody a virtual avatar but that different aspects of embodiment develop at different rates, which could have important implications for applications of embodied virtual reality.

My Virtual Self: The Role of Movement in Children's Sense of Embodiment.

There are vast potential applications for children's entertainment and education with modern virtual reality (VR) experiences, yet we know very little about how the movement or form of such a virtual body can influence children's feelings of control (agency) or the sensation that they own the virtual body (ownership). In two experiments, we gave a total of 197 children aged 4-14 years a virtual hand which moved synchronously or asynchronously with their own movements and had them interact with a VR environment. We found that movement synchrony influenced feelings of control and ownership at all ages. In Experiment 1 only, participants additionally felt haptic feedback either congruently, delayed or not at all - this did not influence feelings of control or ownership. In Experiment 2 only, participants used either a virtual hand or non-human virtual block. Participants embodied both forms to some degree, provided visuomotor signals were synchronous (as indicated by ownership, agency, and location ratings). Yet, only the hand in the synchronous movement condition was described as feeling like part of the body, rather than like a tool (e.g., a mouse or controller). Collectively, these findings highlight the overall dominance of visuomotor synchrony for children's own-body representation; that children can embody non-human forms to some degree; and that embodiment is also somewhat constrained by prior expectations of body form.

Re-association of Body Parts: Illusory Ownership of a Virtual Arm Associated With the Contralateral Real Finger by Visuo-Motor Synchrony.

Illusory ownership can be induced in a virtual body by visuo-motor synchrony. Our aim was to test the possibility of a re-association of the right thumb with a virtual left arm and express the illusory body ownership of the re-associated arm through a synchronous or asynchronous movement of the body parts through action and vision. Participants felt that their right thumb was the virtual left arm more strongly in the synchronous condition than in the asynchronous one, and the feeling of ownership of the virtual arm was also stronger in the synchronous condition. We did not find a significant difference in the startle responses to a sudden knife appearance to the virtual arm between the two synchrony conditions, as there was no proprioceptive drift of the thumb. These results suggest that a re-association of the right thumb with the virtual left arm could be induced by visuo-motor synchronization; however, it may be weaker than the natural association.

My body until proven otherwise: Exploring the time course of the full body illusion.

Evidence from the Full Body Illusion (FBI) has shown that adults can embody full bodies which are not their own when they move synchronously with their own body or are viewed from a first-person perspective. However, there is currently no consensus regarding the time course of the illusion. Here, for the first time, we examined the effect of visuomotor synchrony (synchronous/asynchronous/no movement) on the FBI over time. Surprisingly, we found evidence of embodiment over a virtual body after five seconds in all conditions. Embodiment decreased with increased exposure to asynchronous movement, but remained high in synchronous and no movement conditions. We suggest that embodiment of a body seen from a first-person perspective is felt by default, and that embodiment can then be lost in the face of contradictory cues. These results have significant implications for our understanding of how multisensory cues contribute to embodiment.

Self Beyond the Body: Action-Driven and Task-Relevant Purely Distal Cues Modulate Performance and Body Ownership

Our understanding of body ownership largely relies on the so-called Rubber Hand Illusion (RHI). In this paradigm, synchronous stroking of the real and the rubber hands leads to an illusion of ownership of the rubber hand provided that it is physically, anatomically, and spatially plausible. Self-attribution of an artificial hand also occurs during visuomotor synchrony. In particular, participants experience ownership over a virtual or a rubber hand when the visual feedback of self-initiated movements follows the trajectory of the instantiated motor commands, such as in the Virtual Hand Illusion (VHI) or the moving Rubber Hand Illusion (mRHI). Evidence yields that both when the cues are triggered externally (RHI) and when they result from voluntary actions (VHI and mRHI), the experience of ownership is established through bottom-up integration and top-down prediction of proximodistal cues (visuotactile or visuomotor) within the peripersonal space. It seems, however, that depending on whether the sensory signals are externally (RHI) or self-generated (VHI and mRHI), the top-down expectation signals are qualitatively different. On the one hand, in the RHI the sensory correlations are modulated by top-down influences which constitute empirically induced priors related to the internal (generative) model of the body. On the other hand, in the VHI and mRHI body ownership is actively shaped by processes which allow for continuous comparison between the expected and the actual sensory consequences of the actions. Ample research demonstrates that the differential processing of the predicted and the reafferent information is addressed by the central nervous system via an internal (forward) model or corollary discharge. Indeed, results from the VHI and mRHI suggest that, in action-contexts, the mechanism underlying body ownership could be similar to the forward model. Crucially, forward models integrate across all self-generated sensory signals including not only proximodistal (i.e., visuotactile or visuomotor) but also purely distal sensory cues (i.e., visuoauditory). Thus, if body ownership results from a consistency of a forward model, it will be affected by the (in)congruency of purely distal cues provided that they inform about action-consequences and are relevant to a goal-oriented task. Specifically, they constitute a corrective error signal. Here, we explicitly addressed this question. To test our hypothesis, we devised an embodied virtual reality-based motor task where action outcomes were signaled by distinct auditory cues. By manipulating the cues with respect to their spatial, temporal and semantic congruency, we show that purely distal (visuoauditory) feedback which violates predictions about action outcomes compromises both performance and body ownership. These results demonstrate, for the first time, that body ownership is influenced by not only externally and self-generated cues which pertain to the body within the peripersonal space but also those arising outside of the body. Hence, during goal-oriented tasks body ownership may result from the consistency of forward models.

A Virtual Out-of-Body Experience Reduces Fear of Death.

Immersive virtual reality can be used to visually substitute a person’s real body by a life-sized virtual body (VB) that is seen from first person perspective. Using real-time motion capture the VB can be programmed to move synchronously with the real body (visuomotor synchrony), and also virtual objects seen to strike the VB can be felt through corresponding vibrotactile stimulation on the actual body (visuotactile synchrony). This setup typically gives rise to a strong perceptual illusion of ownership over the VB. When the viewpoint is lifted up and out of the VB so that it is seen below this may result in an out-of-body experience (OBE). In a two-factor between-groups experiment with 16 female participants per group we tested how fear of death might be influenced by two different methods for producing an OBE. In an initial embodiment phase where both groups experienced the same multisensory stimuli there was a strong feeling of body ownership. Then the viewpoint was lifted up and behind the VB. In the experimental group once the viewpoint was out of the VB there was no further connection with it (no visuomotor or visuotactile synchrony). In a control condition, although the viewpoint was in the identical place as in the experimental group, visuomotor and visuotactile synchrony continued. While both groups reported high scores on a question about their OBE illusion, the experimental group had a greater feeling of disownership towards the VB below compared to the control group, in line with previous findings. Fear of death in the experimental group was found to be lower than in the control group. This is in line with previous reports that naturally occurring OBEs are often associated with enhanced belief in life after death.

Dissociation of agency and body ownership following visuomotor temporal recalibration.

Bodily self-consciousness consists of one’s sense of agency (I am causing an action) and body ownership (my body belongs to me). Both stem from the temporal congruence between different modalities, although some visuomotor temporal incongruence is acceptable for agency. To examine the association or dissociation between agency and body ownership in the context of different temporal sensitivities, we applied a temporal recalibration paradigm, in which subjective synchrony between asynchronous hand action and its visual feedback can be perceived after exposure to the asynchronous visuomotor stimulation. In the experiment, participants continuously clasped and unclasped their hand while watching an online video of their hand that was presented with delays of 50, 110, 170, 230, 290, and 350 ms. Then, they rated a video of their hand with a delay of 50 ms (test stimulus) with respect to the synchrony between hand action and hand video and the perceived agency over the video. Moreover, proprioceptive drift of participants’ hand location toward the hand video during the exposure was measured as an index of illusory body ownership. Results indicated that perception of agency emerged over the delayed hand video as subjective visuomotor synchrony was recalibrated, but that body ownership did not emerge for the delayed video, even after the recalibration. We suggest that there is a dissociation between agency and body ownership following visuomotor temporal recalibration.

The robot hand illusion: Inducing proprioceptive drift through visuo-motor congruency

The representation of one׳s own body sets the border of the self, but also shapes the space where we interact with external objects. Under particular conditions, such as in the rubber hand illusion external objects can be incorporated in one׳s own body representation, following congruent visuo-tactile stroking of one׳s own and a fake hand. This procedure induces an illusory sense of ownership for the fake hand and a shift of proprioceptive localization of the own hand towards the fake hand. Here we investigated whether pure visuo-motor, instead of visuo-tactile, congruency between one׳s own hand and a detached myoelectric-controlled robotic hand can induce similar embodiment effects. We found a shift of proprioceptive hand localization toward the robot hand, only following synchronized real hand/robot hand movements. Notably, no modulation was found of the sense of ownership following either synchronous or asynchronous-movement training. Our findings suggest that visuo-motor synchrony can drive the localization of one׳s own body parts in space, even when somatosensory input is kept constant and the experience of body ownership is maintained.

Agency over a phantom limb and electromyographic activity on the stump depend on visuomotor synchrony: a case study.

Most patients, post-amputation, report the experience of a phantom limb. Some even sense voluntary movements when viewing a mirror image of the intact limb superimposed onto the phantom limb. While delayed visual feedback of an action is known to reduce a sense of agency, the effect of delayed visual feedback on phantom motor sensation (i.e., sense of controlling a phantom limb) has not been examined. Using a video-projection system, we examined the effect of delayed visual feedback on phantom motor sensation in an upper-limb amputee (male; left upper-limb amputation). He was instructed to view mirrored video images of his intact hand clasping and unclasping during a phantom limb movement. He then rated the intensity of the phantom motor sensation. Three types of hand movement images were presented as follows: synchronous, asynchronous with a 250-ms delay, and asynchronous with a 500-ms delay. Results showed that phantom motor sensation decreased when the image was delayed by 250 and 500 ms. However, when we instructed the patient to adjust the phase of phantom limb movement to that of the image with a 500-ms delay, phantom motor sensation increased. There was also a positive correlation between intensity of phantom motor sensation and electromyographic (EMG) activity on deltoids at the patient’s stump. These results suggest that phantom motor sensation and EMG activity on the stump depend on visuomotor synchrony and top-down effects.

The Effects of Rotating the Self Out of the Body in the Full Virtual Body Ownership Illusion

It has been shown that it is possible to induce a strong illusion that a virtual body (VB) is one's own body. However, the relative influence of a first-person-perspective (1PP) view of the VB and spatial coincidence of the real body and VB remains unclear. We demonstrate a method that permits separation of these two factors. It provides a 1PP view of a VB, supporting visuomotor synchrony between real body and VB movements, but where the entire scene including the body is rotated 15 degrees upwards through the axis connecting the eyes, so that the VB and real body are coincident only through this axis. In a within-subjects study that compared this 15 degrees rotation with a 0 degrees rotation condition, participants reported only slightly diminished levels of perceived ownership of the VB in the rotated condition and did not detect the rotation of the scene. These results indicate that strong spatial coincidence of the virtual and real bodies is not necessary for a full-body ownership illusion. The rotation method used, similar to the effects of vertical prisms, did not produce significant negative side-effects, thus providing a useful methodology for further investigations of body ownership.

Putting yourself in the skin of a black avatar reduces implicit racial bias

Although it has been shown that immersive virtual reality (IVR) can be used to induce illusions of ownership over a virtual body (VB), information on whether this changes implicit interpersonal attitudes is meager. Here we demonstrate that embodiment of light-skinned participants in a dark-skinned VB significantly reduced implicit racial bias against dark-skinned people, in contrast to embodiment in light-skinned, purple-skinned or with no VB. 60 females participated in this between-groups experiment, with a VB substituting their own, with full-body visuomotor synchrony, reflected also in a virtual mirror. A racial Implicit Association Test (IAT) was administered at least three days prior to the experiment, and immediately after the IVR exposure. The change from pre- to post-experience IAT scores suggests that the dark-skinned embodied condition decreased implicit racial bias more than the other conditions. Thus, embodiment may change negative interpersonal attitudes and thus represent a powerful tool for exploring such fundamental psychological and societal phenomena.

Human Tails: Ownership and Control of Extended Humanoid Avatars

This paper explores body ownership and control of an 'extended' humanoid avatar that features a distinct and flexible tail-like appendage protruding from its coccyx. Thirty-two participants took part in a between-groups study to puppeteer the avatar in an immersive CAVE™ -like system. Participants' body movement was tracked, and the avatar's humanoid body synchronously reflected this motion. However, sixteen participants experienced the avatar's tail moving around randomly and asynchronous to their own movement, while the other participants experienced a tail that they could, potentially, control accurately and synchronously through hip movement. Participants in the synchronous condition experienced a higher degree of body ownership and agency, suggesting that visuomotor synchrony enhanced the probability of ownership over the avatar body despite of its extra-human form. Participants experiencing body ownership were also more likely to be more anxious and attempt to avoid virtual threats to the tail and body. The higher task performance of participants in the synchronous condition indicates that people are able to quickly learn how to remap normal degrees of bodily freedom in order to control virtual bodies that differ from the humanoid form. We discuss the implications and applications of extended humanoid avatars as a method for exploring the plasticity of the brain's representation of the body and for gestural human-computer interfaces.