Virtual Limb Research Articles

Body visual discontinuity affects feeling of ownership and skin conductance responses.

When we look at our hands we are immediately aware that they belong to us and we rarely doubt about the integrity, continuity and sense of ownership of our bodies. Here we explored whether the mere manipulation of the visual appearance of a virtual limb could influence the subjective feeling of ownership and the physiological responses (Skin Conductance Responses, SCRs) associated to a threatening stimulus approaching the virtual hand. Participants observed in first person perspective a virtual body having the right hand-forearm (i) connected by a normal wrist (Full-Limb) or a thin rigid wire connection (Wire) or (ii) disconnected because of a missing wrist (m-Wrist) or a missing wrist plus a plexiglass panel positioned between the hand and the forearm (Plexiglass). While the analysis of subjective ratings revealed that only the observation of natural full connected virtual limb elicited high levels of ownership, high amplitudes of SCRs were found also during observation of the non-natural, rigid wire connection condition. This result suggests that the conscious embodiment of an artificial limb requires a natural looking visual body appearance while implicit reactivity to threat may require physical body continuity, even non-naturally looking, that allows the implementation of protective reactions to threat.

Mirror therapy for the alleviation of phantom limb pain following amputation: A literature review

Background/Aim: Phantom limb pain (PLP) is a debilitating condition that affects 50–85% of patients following an amputation and significantly diminishes their quality of life. Mirror therapy has been reported to have potential success for the alleviation of PLP. However, due to a lack of understanding of the mechanisms underlying PLP and the fact that no current consensus as to the effectiveness of mirror therapy exists, guidelines for treatment protocols are lacking. This review aimed to assess the current best evidence for using mirror therapy to treat PLP in patients with amputation. Methods: The authors conducted a systematic review of original research papers specifically investigating the use of mirror therapy in populations of patients experiencing PLP after unilateral limb amputation. Literature was sourced from PubMed, AMED, CINAHL and Google Scholar. The following search terms were used in combination: ‘phantom pain’; ‘PLP’; ‘phantom limb’; ‘phantom limb pain’; ‘mirror’; ‘mirror treatment’; ‘mirror therapy’; and ‘virtual limb’. All available studies were marked against predetermined inclusion and exclusion criteria. Results: Seven primary articles met the inclusion criteria, all of which reported significant PLP alleviation after using mirror therapy, with a trend for achieving phantom limb movement prior to pain relief. Conclusions: Mirror therapy is a promising intervention for PLP. Regular sessions of mirror therapy are required to maintain treatment effects. Causes of PLP and treatment pathways may be multifactorial; therefore, further well-conducted randomised controlled trials are required to identify best practice.

Mere observation of body discontinuity affects perceived ownership and vicarious agency over a virtual hand

The mental representation of one's body typically implies the continuity of its parts. Here, we used immersive virtual reality to explore whether mere observation of visual discontinuity between the hand and limb of an avatar could influence a person's sense of ownership of the virtual body (feeling of ownership, FO) and being the agent of its actions (vicarious agency, VA). In experiment 1, we tested whether placing different amounts of visual discontinuity between a virtual hand and limb differently modulate the perceived FO and VA. Participants passively observed from a first-person perspective four different versions of a virtual limb: (1) a full limb; a hand detached from the proximal part of the limb because of deletion of (2) the wrist; (3) the wrist and forearm; (4) and the wrist, forearm and elbow. After observing the static or moving virtual limb, participants reported their feeling of ownership (FO) and vicarious agency (VA) over the hand. We found that even a small visual discontinuity between the virtual hand and arm significantly decreased participants' FO over the hand during observation of the static limb. Moreover, in the same condition, we found that passive observation of the avatar's actions induced a decrease in both FO and VA. We replicated the same results in a second study (experiment 2) where we investigated the modulation of FO and VA by comparing the visual body discontinuity with a condition in which the virtual limb was partially occluded. Our data show that mere observation of limb discontinuity can change a person's ownership and agency over a virtual body observed from a first-person perspective, even in the absence of any multisensory stimulation of the real body. These results shed new light on the role of body visual continuity in modulating self-awareness and agency in immersive virtual reality.

A simple ERP method for quantitative analysis of cognitive workload in myoelectric prosthesis control and human-machine interaction.

Common goals in the development of human-machine interface (HMI) technology are to reduce cognitive workload and increase function. However, objective and quantitative outcome measures assessing cognitive workload have not been standardized for HMI research. The present study examines the efficacy of a simple event-related potential (ERP) measure of cortical effort during myoelectric control of a virtual limb for use as an outcome tool. Participants trained and tested on two methods of control, direct control (DC) and pattern recognition control (PRC), while electroencephalographic (EEG) activity was recorded. Eighteen healthy participants with intact limbs were tested using DC and PRC under three conditions: passive viewing, easy, and hard. Novel auditory probes were presented at random intervals during testing, and significant task-difficulty effects were observed in the P200, P300, and a late positive potential (LPP), supporting the efficacy of ERPs as a cognitive workload measure in HMI tasks. LPP amplitude distinguished DC from PRC in the hard condition with higher amplitude in PRC, consistent with lower cognitive workload in PRC relative to DC for complex movements. Participants completed trials faster in the easy condition using DC relative to PRC, but completed trials more slowly using DC relative to PRC in the hard condition. The results provide promising support for ERPs as an outcome measure for cognitive workload in HMI research such as prosthetics, exoskeletons, and other assistive devices, and can be used to evaluate and guide new technologies for more intuitive HMI control.

A BP-GA Approach to Solve the Inverse Kinematics of Virtual Human's Upper Limb Kinematic Chain

The human’s Upper Limb Kinematic Chain (ULKC) is the most frequently used and most complicated kinematic chain. It is difficult to compute its Inverse Kinematic (IK) solution quickly and accurately by using neural network or Genetic Algorithm (GA) due to ULKC’s high degree of freedom. Combining BP neural network and GA, we proposed a method to solve its IK problem. Firstly, the joint-units of ULKC and its mathematical model were constructed based on D-H method. Then the BP neural network produced a local optimal solution, which could be served as an individual of GA’s initial population. We could determine the searching domain by its optimal solution. Finally, the high accuracy solution was obtained by using the adaptive GA. The experimental results showed that the proposed approach could obtain the high accuracy solution efficiently with the BP neural network high-speed and GA high-accuracy.

Neural representation of muscle dynamics in voluntary movement control

Several theories of motor control posit that the nervous system has access to a neural representation of muscle dynamics. Yet, this has not been tested experimentally. Should such a representation exist, it was hypothesized that subjects who learned to control a virtual limb using virtual muscles would improve performance faster and show greater generalization than those who learned with a less dynamically complex virtual force generator. Healthy adults practiced using their biceps brachii activity to move a myoelectrically controlled virtual limb from rest to a standard target position with maximum speed and accuracy. Throughout practice, generalization was assessed with untrained target trials and sensitivity to actuator dynamics was probed by unexpected actuator model switches. In a muscle model subject group (n = 10), the biceps electromyographic signal activated a virtual muscle that pulled on the virtual limb with a force governed by muscle dynamics, defined by a nonlinear force-length-velocity relation and series elastic stiffness. A force generator group (n = 10) performed the same task, but the actuation force was a linear function of the biceps activation signal. Both groups made significant errors with unexpected actuator dynamics switches, supporting task sensitivity to actuator dynamics. The muscle model group improved performance as fast as the force generator group and showed greater generalization in early practice, despite using an actuator with more complex dynamics. These results are consistent with a preexisting neural representation of muscle dynamics, which may have offset any learning challenges associated with the more dynamically complex virtual muscle model.

Treatment of phantom limb pain (PLP) based on augmented reality and gaming controlled by myoelectric pattern recognition: a case study of a chronic PLP patient

A variety of treatments have been historically used to alleviate phantom limb pain (PLP) with varying efficacy. Recently, virtual reality (VR) has been employed as a more sophisticated mirror therapy. Despite the advantages of VR over a conventional mirror, this approach has retained the use of the contralateral limb and is therefore restricted to unilateral amputees. Moreover, this strategy disregards the actual effort made by the patient to produce phantom motions. In this work, we investigate a treatment in which the virtual limb responds directly to myoelectric activity at the stump, while the illusion of a restored limb is enhanced through augmented reality (AR). Further, phantom motions are facilitated and encouraged through gaming. The proposed set of technologies was administered to a chronic PLP patient who has shown resistance to a variety of treatments (including mirror therapy) for 48 years. Individual and simultaneous phantom movements were predicted using myoelectric pattern recognition and were then used as input for VR and AR environments, as well as for a racing game. The sustained level of pain reported by the patient was gradually reduced to complete pain-free periods. The phantom posture initially reported as a strongly closed fist was gradually relaxed, interestingly resembling the neutral posture displayed by the virtual limb. The patient acquired the ability to freely move his phantom limb, and a telescopic effect was observed where the position of the phantom hand was restored to the anatomically correct distance. More importantly, the effect of the interventions was positively and noticeably perceived by the patient and his relatives. Despite the limitation of a single case study, the successful results of the proposed system in a patient for whom other medical and non-medical treatments have been ineffective justifies and motivates further investigation in a wider study.

Manipulation of visual biofeedback during gait with a time delayed adaptive Virtual Mirror Box.

BackgroundA mirror placed in the mid-sagittal plane of the body has been used to reduce phantom limb pain and improve movement function in medical conditions characterised by asymmetrical movement control. The mirrored illusion of unimpaired limb movement during gait might enhance the effect, but a physical mirror is only capable of showing parallel movement of limbs in real time typically while sitting. We aimed to overcome the limitations of physical mirrors by developing and evaluating a Virtual Mirror Box which delays the mirrored image of limbs during gait to ensure temporal congruency with the impaired physical limb.MethodsAn application was developed in the CAREN system’s D-Flow software which mirrors selected limbs recorded by real-time motion capture to the contralateral side. To achieve phase shifted movement of limbs during gait, the mirrored virtual limbs are also delayed by a continuously calculated amount derived from past gait events. In order to accommodate non-normal proportions and offsets of pathological gait, the movements are morphed so that the physical and virtual contact events match on the mirrored side. Our method was tested with a trans-femoral amputee walking on a treadmill using his artificial limb. Joint angles of the elbow and knee were compared between the intact and mirrored side using cross correlation, root mean squared difference and correlation coefficients.ResultsThe time delayed adaptive virtual mirror box produced a symmetrical looking gait of the avatar coupled with a reduction of the difference between the intact and virtual knee and elbow angles (10.86° and 5.34° reduced to 4.99° and 2.54° respectively). Dynamic morphing of the delay caused a non-significant change of toe-off events when compared to delaying by 50% of the previous gait cycle, as opposed to the initial contact events which showed a practically negligible but statistically significant increase (p < 0.05).ConclusionsAdding an adaptive time delay to the Virtual Mirror Box has extended its use to treadmill gait, for the first time. Dynamic morphing resulted in a compromise between mirrored movement of the intact side and gait events of the virtual limbs matched with physical events of the impaired side. Asymmetrical but repeatable gait is expected to provide even more faithful mirroring.

An approach to solving the inverse kinematics problem of virtual human's lower limbs kinematic chain

The lower limbs kinematic chain (LLKC) is the important part of human body. It contains two joints with four degrees of freedom (DOF). As it is difficult to compute LLKC’s inverse kinematic solution quickly and accurately by using neural network or genetic algorithm (GA) due to its high degree of freedom, we proposed a BP-GA approach by combining BP neural network and GA to resolve it. Firstly, the mathematical model of LLKC was built based on D-H method. Then BP neural network output a local optimal solution, which could be served as an individual of GA initial population. And the searching domain of GA could be determined by local optimal solution. Finally, the high-accuracy solution was searched by using the adaptive GA. The actions of walking, squatting and seated position were simulated and results showed that the precise solution could be calculated efficiently by using the proposed approach with the BP neural network high-speed and GA high-accuracy.

Validation of a Selective Ensemble-Based Classification Scheme for Myoelectric Control Using a Three-Dimensional Fitts' Law Test

When controlling a powered upper limb prosthesis it is important not only to know how to move the device, but also when not to move. A novel approach to pattern recognition control, using a selective multiclass one-versus-one classification scheme has been shown to be capable of rejecting unintended motions. This method was shown to outperform other popular classification schemes when presented with muscle contractions that did not correspond to desired actions. In this work, a 3-D Fitts' Law test is proposed as a suitable alternative to using virtual limb environments for evaluating real-time myoelectric control performance. The test is used to compare the selective approach to a state-of-the-art linear discriminant analysis classification based scheme. The framework is shown to obey Fitts' Law for both control schemes, producing linear regression fittings with high coefficients of determination (R(2) > 0.936). Additional performance metrics focused on quality of control are discussed and incorporated in the evaluation. Using this framework the selective classification based scheme is shown to produce significantly higher efficiency and completion rates, and significantly lower overshoot and stopping distances, with no significant difference in throughput.

Enhancement of motor imagery‐related cortical activation during first‐person observation measured by functional near‐infrared spectroscopy

It is known that activity in secondary motor areas during observation of human limbs performing actions is affected by the observer's viewpoint, with first-person views generally leading to stronger activation. However, previous neuroimaging studies have displayed limbs in front of the observer, providing an offset view of the limbs without a truly first-person viewpoint. It is unknown to what extent these pseudo-first-person viewpoints have affected the results published to date. In this experiment, we used a horizontal two-dimensional mirrored display that places virtual limbs at the correct egocentric position relative to the observer. We compared subjects using the mirrored and conventional displays while recording over the premotor cortex with functional near-infrared spectroscopy. Subjects watched a first-person view of virtual arms grasping incoming balls on-screen; they were instructed to either imagine the virtual arm as their own [motor imagery during observation (MIO)] or to execute the movements [motor execution (ME)]. With repeated-measures anova, the hemoglobin difference as a direct index of cortical oxygenation revealed significant main effects of the factors hemisphere (P = 0.005) and condition (P ≤ 0.001) with significant post hoc differences between MIO-mirror and MIO-conventional (P = 0.024). These results suggest that the horizontal mirrored display provides a more accurate first-person view, enhancing subjects' ability to perform motor imagery during observation. Our results may have implications for future experimental designs involving motor imagery, and may also have applications in video gaming and virtual reality therapy, such as for patients following stroke.

Illusory Hand Ownership Induced by an MRI Compatible Immersive Virtual Reality Device

In this paper, we introduce a virtual reality based setup for triggering and studying illusory sensations of limb ownership as known from the popular rubber hand illusion. Unlike a similar setup introduced recently, this experiment is triggered by software automatically, which provides for a good synchrony and for high accuracy of visuotactile stimulation. The system is usable inside a magnetic resonance scanner which allows observing cortical activations at the same time. We tested two participants, one highly and the other lowly susceptible to the induction of illusory limb ownership, assessing their subjective experiences as well as their brain activity using functional magnetic resonance imaging. We present preliminary results suggesting that our setup induces illusory ownership experiences for a virtual limb successfully, accompanied by neuronal activation in integrative brain areas.

EMG Signal Decoded Based Virtual Artificial Intelligence Hand Control System

An electromyography (EMG) decoded based virtual artificial intelligence control system has been developed to quantify the performance of real-time control of a multifunctional myoelectric prosthesis. To develop this platform system, a three-dimensional upper limb was simulated by using Solidworks and then implemented into an integrated scene of virtual artificial limb ,which was programmed in virtual reality modeling language (VRML) and performed through Simulink toolbox of the MATLAB. By decoding surface electromyography (sEMG) signals collected from arm muscle surface, the platform system can identify thesix classes of different arm and hand movements and control the virtual artificial limb and/or the physical arms simultaneously. The VR-based platform also provides a relaxant and enjoyable training environment for prosthesis-users in clinic.

Bio-driven system-based virtual reality for prosthetic and rehabilitation systems

Current prosthetic and rehabilitation devices, used for those who are limbless or born with congenital defects or required rehabilitation, are difficult to use. The users have problems to adapt to their new hosts or receiving any bio-feedback despite rehabilitation process and retraining, particularly when working with electromyogram (EMG) signals. In characterizing virtual human limbs, as a potential prosthetic device in three dimensions (3D) virtual reality, patients are able to familiarize themselves with their new appendage and its capabilities or can see their movements’ intention in a Virtual Training Environment. This paper presents a virtual reality (VR)-based design and implementation of a below-shoulder 3D human arm capable of 10-class EMG-based motions driven system of biomedical EMG signal. The method considers a signal classification output as potential control stimulus to drive the virtual limb. A hierarchical design methodology is adopted based on anatomical structure to congruent with virtual reality modeling language (VRML) architecture used in order to progressively build the user interface model and its inherent functionality. The resulting simulation is based on a portable, self-contained VRML prototype implementation paired with an instrumental virtual control-select board capable of actuating any combinations of singular or paired kinematic of 10-class EMG motions. The simulation allows for multiple degree-of-freedom profiles as the classes can be activated independently, or in conjunction with others, allowing enhanced arm movement. Provisions for direct classified control inputs are built into the prototype for holistic system construction.

Virtual limbs and body space: The effects of the rubber hand illusion

Virtual limbs and body space: The effects of the rubber hand illusion

Inducing illusory ownership of a virtual body

We discuss three experiments that investigate how virtual limbs and bodies can come to feel like real limbs and bodies. The first experiment shows that an illusion of ownership of a virtual arm appearing to project out of a person's shoulder can be produced by tactile stimulation on a person's hidden real hand and synchronous stimulation on the seen virtual hand. The second shows that the illusion can be produced by synchronous movement of the person's hidden real hand and a virtual hand. The third shows that a weaker form of the illusion can be produced when a brain–computer interface is employed to move the virtual hand by means of motor imagery without any tactile stimulation. We discuss related studies that indicate that the ownership illusion may be generated for an entire body. This has important implications for the scientific understanding of body ownership and several practical applications.

Virtual limbs and body space: Critical features for the distinction between body space and near-body space.

Hari and Jousmaki (1996) found that motor activity is initiated more efficiently in response to stimuli located on the responding limb as opposed to near the limb. Our research investigated the basis for this difference in spatial coding. We conducted 8 experiments using virtual reality to manipulate the visual feedback resulting from limb movement (spatial location, limb appearance, and nature of response consequences). Our findings replicated the Hari and Jousmaki (1996) effect to reveal that visual appearance and correspondence between the visual and felt position of the limbs was inconsequential. However, it was crucial that visual feedback was a predictable consequence of movement. When this predictability was present, the distinction between the space on the limb (predictable surface) and off the limb (rest of the environment) emerged. We suggest that there may be a functional distinction in coding of space that is and is not predictably controlled by the individual.

Exploratory findings with virtual reality for phantom limb pain; from stump motion to agency and analgesia

Purpose. Phantom limb pain is chronic and intractable. Recently, virtual reality (VR) and motion capture technology has replicated the mirror box device of Ramachandran (Ramachandran et al. Nature 1995, 377, 489–490; Ramachandran and Rogers-Ramachandran Proc R Soc Biol Sci 1996, 263, 377–386) and led to reductions in this pain. We present results from a novel variation on this method which captures motion data directly from a patient's stump (rather than using the opposite remaining limb) and then transforms it into goal directed, virtual action enacted by an avatar in a VR environment.Method. A sample of subjects with ‘arm’ (n = 7) and ‘leg’ (n = 7) amputations underwent trials of a virtual reality (VR) system, controlled by motion captured from their stump which was translated into movements of a virtual limb within the VR environment. Measures of pain in the phantom limb were elicited from patients before and during this exercise as they attempted to gain agency for the movement they saw, and feel embodied within the limb. After this each subject was interviewed about their experiences.Results. Five subjects in each group felt the virtual limb to be moved by them and felt sensations of movement within it. With this they also reported reductions in their phantom limb pain greater than expected from distraction alone. No carry over effect was seen.Conclusions. This technique, which has shown similar success rates to trials of a virtual mirror box, is relatively cheap and portable, and will allow further trials in a home environment.

Observing Virtual Arms that You Imagine Are Yours Increases the Galvanic Skin Response to an Unexpected Threat

Multi-modal visuo-tactile stimulation of the type performed in the rubber hand illusion can induce the brain to temporarily incorporate external objects into the body image. In this study we show that audio-visual stimulation combined with mental imagery more rapidly elicits an elevated physiological response (skin conductance) after an unexpected threat to a virtual limb, compared to audio-visual stimulation alone. Two groups of subjects seated in front of a monitor watched a first-person perspective view of slow movements of two virtual arms intercepting virtual balls rolling towards the viewer. One group was instructed to simply observe the movements of the two virtual arms, while the other group was instructed to observe the virtual arms and imagine that the arms were their own. After 84 seconds the right virtual arm was unexpectedly “stabbed” by a knife and began “bleeding”. This aversive stimulus caused both groups to show a significant increase in skin conductance. In addition, the observation-with-imagery group showed a significantly higher skin conductance (p<0.05) than the observation-only group over a 2-second period shortly after the aversive stimulus onset. No corresponding change was found in subjects' heart rates. Our results suggest that simple visual input combined with mental imagery may induce the brain to measurably temporarily incorporate external objects into its body image.

Towards a digital body: The virtual arm illusion

The integration of the human brain with computers is an interesting new area of applied neuroscience, where one application is replacement of a person's real body by a virtual representation. Here we demonstrate that a virtual limb can be made to feel part of your body if appropriate multisensory correlations are provided. We report an illusion that is invoked through tactile stimulation on a person's hidden real right hand with synchronous virtual visual stimulation on an aligned 3D stereo virtual arm projecting horizontally out of their shoulder. An experiment with 21 male participants showed displacement of ownership towards the virtual hand, as illustrated by questionnaire responses and proprioceptive drift. A control experiment with asynchronous tapping was carried out with a different set of 20 male participants who did not experience the illusion. After 5 min of stimulation the virtual arm rotated. Evidence suggests that the extent of the illusion was also correlated with the degree of muscle activity onset in the right arm as measured by EMG during this period that the arm was rotating, for the synchronous but not the asynchronous condition. A completely virtual object can therefore be experienced as part of one's self, which opens up the possibility that an entire virtual body could be felt as one's own in future virtual reality applications or online games, and be an invaluable tool for the understanding of the brain mechanisms underlying body ownership.