Estimates Of Hand Position Research Articles

Developing RPC-Net: Leveraging High-Density Electromyography and Machine Learning for Improved Hand Position Estimation.

The purpose of this study was to develop and evaluate the performance of RPC-Net (Recursive Prosthetic Control Network), a novel method using simple neural network architectures to translate electromyographic activity into hand position with high accuracy and computational efficiency. RPC-Net uses a regression-based approach to convert forearm electromyographic signals into hand kinematics. We tested the adaptability of the algorithm to different conditions and compared its performance with that of solutions from the academic literature. RPC-Net demonstrated a high degree of accuracy in predicting hand position from electromyographic activity, outperforming other solutions with the same computational cost. Including previous position data consistently improved results across subjects and conditions. RPC-Net showed robustness against a reduction in the number of electromyography electrodes used and shorter input signals, indicating potential for further reduction in computational cost. The results demonstrate that RPC-Net is capable of accurately translating forearm electromyographic activity into hand position, offering a practical and adaptable tool that may be accessible in clinical settings. The development of RPC-Net represents a significant advancement. In clinical settings, its application could enable prosthetic devices to be controlled in a way that feels more natural, improving the quality of life for individuals with limb loss.

HDE-Array: Development and Validation of a New Dry Electrode Array Design to Acquire HD-sEMG for Hand Position Estimation.

This paper aims to introduce HDE-Array (High-Density Electrode Array), a novel dry electrode array for acquiring High-Density surface electromyography (HD-sEMG) for hand position estimation through RPC-Net (Recursive Prosthetic Control Network), a neural network defined in a previous study. We aim to demonstrate the hypothesis that the position estimates returned by RPC-Net using HD-sEMG signals acquired with HDE-Array are as accurate as those obtained from signals acquired with gel electrodes. We compared the results, in terms of precision of hand position estimation by RPC-Net, using signals acquired by traditional gel electrodes and by HDE-Array. As additional validation, we performed a variance analysis to confirm that the presence of only two rows of electrodes does not result in an excessive loss of information, and we characterized the electrode-skin impedance to assess the effects of the voltage divider effect and power line interference. Performance tests indicated that RPC-Net, used with HDE-Array, achieved comparable or superior results to those observed when used with the gel electrode setup. The dry electrodes demonstrated effective performance even with a simplified setup, highlighting potential cost and usability benefits. These results suggest improvements in the accessibility and user-friendliness of upper-limb rehabilitation devices and underscore the potential of HDE-Array and RPC-Net to revolutionize control for medical and non-medical applications.

Approaches for Hybrid Coregistration of Marker-Based and Markerless Coordinates Describing Complex Body/Object Interactions.

Full-body motion capture is essential for the study of body movement. Video-based, markerless, mocap systems are, in some cases, replacing marker-based systems, but hybrid systems are less explored. We develop methods for coregistration between 2D video and 3D marker positions when precise spatial relationships are not known a priori. We illustrate these methods on three-ball cascade juggling in which it was not possible to use marker-based tracking of the balls, and no tracking of the hands was possible due to occlusion. Using recorded video and motion capture, we aimed to transform 2D ball coordinates into 3D body space as well as recover details of hand motion. We proposed four linear coregistration methods that differ in how they optimize ball-motion constraints during hold and flight phases, using an initial estimate of hand position based on arm and wrist markers. We found that minimizing the error between ball and hand estimate was globally suboptimal, distorting ball flight trajectories. The best-performing method used gravitational constraints to transform vertical coordinates and ball-hold constraints to transform lateral coordinates. This method enabled an accurate description of ball flight as well as a reconstruction of wrist movements. We discuss these findings in the broader context of video/motion capture coregistration.

Sensing hand position in Ehlers-Danlos syndrome

Purpose To explore the effect of joint hypermobility on acuity, and precision, of hand proprioception. Materials and methods We compared proprioceptive acuity, and precision, between EDS patients and controls. We then measured any changes in their estimates of hand position after participants adapted their reaches in response to altered visual feedback of their hand. The Beighton Scale was used to quantify the magnitude of joint hypermobility. Results There were no differences between the groups in the accuracy of estimates of hand location, nor in the visually induced changes in hand location. However, EDS patients’ estimates were less precise when based purely on proprioception and could be partially predicted by Beighton score. Conclusions EDS patients are less precise at estimating their hand’s location when only afferent information is available, but the presence of efferent signalling may reduce this imprecision. Those who are more hypermobile are more likely to be imprecise.

Somatotopic Specificity of Perceptual and Neurophysiological Changes Associated with Visuo-proprioceptive Realignment.

When visual and proprioceptive estimates of hand position disagree (e.g., viewing the hand underwater), the brain realigns them to reduce mismatch. This perceptual change is reflected in primary motor cortex (M1) excitability, suggesting potential relevance for hand movement. Here, we asked whether fingertip visuo-proprioceptive misalignment affects only the brain's representation of that finger (somatotopically focal), or extends to other parts of the limb that would be needed to move the misaligned finger (somatotopically broad). In Experiments 1 and 2, before and after misaligned or veridical visuo-proprioceptive training at the index finger, we used transcranial magnetic stimulation to assess M1 representation of five hand and arm muscles. The index finger representation showed an association between M1 excitability and visuo-proprioceptive realignment, as did the pinkie finger representation to a lesser extent. Forearm flexors, forearm extensors, and biceps did not show any such relationship. In Experiment 3, participants indicated their proprioceptive estimate of the fingertip, knuckle, wrist, and elbow, before and after misalignment at the fingertip. Proprioceptive realignment at the knuckle, but not the wrist or elbow, was correlated with realignment at the fingertip. These results suggest the effects of visuo-proprioceptive mismatch are somatotopically focal in both sensory and motor domains.

Proprioceptive recalibration following implicit visuomotor adaptation is preserved in Parkinson's disease.

Individuals with Parkinson's disease (PD) and healthy adults demonstrate similar levels of visuomotor adaptation provided that the distortion is small or introduced gradually, and hence, implicit processes are engaged. Recently, implicit processes underlying visuomotor adaptation in healthy individuals have been proposed to include proprioceptive recalibration (i.e., shifts in one's proprioceptive sense of felt hand position to match the visual estimate of their hand experienced during reaches with altered visual feedback of the hand). In the current study, we asked if proprioceptive recalibration is preserved in PD patients. PD patients tested during their "off" and "on" medication states and age-matched healthy controls reached to visual targets, while visual feedback of their unseen hand was gradually rotated 30° clockwise or translated 4cm rightwards of their actual hand trajectory. As expected, PD patients and controls produced significant reach aftereffects, indicating visuomotor adaptation after reaching with the gradually introduced visuomotor distortions. More importantly, following visuomotor adaptation, both patients and controls showed recalibration in hand position estimates, and the magnitude of this recalibration was comparable between PD patients and controls. No differences for any measures assessed were observed across medication status (i.e., PD off vs PD on). Results reveal that patients are able to adjust their sensorimotor mappings and recalibrate proprioception following adaptation to a gradually introduced visuomotor distortion, and that dopaminergic intervention does not affect this proprioceptive recalibration. These results suggest that proprioceptive recalibration does not involve striatal dopaminergic pathways and may contribute to the preserved visuomotor adaptation that arises implicitly in PD patients.

Adaptive sensor fusion labeling framework for hand pose recognition in robot teleoperation

Purpose The purpose of this paper is to mainly center on the touchless interaction between humans and robots in the real world. The accuracy of hand pose identification and stable operation in a non-stationary environment is the main challenge, especially in multiple sensors conditions. To guarantee the human-machine interaction system’s performance with a high recognition rate and lower computational time, an adaptive sensor fusion labeling framework should be considered in surgery robot teleoperation. Design/methodology/approach In this paper, a hand pose estimation model is proposed consisting of automatic labeling and classified based on a deep convolutional neural networks (DCNN) structure. Subsequently, an adaptive sensor fusion methodology is proposed for hand pose estimation with two leap motions. The sensor fusion system is implemented to process depth data and electromyography signals capturing from Myo Armband and leap motion, respectively. The developed adaptive methodology can perform stable and continuous hand position estimation even when a single sensor is unable to detect a hand. Findings The proposed adaptive sensor fusion method is verified with various experiments in six degrees of freedom in space. The results showed that the clustering model acquires the highest clustering accuracy (96.31%) than other methods, which can be regarded as real gestures. Moreover, the DCNN classifier gets the highest performance (88.47% accuracy and lowest computational time) than other methods. Originality/value This study can provide theoretical and engineering guidance for hand pose recognition in surgery robot teleoperation and design a new deep learning model for accuracy enhancement.

External error attribution dampens efferent-based predictions but not proprioceptive changes in hand localization

In learning and adapting movements in changing conditions, people attribute the errors they experience to a combined weighting of internal or external sources. As such, error attribution that places more weight on external sources should lead to decreased updates in our internal models for movement of the limb or estimating the position of the effector, i.e. there should be reduced implicit learning. However, measures of implicit learning are the same whether or not we induce explicit adaptation with instructions about the nature of the perturbation. Here we evoke clearly external errors by either demonstrating the rotation on every trial, or showing the hand itself throughout training. Implicit reach aftereffects persist, but are reduced in both groups. Only for the group viewing the hand, changes in hand position estimates suggest that predicted sensory consequences are not updated, but only rely on recalibrated proprioception. Our results show that estimating the position of the hand incorporates source attribution during motor learning, but recalibrated proprioception is an implicit process unaffected by external error attribution.

A condition that produces sensory recalibration and abolishes multisensory integration

We examined the influence of extended exposure to a visuomotor rotation, which induces both motor adaptation and sensory recalibration, on (partial) multisensory integration in a cursor-control task. Participants adapted to a 30° (adaptation condition) or 0° (control condition) visuomotor rotation by making center-out movements to remembered targets. In subsequent test trials of sensory integration, they made center-out movements with variable visuomotor rotations and judged the position of hand or cursor at the end of these movements. Test trials were randomly embedded among twice the number of maintenance trials with 30° or 0° rotation. The biases of perceived hand (or cursor) position toward the cursor (or hand) position were measured. We found motor adaptation together with proprioceptive and visual recalibrations in the adaptation condition. Unexpectedly, multisensory integration was absent in both the adaptation and control condition. The absence stemmed from the extensive experience of constant visuomotor rotations of 30° or 0°, which probably produced highly precise predictions of the visual consequences of hand movements. The frequently confirmed predictions then dominated the estimate of the visual movement consequences, leaving no influence of the actual visuomotor rotations in the minority of test trials. Conversely, multisensory integration was present for sensed hand positions when these were indirectly assessed from movement characteristics, indicating that the relative weighting of discrepant estimates of hand position was different for motor control. The existence of a condition that abolishes multisensory integration while keeping sensory recalibration suggests that mechanisms that reduce sensory discrepancies (partly) differ between integration and recalibration.

The effects of awareness of the perturbation during motor adaptation on hand localization.

Awareness of task demands is often used during rehabilitation and sports training by providing instructions which appears to accelerate learning and improve performance through explicit motor learning. However, the effects of awareness of perturbations on the changes in estimates of hand position resulting from motor learning are not well understood. In this study, people adapted their reaches to a visuomotor rotation while either receiving instructions on the nature of the perturbation, experiencing a large rotation, or both to generate awareness of the perturbation and increase the contribution of explicit learning. We found that instructions and/or larger rotations allowed people to activate or deactivate part of the learned strategy at will and elicited explicit changes in open-loop reaches, while a small rotation without instructions did not. However, these differences in awareness, and even manipulations of awareness and perturbation size, did not appear to affect learning-induced changes in hand-localization estimates. This was true when estimates of the adapted hand location reflected changes in proprioception, produced when the hand was displaced by a robot, and also when hand location estimates were based on efferent-based predictions of self-generated hand movements. In other words, visuomotor adaptation led to significant shifts in predicted and perceived hand location that were not modulated by either instruction or perturbation size. Our results indicate that not all outcomes of motor learning benefit from an explicit awareness of the task. Particularly, proprioceptive recalibration and the updating of predicted sensory consequences appear to be largely implicit. (data: https://doi.org/10.17605/osf.io/mx5u2, preprint: https://doi.org/10.31234/osf.io/y53c2)

Increase in weighting of vision vs. proprioception associated with force field adaptation

Hand position can be estimated by vision and proprioception (position sense). The brain is thought to weight and integrate these percepts to form a multisensory estimate of hand position with which to guide movement. Force field adaptation, a type of cerebellum-dependent motor learning, is associated with both motor and proprioceptive changes. The cerebellum has connections with multisensory parietal regions; however, it is unknown if force adaptation is associated with changes in multisensory perception. If force adaptation affects all relevant sensory modalities similarly, the brain’s weighting of vision vs. proprioception should be maintained. Alternatively, if force perturbation is interpreted as somatosensory unreliability, vision may be up-weighted relative to proprioception. We assessed visuo-proprioceptive weighting with a perceptual estimation task before and after subjects performed straight-ahead reaches grasping a robotic manipulandum. Each subject performed one session with a clockwise or counter-clockwise velocity-dependent force field, and one session in a null field. Subjects increased their weight of vision vs. proprioception in the force field session relative to the null session, regardless of force field direction, in the straight-ahead dimension (F1,44 = 5.13, p = 0.029). This suggests that force field adaptation is associated with an increase in the brain’s weighting of vision vs. proprioception.

Experiencing the Cross-Sensory Error Signal During Movement Leads to Proprioceptive Recalibration

Reaching to targets in a virtual reality environment with misaligned visual feedback of the hand results in changes in movements (visuomotor adaptation) and sense of felt hand position (proprioceptive recalibration). We asked if proprioceptive recalibration arises even when the misalignment between visual and proprioceptive estimates of hand position is only experienced during movement. Participants performed a “shooting task” through the targets with a cursor that was rotated 30° clockwise relative to hand motion. Results revealed that, following training on the shooting task, participants adapted their reaches to all targets by approximately 16° and recalibrated their sense of felt hand position by 8°. Thus, experiencing a sensory misalignment between visual and proprioceptive estimates of hand position during movement leads to proprioceptive recalibration.

Adaptation to proprioceptive targets following visuomotor adaptation.

In the following study, we asked if reaches to proprioceptive targets are updated following reach training with a gradually introduced visuomotor perturbation. Subjects trained to reach with distorted hand-cursor feedback, such that they saw a cursor that was rotated or translated relative to their actual hand movement. Following reach training trials with the cursor, subjects reached to Visual (V), Proprioceptive (P) and Visual + Proprioceptive (VP) targets with no visual feedback of their hand. Comparison of reach endpoints revealed that reaches to VP targets followed similar trends as reaches to P targets, regardless of the training distortion introduced. After reaching with a rotated cursor, subjects adapted their reaches to all target types in a similar manner. However, after reaching with a translated cursor, subjects adapted their reach to V targets only. Taken together, these results show that following training with a visuomotor distortion, subjects primarily rely on proprioceptive information when reaching to VP targets. Furthermore, results indicate that reach adaptation to P targets depends on the distortion presented. Training with a rotation distortion leads to changes in reaches to both V and P targets, while a translation distortion, which introduces a constant discrepancy between visual and proprioceptive estimates of hand position throughout the reach, affects changes to V but not P targets.

Modality-specific Changes in Motor Cortex Excitability After Visuo-proprioceptive Realignment.

Spatial realignment of visual and proprioceptive estimates of hand position is necessary both to keep the estimates in register over time and to compensate for sensory perturbations. Such realignment affects perceived hand position, which the brain must use to plan hand movements. We would therefore expect visuo-proprioceptive realignment to affect the motor system at some level, but the physiological basis of this interaction is unknown. Here, we asked whether activity in primary motor cortex (M1), a well-known substrate of motor control, shows evidence of change after visuo-proprioceptive realignment. In two sessions each, 32 healthy adults experienced spatially misaligned or veridical visual and proprioceptive information about their static left index finger. Participants indicated perceived finger position with no performance feedback or knowledge of results. Using TMS over the M1 representation of the misaligned finger, we found no average difference between sessions. However, regression analysis indicated that, in the misaligned session only, proprioceptive realignment was linked with a decrease in M1 activity and visual realignment was linked with an increase in M1 activity. Proprioceptive and visual realignment were inversely related to each other. These results suggest that visuo-proprioceptive realignment does indeed have a physiological impact on the motor system. The lack of a between-session mean difference in M1 activity suggests that the basis of the effect is not the multisensory realignment computation itself, independent of modality. Rather, the changes in M1 are consistent with a modality-specific neural mechanism, such as modulation of somatosensory cortex or dorsal stream visual areas that impact M1.

慣性センサを用いた手先位置姿勢推定とロボット教示システムへの適用

慣性センサを用いた手先位置姿勢推定とロボット教示システムへの適用

Separating Predicted and Perceived Sensory Consequences of Motor Learning.

During motor adaptation the discrepancy between predicted and actually perceived sensory feedback is thought to be minimized, but it can be difficult to measure predictions of the sensory consequences of actions. Studies attempting to do so have found that self-directed, unseen hand position is mislocalized in the direction of altered visual feedback. However, our lab has shown that motor adaptation also leads to changes in perceptual estimates of hand position, even when the target hand is passively displaced. We attribute these changes to a recalibration of hand proprioception, since in the absence of a volitional movement, efferent or predictive signals are likely not involved. The goal here is to quantify the extent to which changes in hand localization reflect a change in the predicted sensory (visual) consequences or a change in the perceived (proprioceptive) consequences. We did this by comparing changes in localization produced when the hand movement was self-generated (‘active localization’) versus robot-generated (‘passive localization’) to the same locations following visuomotor adaptation to a rotated cursor. In this passive version, there should be no predicted consequences of these robot-generated hand movements. We found that although changes in localization were somewhat larger in active localization, the passive localization task also elicited substantial changes. Our results suggest that the change in hand localization following visuomotor adaptation may not be based entirely on updating predicted sensory consequences, but may largely reflect changes in our proprioceptive state estimate.

Proprioceptive recalibration arises slowly compared to reach adaptation.

When subjects reach in a novel visuomotor environment (e.g. while viewing a cursor representing their hand that is rotated from their hand's actual position), they typically adjust their movements (i.e. bring the cursor to the target), thus reducing reaching errors. Additionally, research has shown that reaching with altered visual feedback of the hand results in sensory changes, such that proprioceptive estimates of hand position are shifted in the direction of the visual feedback experienced (Cressman and Henriques in J Neurophysiol 102:3505-3518, 2009). This study looked to establish the time course of these sensory changes. Additionally, the time courses of implicit sensory and motor changes were compared. Subjects reached to a single visual target while seeing a cursor that was either aligned with their hand position (50 trials) or rotated 30° clockwise relative to their hand (150 trials). Reach errors and proprioceptive estimates of felt hand position were assessed following the aligned reach training trials and at seven different times during the rotated reach training trials by having subjects reach to the target without visual feedback, and provide estimates of their hand relative to a visual reference marker, respectively. Results revealed a shift in proprioceptive estimates throughout the rotated reach training trials; however, significant sensory changes were not observed until after 70 trials. In contrast, results showed a greater change in reaches after a limited number of reach training trials with the rotated cursor. These findings suggest that proprioceptive recalibration arises more slowly than reach adaptation.

The role of internal forward models and proprioception in hand position estimation.

Our ability to properly move and react in different situations is largely dependent on our perception of our limbs' position. At least three sources- vision, proprioception, and internal forward models (FMs)- seem to contribute to this perception. To the best of our knowledge, the effect of each source has not been studied individually. Specifically, role of FM has been ignored in some previous studies. We hypothesized that FM has a critical role in subjects' perception which needs to be considered in the relevant studies to obtain more reliable results. Therefore, we designed an experiment with the goal of investigating FM and proprioception role in subjects' perception of their hand's position. Three groups of subjects were recruited in the study. Based on the experiment design, it was supposed that subjects in different groups relied on proprioception, FM, and both of them for estimating their unseen hand's position. Comparing the results of three groups revealed significant difference between their estimation' errors. FM provided minimum estimation error, while proprioception had a bias error in the tested region. Integrating proprioception with FM decreased this error. Integration of two Gaussian functions, fitted to the error distribution of FM and proprioception groups, was simulated and created a mean error value almost similar to the experimental observation. These results suggest that FM role needs to be considered when studying the perceived position of the limbs. This can lead to gain better insights into the mechanisms underlying the perception of our limbs' position which might have potential clinical and rehabilitation applications, e.g., in the postural control of elderly which are at high risk of falls and injury because of deterioration of their perception with age.

The cerebellum is not necessary for visually driven recalibration of hand proprioception

Decades of research have implicated both cortical and subcortical areas, such as the cerebellum, as playing an important role in motor learning, and even more recently, in predicting the sensory consequences of movement. Still, it is unknown whether the cerebellum also plays a role in recalibrating sensory estimates of hand position following motor learning. To test this, we measured proprioceptive estimates of static hand position in 19 cerebellar patients with local ischemic lesions and 19 healthy controls, both before and after reach training with altered visual feedback of the hand. This altered visual feedback, (30° cursor-rotation) was gradually introduced in order to facilitate reach adaptation in the patient group. We included two different types of training (in separate experiments): the typical visuomotor rotation training where participants had full volition of their hand movements when reaching with the cursor, and sensory exposure training where the direction of participants׳ hand movements were constrained and gradually deviated from the cursor motion (Cressman, E. K., Henriques, D. Y., 2010. Reach adaptation and proprioceptive recalibration following exposure to misaligned sensory input. J. Neurophysiol., vol. 103, pp. 1888–1895). We found that both healthy individuals and patients showed equivalent shifts in their felt hand position following both types of training. Likewise, as expected given that the cursor-rotation was introduced gradually, patients showed comparable reach aftereffects to those of controls in both types of training. The robust change in felt hand position across controls and cerebellar patients suggests that the cerebellum is not involved in proprioceptive recalibration of the hand.

The role of the cross-sensory error signal in visuomotor adaptation

Reaching to targets with misaligned visual feedback of the hand leads to changes in proprioceptive estimates of hand position and reach aftereffects. In such tasks, subjects are able to make use of two error signals: the discrepancy between the desired and actual movement, known as the sensorimotor error signal, and the discrepancy between visual and proprioceptive estimates of hand position, which we refer to as the cross-sensory error signal. We have recently shown that mere exposure to a sensory discrepancy in the absence of goal-directed movement (i.e. no sensorimotor error signal) is sufficient to produce similar changes in felt hand position and reach aftereffects. Here, we sought to determine the extent that this cross-sensory error signal can contribute to proprioceptive recalibration and movement aftereffects by manipulating the magnitude of this signal in the absence of volitional aiming movements. Subjects pushed their hand out along a robot-generated linear path that was gradually rotated clockwise relative to the path of a cursor. On all trials, subjects viewed a cursor that headed directly towards a remembered target while their hand moved out synchronously. After exposure to a 30° rotated hand-cursor distortion, subjects recalibrated their sense of felt hand position and adapted their reaches. However, no additional increases in recalibration or aftereffects were observed following further increases in the cross-sensory error signal (e.g. up to 70°). This is in contrast to our previous study where subjects freely reached to targets with misaligned visual hand position feedback, hence experiencing both sensorimotor and cross-sensory errors, and the distortion magnitude systematically predicted increases in proprioceptive recalibration and reach aftereffects. Given these findings, we suggest that the cross-sensory error signal results in changes to felt hand position which drive partial reach aftereffects, while larger aftereffects that are produced after visuomotor adaptation (and that vary with the size of distortion) are related to the sensorimotor error signal.