Control Of Upper Limb Prostheses Research Articles

Impact of Shared Control Modalities on Performance and Usability of Semi-autonomous Prostheses.

Semi-autonomous (SA) control of upper-limb prostheses can improve the performance and decrease the cognitive burden of a user. In this approach, a prosthesis is equipped with additional sensors (e.g., computer vision) that provide contextual information and enable the system to accomplish some tasks automatically. Autonomous control is fused with a volitional input of a user to compute the commands that are sent to the prosthesis. Although several promising prototypes demonstrating the potential of this approach have been presented, methods to integrate the two control streams (i.e., autonomous and volitional) have not been systematically investigated. In the present study, we implemented three shared control modalities (i.e., sequential, simultaneous, and continuous) and compared their performance, as well as the cognitive and physical burdens imposed on the user. In the sequential approach, the volitional input disabled the autonomous control. In the simultaneous approach, the volitional input to a specific degree of freedom (DoF) activated autonomous control of other DoFs, whereas in the continuous approach, autonomous control was always active except for the DoFs controlled by the user. The experiment was conducted in ten able-bodied subjects, and these subjects used an SA prosthesis to perform reach-and-grasp tasks while reacting to audio cues (dual tasking). The results demonstrated that, compared to the manual baseline (volitional control only), all three SA modalities accomplished the task in a shorter time and resulted in less volitional control input. The simultaneous SA modality performed worse than the sequential and continuous SA approaches. When systematic errors were introduced in the autonomous controller to generate a mismatch between the goals of the user and controller, the performance of SA modalities substantially decreased, even below the manual baseline. The sequential SA scheme was the least impacted one in terms of errors. The present study demonstrates that a specific approach for integrating volitional and autonomous control is indeed an important factor that significantly affects the performance and physical and cognitive load, and therefore these should be considered when designing SA prostheses.

The effects of forearm movements on human gait during walking with various self-selected speeds

The forearms significantly contribute to the upper extremity movements and, consequently, whole-body responses during locomotion. The purpose of this study is to provide a more in-depth understanding of the mechanism controlling forearm movements during walking by comprehensively investigating the effects of the forearms on the lower and upper limb movements. Such an understanding can provide critical information for the design and control of robotic upper-limb prostheses. Twelve healthy young participants were recruited to compare their gait during (1) natural walking, (2) walking while wearing a pair of artificial passive forearms and having their actual forearms restrained by orthopedic braces, and (3) walking with only having their forearms restrained by the braces (i.e., no artificial forearms). While the passive forearms in condition 2 were to determine if the forearm movements were passively or actively controlled, condition 3 was to account for the effects of restraining the forearms in condition 2. The participants' lower-limb joint angles and spatiotemporal parameters remained unchanged across the three conditions while walking at their normal and fast self-selected gait speeds. However, significant decreases were observed in the shoulder and trunk angles, the interlimb coordination, and the shoulder-trunk correlations when walking with the artificial forearms. These observations were in tandem with the increased muscle activity of the biceps, trapeziuses, and posterior deltoids, which controlled the shoulder motion and trunk rotation during walking with the artificial forearms across both normal and fast self-selected speeds. Although not significant, the metabolic energy analysis of five participants revealed an increase during walking with artificial forearms. The results support the idea that the body actively controls the forearm movements through the shoulder and trunk rotations to mitigate the undesired disturbances induced by the passive forearm movements during locomotion.

Artificial Perception and Semiautonomous Control in Myoelectric Hand Prostheses Increases Performance and Decreases Effort

Dexterous control of upper limb prostheses with multiarticulated wrists/hands is still a challenge due to the limitations of myoelectric man-machine interfaces. Multiple factors limit the overall performance and usability of these interfaces, such as the need to control degrees of freedom sequentially and not concurrently, and the inaccuracies in decoding the user intent from weak or fatigued muscles. In this article, we developed a novel man-machine interface that endows a myoelectric prosthesis (MYO) with artificial perception, estimation of user intention, and intelligent control (MYO-PACE) to continuously support the user with automation while preparing the prosthesis for grasping. We compared the MYO-PACE against state-of-the-art myoelectric control (pattern recognition) in laboratory and clinical tests. For this purpose, eight able-bodied and two amputee individuals performed a standard clinical test consisting of a series of manipulation tasks (portion of the SHAP test), as well as a more complex sequence of transfer tasks in a cluttered scene. In all tests, the subjects not only completed the trials faster using the MYO-PACE but also achieved more efficient myoelectric control. These results demonstrate that the implementation of advanced perception, context interpretation, and autonomous decision-making into active prostheses improves control dexterity. Moreover, it also effectively supports the user by speeding up the preshaping phase of the movement and decreasing muscle use.

Human Machine Interfaces in Upper-Limb Prosthesis Control: A Survey of Techniques for Preprocessing and Processing of Biosignals

Prostheses provide a means for individuals with amputations to regain some of the lost functions of their amputated limb. Human-machine interfaces (HMIs), used for controlling prosthetic devices, play a critical role in users' experiences with prostheses. This review article provides an overview of the HMIs commonly adopted for upper-limb prosthesis control and inspects collected signals and their processing methods.

Phantom motion intent decoding for transhumeral prosthesis control with fused neuromuscular and brain wave signals

In recent years, the electroencephalography (EEG) brain–computer interface (BCI) has been researched in the area of upper-limb prosthesis control due to the promise of being able to record neurological signals which follow activation patterns in the cortex directly from the brain with non-invasive electrodes. This is seen as a way of bypassing the limitation posed by acquiring neuromuscular signals predominantly with electromyography (EMG) directly from the stump, which possesses residual limb anatomy post-amputation. In this study, the sequential forward selection algorithm to form a 10-optimal-channel representation, alongside an extended signal feature vector was applied, to investigate the motion intent decoding performance of EMG-only, EEG-only, and a fused EMG–EEG sensing configuration for four transhumeral amputees with varying stump lengths. The results showed a considerable improvement for the EMG-only configuration with the advanced feature vector, but only a small increase for the EEG-only, and thus a marginal improvement when information from both signals was fused together. This is likely due to the EEG requiring a greater number of channels spread across the skull to provide a reliable intent decoding. Further work will now involve optimisation studies to find a greater representation of electrode representation and parsimony, to minimise the number of channels while boosting motion intent decoding accuracy.

Recurrent Neural Network for electromyographic gesture recognition in transhumeral amputees

Gesture recognition is a key aspect of myoelectric control of upper-limb prostheses and is rather complex to achieve for transhumeral amputees. The prosthesis control of upper arm movements must rely only on the arm muscles, which were not involved in these gestures before the amputation. For decades, machine learning has been used in research for upper-limb gesture recognition. However, reported classification accuracies for transhumeral amputees have not improved significantly since the 1990s. Latest developments in deep learning suggest it can outperform classical machine learning both in accuracy and processing time. This study aims to determine if a deep learning approach, specifically a Recurrent Neural Network (RNN), could better recognize the movement intents in transhumeral amputees. To do so, the classification accuracy and the processing time of the RNN were measured and compared to two state-of-the-art approaches that use a linear discriminant analysis (LDA) and a multilayer perceptron (MLP) respectively. All three approaches were used to classify the signals of five transhumeral amputees between 6 upper-limb gestures. For subjects 1, 3 and 5, the classification accuracy was significantly higher (p = 0.0002) for the RNN (79.7%) compared to the LDA (67,1%) and the MLP (74,1%). Additionally, the RNN had a much smaller processing time, under 7 ms, compared to 385 ms and 377 ms for the LDA and the MLP respectively. Consequently, the RNN is better suited for a real-time prosthesis control that occurs between 100–250 ms. Results suggest deep learning as a viable solution for gesture recognition in transhumeral amputees.

EMG-Based Hand Gesture Classification with Long Short-Term Memory Deep Recurrent Neural Networks.

Electromyogram (EMG) pattern recognition has been utilized with the traditional machine and deep learning architectures as a control strategy for upper-limb prostheses. However, most of these learning architectures, including those in convolutional neural networks, focus the spatial correlations only; but muscle contractions have a strong temporal dependency. Our primary aim in this paper is to investigate the effectiveness of recurrent deep learning networks in EMG classification as they can learn long-term and non-linear dynamics of time series. We used a Long Short-Term Memory (LSTM-based) neural network to perform multiclass classification with six grip gestures at three different force levels (low, medium, and high) generated by nine amputees. Four different feature sets were extracted from the raw signals and fed to LSTM. Moreover, to investigate a generalization of the proposed method, three different training approaches were tested including 1) training the network with feature extracted from one specific force level and testing it with the same force level, 2) training the network with one specific force level and testing it with two remained force levels, and 3) training the network with all of the force levels and testing it with a single force level. Our results show that LSTM-based neural network can provide reliable performance with average classification errors of around 9% across all nine amputees and force levels. We demonstrate the applicability of deep learning for upperlimb prosthesis control.

A “Biomechatronic EPP” Upper-Limb Prosthesis Control Configuration and its Performance Comparison to Other Control Configurations

Upper-limb prosthetic technology has significantly changed in recent years. The devices available and those under development, have progressed the state of the art considerably. However, most are based on velocity control and fail to activate the proprioception of the amputee, as they do not provide their user with adequate feedback. A novel control configuration called “Biomechatronic EPP” has been developed to overcome these shortcomings, which is based on the concept of the Extended Physiological Proprioception (EPP), an upper-limb prosthesis proven to be functionally superior to velocity control configurations. The performance of the “Biomechatronic EPP” is compared to that of three other control toplogies including: a “Classic EPP” controller, an “Unconnected” controller and an “EMG” controller. Fourteen able-bodied subjects engaged in a 1-D Fitts’ Law style task, designed with the Psychophysics Toolbox, a free set of MATLAB. Performance was evaluated using several measures. Overall, the experimental results show that the performance of “Biomechatronic EPP” is comparable to “Classic EPP” and superior to “Unconnected” and “EMG”. The proposed Biomechatronic EPP control configuration is an alternative to various invasive and non-invasive sensory feedback control integration methods for prosthesis control.

Performance Evaluation of Convolutional Neural Network for Hand Gesture Recognition Using EMG.

Electromyography (EMG) is a measure of electrical activity generated by the contraction of muscles. Non-invasive surface EMG (sEMG)-based pattern recognition methods have shown the potential for upper limb prosthesis control. However, it is still insufficient for natural control. Recent advancements in deep learning have shown tremendous progress in biosignal processing. Multiple architectures have been proposed yielding high accuracies (>95%) for offline analysis, yet the delay caused due to optimization of the system remains a challenge for its real-time application. From this arises a need for optimized deep learning architecture based on fine-tuned hyper-parameters. Although the chance of achieving convergence is random, however, it is important to observe that the performance gain made is significant enough to justify extra computation. In this study, the convolutional neural network (CNN) was implemented to decode hand gestures from the sEMG data recorded from 18 subjects to investigate the effect of hyper-parameters on each hand gesture. Results showed that the learning rate set to either 0.0001 or 0.001 with 80-100 epochs significantly outperformed (p < 0.05) other considerations. In addition, it was observed that regardless of network configuration some motions (close hand, flex hand, extend the hand and fine grip) performed better (83.7% ± 13.5%, 71.2% ± 20.2%, 82.6% ± 13.9% and 74.6% ± 15%, respectively) throughout the course of study. So, a robust and stable myoelectric control can be designed on the basis of the best performing hand motions. With improved recognition and uniform gain in performance, the deep learning-based approach has the potential to be a more robust alternative to traditional machine learning algorithms.

Targeted muscle reinnervation to improve electromyography signals for advanced myoelectric prosthetic limbs: a series of seven patients.

Upper limb amputation is a devastating injury. Patients may choose to use a passive prosthesis, a traditional body-powered prosthesis or a myoelectric prosthesis driven by electromyography (EMG) signals generated by underlying muscles. Targeted muscle reinnervation (TMR) aims to surgically create strong and reliable signals to permit the intuitive use of a myoelectric prosthesis with the greatest number of movements possible. We review the Alfred Hospital experience of using TMR to improve upper limb prosthesis control. A retrospective review of all cases of TMR performed at the Alfred Hospital was undertaken. Patient demographics, injury, surgical complications and outcomes were examined. Comparison was made to preoperative prosthesis use. Seven patients have undergone TMR to improve upper limb prosthesis control at the Alfred Hospital between 2015 and 2018. Within the patient group, pre-TMR EMG signal numbers ranged from 1 to 2, and post-TMR signal numbers ranged from 3 to 5. Six patients were able to achieve six degrees of freedom post-operatively, and one patient achieved four degrees. No patients required the use of co-contraction to switch function post-operatively. There were no significant surgical complications. The use of TMR to improve and increase the number of EMG signals has been successful in generating more degrees of freedom for upper limb amputees with myoelectric prostheses.

Ultrasonography and electromyography based hand motion intention recognition for a trans-radial amputee: A case study

Surface electromyography (sEMG) has dominated upper-limb prosthesis control for decades due to its simplicity and effectiveness [1–3]. However, the inherent variability of EMG signal hinders the flexible and accurate control of advanced multi-functional prosthesis. This study is an attempt to use ultrasonography (US) as an alternative for prosthetic hand control. A type of multi-sensory module, comprising a single-element ultrasound channel and one sEMG bipolar channel, is customised to ensure a fair comparison between these two modalities. Three machine-learning-oriented approaches were adopted to evaluate the performance in motion classification based on datasets captured from a trans-radial amputee. The experimental results demonstrated that the ultrasound outperformed the sEMG in random (98.9% vs 70.4%) and enhanced-trial-wise (74.10% vs 61.83%) cross-validation, but fell behind the sEMG in trial-wise (39.47% vs 58.04%) validation that is the closest comparison to a real life prosthetic control. This study preliminarily implies that 1) A-mode ultrasound signal can be more stable than the sEMG with minimum electrode shift, but more sensitive to external interference than the sEMG; and 2) to maintain high classification accuracy, US approach may require harsher electrode fixing mechanism or advanced on-line calibration approach.

Multi-Grip Classification-Based Prosthesis Control With Two EMG-IMU Sensors.

In the field of upper-limb myoelectric prosthesis control, the use of statistical and machine learning methods has been long proposed as a means of enabling intuitive grip selection and actuation. Recently, this paradigm has found its way toward commercial adoption. Machine learning-based prosthesis control typically relies on the use of a large number of electrodes. Here, we propose an end-to-end strategy for multi-grip, classification-based prosthesis control using only two sensors, comprising electromyography (EMG) electrodes and inertial measurement units (IMUs). We emphasize the importance of accurately estimating posterior class probabilities and rejecting predictions made with low confidence, so as to minimize the rate of unintended prosthesis activations. To that end, we propose a confidence-based error rejection strategy using grip-specific thresholds. We evaluate the efficacy of the proposed system with real-time pick and place experiments using a commercial multi-articulated prosthetic hand and involving 12 able-bodied and two transradial (i.e., below-elbow) amputee participants. Results promise the potential for deploying intuitive, classification-based multi-grip control in existing upper-limb prosthetic systems subject to small modifications.

On the Visuomotor Behavior of Amputees and Able-Bodied People During Grasping.

Visual attention is often predictive for future actions in humans. In manipulation tasks, the eyes tend to fixate an object of interest even before the reach-to-grasp is initiated. Some recent studies have proposed to exploit this anticipatory gaze behavior to improve the control of dexterous upper limb prostheses. This requires a detailed understanding of visuomotor coordination to determine in which temporal window gaze may provide helpful information. In this paper, we verify and quantify the gaze and motor behavior of 14 transradial amputees who were asked to grasp and manipulate common household objects with their missing limb. For comparison, we also include data from 30 able-bodied subjects who executed the same protocol with their right arm. The dataset contains gaze, first person video, angular velocities of the head, and electromyography and accelerometry of the forearm. To analyze the large amount of video, we developed a procedure based on recent deep learning methods to automatically detect and segment all objects of interest. This allowed us to accurately determine the pixel distances between the gaze point, the target object, and the limb in each individual frame. Our analysis shows a clear coordination between the eyes and the limb in the reach-to-grasp phase, confirming that both intact and amputated subjects precede the grasp with their eyes by more than 500 ms. Furthermore, we note that the gaze behavior of amputees was remarkably similar to that of the able-bodied control group, despite their inability to physically manipulate the objects.

Can Multi-DoF Training Improve Robustness of Muscle Synergy Inspired Myocontrollers?

Non-negative Matrix Factorization (NMF) has been effective in extracting commands from surface electromyography (EMG) for the control of upper-limb prostheses. This approach enables Simultaneous and Proportional Control (SPC) over multiple degrees-of-freedom (DoFs) in a minimally supervised way. Here, like with other myoelectric approaches, robustness remains essential for clinical adoption, with device donning/doffing being a known cause for performance degradation. Previous research has demonstrated that NMF-based myocontrollers, trained on just single-DoF activations, permit a certain degree of user adaptation to a range of disturbances. In this study, we compare this traditional NMF controller with its sparsity constrained variation that allows initialization using both single and combined-DoF activations (NMF-C). The evaluation was done on 12 able bodied participants through a set of online target-reaching tests. Subjects were fitted with an 8-channel bipolar EMG setup, which was shifted by 1cm in both transversal directions throughout the experiments without system retraining. In the baseline condition NMF performed somewhat better than NMFC, but it did suffer more following the electrode repositioning, making the two perform on par. With no significant difference present across the conditions, results suggest that there is no immediate advantage from the naïve inclusion of more comprehensive training sets to the classic synergy-inspired implementation of SPC.

Counteracting Electrode Shifts in Upper-Limb Prosthesis Control via Transfer Learning.

Research on machine learning approaches for upper-limb prosthesis control has shown impressive progress. However, translating these results from the lab to patient's everyday lives remains a challenge because advanced control schemes tend to break down under everyday disturbances, such as electrode shifts. Recently, it has been suggested to apply adaptive transfer learning to counteract electrode shifts using as little newly recorded training data as possible. In this paper, we present a novel, simple version of transfer learning and provide the first user study demonstrating the effectiveness of transfer learning to counteract electrode shifts. For this purpose, we introduce the novel Box and Beans test to evaluate prosthesis proficiency and compare user performance with an initial simple pattern recognition system, the system under electrode shifts, and the system after transfer learning. Our results show that transfer learning could significantly alleviate the impact of electrode shifts on user performance in the Box and Beans test.

Priorities for the design and control of upper limb prostheses: A focus group study

BackgroundCommon prosthetic options do not allow for enough independent control signals to control all the movements of the arm. Invasive approaches to obtain prosthetic control signals are being developed to provide people with upper limb loss improved prosthetic control and feedback. Objective/HypothesisThis study explored the prosthetic qualities that are important to users and examined the factors that play into the decision to consider invasive prosthetic interfaces that allow for enhanced prosthetic control. MethodsIndividuals participated in semi-structured focus groups or in individual semi-structured interviews (N = 11). A semi-structured interview guide containing open-ended questions was used to learn about ideal prosthesis qualities and interest in prosthetic technology interfaces including targeted muscle reinnervation, peripheral nerve interface, and cortical interface. Qualitative content analysis with an inductive approach was used for transcript analysis. ResultsParticipants were most interested in improving the dexterity and durability of prosthetic options. Recovery time, anticipated risk, medical co-morbidities, and baseline functional status influenced willingness to consider invasive prosthetic interfaces. Participants were interested in learning more about all three invasive interfaces but had the most concerns about cortical interfaces. ConclusionsAttitudes toward invasive control interfaces vary. Further education on invasive control interfaces and additional conversations between prosthetic developers and people with limb loss will help to develop effective prosthetic devices that potential consumers will use.

Neuromorphic vision and tactile fusion for upper limb prosthesis control.

A major issue with upper limb prostheses is the disconnect between sensory information perceived by the user and the information perceived by the prosthesis. Advances in prosthetic technology introduced tactile information for monitoring grasping activity, but visual information, a vital component in the human sensory system, is still not fully utilized as a form of feedback to the prosthesis. For able-bodied individuals, many of the decisions for grasping or manipulating an object, such as hand orientation and aperture, are made based on visual information before contact with the object. We show that inclusion of neuromorphic visual information, combined with tactile feedback, improves the ability and efficiency of both able-bodied and amputee subjects to pick up and manipulate everyday objects. We discovered that combining both visual and tactile information in a real-time closed loop feedback strategy generally decreased the completion time of a task involving picking up and manipulating objects compared to using a single modality for feedback. While the full benefit of the combined feedback was partially obscured by experimental inaccuracies of the visual classification system, we demonstrate that this fusion of neuromorphic signals from visual and tactile sensors can provide valuable feedback to a prosthetic arm for enhancing real-time function and usability.

Characterization of the Sense of Agency over the Actions of Neural-machine Interface-operated Prostheses.

This work describes a methodological framework that can be used to explicitly and implicitly characterize the sense of agency developed over the neural-machine interface (NMI) control of sensate virtual or robotic prosthetic hands. The formation of agency is fundamental in distinguishing the actions that we perform with our limbs as being our own. By striving to incorporate advanced upper-limb prostheses into these same perceptual mechanisms, we can begin to integrate an artificial limb more closely into the user's existing cognitive framework for limb control. This has important implications in promoting user acceptance, use, and effective control of advanced upper-limb prostheses. In this protocol, participants control a virtual prosthetic hand and receive kinesthetic sensory feedback through their preexisting NMIs. A series of virtual grasping tasks are performed and perturbations are systematically introduced to the kinesthetic feedback and virtual hand movements. Two separate measures of agency are employed: established psychophysical questionnaires (to capture the explicit experience of agency) and a time interval estimate task to capture the implicit sense of agency (intentional binding). Results of this protocol (questionnaire scores and time interval estimates) can be analyzed to quantify the extent of agency formation.

Towards the reduction of the effects of muscle fatigue on myoelectric control of upper limb prostheses

This paper presents an investigation focused on the impact of muscle fatigue on a pattern recognition scheme for myoelectric control that uses three features sets and a Linear Discriminant Analysis classifier. Separability and repeatability between classes were used to evaluate the features changes while muscle fatigue was induced. Results show that while muscle fatigue is increasing over time, both separability and repeatability of the classes decrease. Finally two training schemes that use data acquired under fatigue, multiconditional training and selective classification, were evaluated using the Total Error Rate (TER). Results indicate that, when LDA classifier was trained whit no-fatigue, moderated fatigue and fatigue data, TER decreased to moderated and fatigue data, but increased to no-fatigue data. On the other hand, using three LDA classifiers to each of the condition, TER decreased to 9.26 % and 11 % in moderated fatigue and fatigue cases, while no-fatigue case was not affected.

Targeted Muscle Reinnervation Technique in Below-Knee Amputation.

Approximately 25 percent of major limb amputees will develop chronic localized symptomatic neuromas and phantom limb pain in the residual limb. A method to treat and possibly prevent these pain symptoms is targeted reinnervation. Previous studies prove that targeted reinnervation successfully treats and, in some cases, resolves peripheral neuropathy and phantom limb pain in patients who have undergone previous amputation (i.e., secondary targeted reinnervation). This article seeks to share the authors' clinical indications and surgical technique for targeted muscle reinnervation in below-knee amputation, a surgical description currently absent from our literature. Targeted reinnervation for the below-knee amputee has been performed on 22 patients at the authors' institution. Each patient has been followed on an outpatient basis for 1 year to evaluate symptoms of neuroma or phantom limb pain, patient satisfaction, and functionality. All subjects have denied neuroma pain following amputation. The majority of subjects reported phantom pain at 1 month. However, at 3 months, all patients reported resolution of this pain. Dumanian et al. first noted the improvement of symptomatic neuroma and phantom limb pain in patients undergoing targeted reinnervation to provide intuitive control of upper limb prostheses. These findings have been substantiated by multiple previous studies at various amputation levels. This study extends the success of targeted muscle reinnervation to below-knee amputations and provides a description for this technique.