Multifunctional Prosthetic Hand Research Articles

Using eye tracking to assess learning of a multifunction prosthetic hand: an exploratory study from a rehabilitation perspective

BackgroundEye tracking technology not only reveals the acquisition of visual information at fixation but also has the potential to unveil underlying cognitive processes involved in learning to use a multifunction prosthetic hand. It also reveals gaze behaviours observed during standardized tasks and self-chosen tasks. The aim of the study was to explore the use of eye tracking to track learning progress of multifunction hands at two different time points in prosthetic rehabilitation.MethodsThree amputees received control training of a multifunction hand with new control strategy. Detailed description of control training was collected first. They wore Tobii Pro2 eye-tracking glasses and performed a set of standardized tasks (required to switch to different grips for each task) after one day of training and at one-year-follow-up (missing data for Subject 3 at the follow up due to socket problem). They also performed a self-chosen task (free to use any grip for any object) and were instructed to perform the task in a way how they would normally do at home. The gaze-overlaid videos were analysed using the Tobii Pro Lab and the following metrics were extracted: fixation duration, saccade amplitude, eye-hand latency, fixation count and time to first fixation.ResultsDuring control training, the subjects learned 3 to 4 grips. Some grips were easier, and others were more difficult because they forgot or were confused with the switching strategies. At the one-year-follow-up, a decrease in performance time, fixation duration, eye-hand latency, and fixation count was observed in Subject 1 and 2, indicating an improvement in the ability to control the multifunction hand and a reduction of cognitive load. An increase in saccade amplitude was observed in both subjects, suggesting a decrease in difficulty to control the prosthetic hand. During the standardized tasks, the first fixation of all three subjects were on the multifunction hand in all objects. During the self-chosen tasks, the first fixations were mostly on the objects first.ConclusionThe qualitative data from control training and the quantitative eye tracking data from clinical standardized tasks provided a rich exploration of cognitive processing in learning to control a multifunction hand. Many prosthesis users prefer multifunction hands and with this study we have demonstrated that a targeted prosthetic training protocol with reliable assessment methods will help to lay the foundation for measuring functional benefits of multifunction hands.

Simultaneous hand/wrist motion recognition and continuous grasp force estimation based on nonlinear spectral sEMG features for transradial amputees

Simultaneously and accurately estimating both grasp motion classes and force based on surface electromyogram (EMG) of residual arms is essential for precise grasping with a dexterous prosthetic hand. However, accurate grasping-mode prediction and grasping-force estimation often contradict each other. When performing hand grasping motion, different grasping forces will generate different EMG patterns, which would decay the grasp motion prediction accuracy. In this study, we proposed a framework of fusing motion classification and continuous grasp force estimation based on a new set of nonlinear Mel-Frequency Spectrum features of EMG signals. The performance of the proposed method was tested on eight able bodied (AB) subjects and four transradial amputees (TR) who are the end-users of dexterous prostheses. The experimental results showed that the newly proposed features obtained a significant reduction in the average classification error rate compared to other well-known feature sets, achieving improvements of about 5 % to 28 % in the average classification accuracy across all subjects and force levels. Simultaneously, the proposed features obtained the best continuous grasp force estimation accuracies with R2 (coefficient of determination) of 0.96 and 0.83, and root mean square of maximum voluntary contractions of grasp force (std%GFMVC) of 6.2 % and 10.5 % for the AB and TR subjects, respectively, compared to the previous EMG features. This study suggested that the proposed method would be helpful for the stable and fine manipulation of multifunctional prosthetic hand.

Soft Robots for Cluttered Environments Based on Origami Anisotropic Stiffness Structure (OASS) Inspired by Desert Iguana

Rigidity and softness are essential for robot motion and manipulation in various complex scenarios. To integrate these two contrary features, several variable‐stiffness structures are investigated while requiring a longer processing time and more energy for switching from a stiffer to softer status, with only one status at a specific point in time. Inspired by the combined softness–rigidity hybrid property of desert iguanas’ skin, the concept of an anisotropic stiffness structure is developed, which simultaneously possesses rigidity and softness properties in different directions, leading to the development of novel soft robots. This anisotropic stiffness structure comprises a silicone–paper composite with multiple superimposed origami patterns and has high stiffness and softness properties. The anisotropic stiffness structure is then constructed by developing three novel soft robots: a crawling robot capable of extending and contracting with a payload of 117 times its own weight, a multifunctional prosthetic hand capable of grasping fragile items with its soft side and lifting and crushing items with its hard side, and a snake robot capable of traveling with the soft side and extruding out with over 75% of its body length with the stiffer side.

Myoelectric prosthesis hand grasp control following targeted muscle reinnervation in individuals with transradial amputation.

Despite the growing availability of multifunctional prosthetic hands, users' control and overall functional abilities with these hands remain limited. The combination of pattern recognition control and targeted muscle reinnervation (TMR) surgery, an innovative technique where amputated nerves are transferred to reinnervate new muscle targets in the residual limb, has been used to improve prosthesis control of individuals with more proximal upper limb amputations (i.e., shoulder disarticulation and transhumeral amputation). The goal of this study was to determine if prosthesis hand grasp control improves following transradial TMR surgery. Eight participants were trained to use a multi-articulating hand prosthesis under myoelectric pattern recognition control. All participated in home usage trials pre- and post-TMR surgery. Upper limb outcome measures were collected following each home trial. Three outcome measures (Southampton Hand Assessment Procedure, Jebsen-Taylor Hand Function Test, and Box and Blocks Test) improved 9-12 months post-TMR surgery compared with pre-surgery measures. The Assessment of Capacity for Myoelectric Control and Activities Measure for Upper Limb Amputees outcome measures had no difference pre- and post-surgery. An offline electromyography analysis showed a decrease in grip classification error post-TMR surgery compared to pre-TMR surgery. Additionally, a majority of subjects noted qualitative improvements in their residual limb and phantom limb sensations post-TMR. The potential for TMR surgery to result in more repeatable muscle contractions, possibly due to the reduction in pain levels and/or changes to phantom limb sensations, may increase functional use of many of the clinically available dexterous prosthetic hands.

Can Deep Synthesis of EMG Overcome the Geometric Growth of Training Data Required to Recognize Multiarticulate Motions?

By being predicated on supervised machine learning, pattern recognition approaches to myoelectric prosthesis control require electromyography (EMG) training data collected concurrently with every detectable motion. Within this framework, calibration protocols for simultaneous control of multifunctional prosthetic hands rapidly become prohibitively long-the number of unique motions grows geometrically with the number of controllable degrees of freedom (DoFs). This paper proposes a technique intended to circumvent this combinatorial explosion. Using EMG windows from 1-DoF motions as input and EMG windows from 2-DoF motions as targets, we train generative deep learning models to synthesize EMG windows appertaining to multi-DoF motions. Once trained, such models can be used to complete datasets consisting of only 1-DoF motions, enabling simple calibration protocols with durations that scale linearly with the number of DoFs. We evaluated synthetic EMG produced in this way via a classification task using a database of forearm surface EMG collected during 1-DoF and 2-DoF motions. Multi-output classifiers were trained on either (I) real data from 1-DoF and 2-DoF motions, (II) real data from only 1-DoF motions, or (III) real data from 1-DoF motions appended with synthetic EMG from 2-DoF motions. When tested on data containing all possible motions, classifiers trained on synthetic-appended data (III) significantly outperformed classifiers trained on 1-DoF real data (II), although significantly underperformed classifiers trained on both 1- and 2-DoF real data (I) (I < 0.05). These findings suggest that it is feasible to model EMG concurrent with multiarticulate motions as nonlinear combinations of EMG from constituent 1-DoF motions, and that such modelling can be harnessed to synthesize realistic training data.

A Gesture Recognition System Based on Time Domain Features and Linear Discriminant Analysis

Surface electromyogram (sEMG) signals have been used to control multifunctional prosthetic hands. Researchers usually focused on the use of several channels with sEMG signals to identify more gestures without limiting the number of sEMG sensors. However, the residual muscles of an amputee are limited. Therefore, the point of a successful recognition system is to decrease the channels of sEMG signals to classify more gestures. To achieve this goal, we proposed a novel gesture recognition system, in which three channels of sEMG signals can classify nine gestures. In this recognition system, the time domain features, root mean square ratio, and autoregressive model, were selected to extract the features of the sEMG signals as compared with the time–frequency domain features. Furthermore, the linear discriminant analysis was adopted as the classifier. Consequently, the average accuracy rate of the presented system was 91.7%. Therefore, the proposed gesture recognition system is feasible to identify more gestures with less sensors.

Single-channel surface electromyography (sEMG) based control of a multi-functional prosthetic hand

Patients suffering from upper limb amputation experience a significant reduction in their ability to perform various tasks. The currently available myoelectric prostheses seek to reinstate the lost abilities of the amputees. However, most users abandon their devices due to high cost, large size, heavy weight, complexity, and limited functional control. This paper presents a multifunctional prosthetic hand that can perform six hand activities deploying a single-channel surface electromyography (sEMG) sensor. EMG signals for fifteen subjects (five amputees and ten intact) were acquired for six contraction levels of forearm muscles using the designed sensor. These levels were further classified to recognize six predefined hand gestures by a fuzzy logic classifier. The proposed system showed an excellent success (> 95%) and other performance parameters above 96%. The intended classification-based control scheme was further realized in real-time to achieve six grip patterns for the developed prosthetic prototype. The hand was tested on five subjects (two amputees and three intact) showed a percentage of success above 91% for accomplishing the dexterous grasping operations. The proposed approach is modest, efficient, and provides a low-cost solution to amputees with multiple degrees of freedom with a single channel EMG system.

Design of a High-Speed Prosthetic Finger Driven by Peano-HASEL Actuators.

Current designs of powered prosthetic limbs are limited by the nearly exclusive use of DC motor technology. Soft actuators promise new design freedom to create prosthetic limbs which more closely mimic intact neuromuscular systems and improve the capabilities of prosthetic users. This work evaluates the performance of a hydraulically amplified self-healing electrostatic (HASEL) soft actuator for use in a prosthetic hand. We compare a linearly-contracting HASEL actuator, termed a Peano-HASEL, to an existing actuator (DC motor) when driving a prosthetic finger like those utilized in multi-functional prosthetic hands. A kinematic model of the prosthetic finger is developed and validated, and is used to customize a prosthetic finger that is tuned to complement the force-strain characteristics of the Peano-HASEL actuators. An analytical model is used to inform the design of an improved Peano-HASEL actuator with the goal of increasing the fingertip pinch force of the prosthetic finger. When compared to a weight-matched DC motor actuator, the Peano-HASEL and custom finger is 10.6 times faster, has 11.1 times higher bandwidth, and consumes 8.7 times less electrical energy to grasp. It reaches 91% of the maximum range of motion of the original finger. However, the DC motor actuator produces 10 times the fingertip force at a relevant grip position. In this body of work, we present ways to further increase the force output of the Peano-HASEL driven prosthetic finger system, and discuss the significance of the unique properties of Peano-HASELs when applied to the field of upper-limb prosthetic design. This approach toward clinically-relevant actuator performance paired with a substantially different form-factor compared to DC motors presents new opportunities to advance the field of prosthetic limb design.

Attribute-Driven Granular Model for EMG-Based Pinch and Fingertip Force Grand Recognition.

Fine multifunctional prosthetic hand manipulation requires precise control on the pinch-type and the corresponding force, and it is a challenge to decode both aspects from myoelectric signals. This paper proposes an attribute-driven granular model (AGrM) under a machine-learning scheme to solve this problem. The model utilizes the additionally captured attribute as the latent variable for a supervised granulation procedure. It was fulfilled for EMG-based pinch-type classification and the fingertip force grand prediction. In the experiments, 16 channels of surface electromyographic signals (i.e., main attribute) and continuous fingertip force (i.e., subattribute) were simultaneously collected while subjects performing eight types of hand pinches. The use of AGrM improved the pinch-type recognition accuracy to around 97.2% by 1.8% when constructing eight granules for each grasping type and received more than 90% force grand prediction accuracy at any granular level greater than six. Further, sensitivity analysis verified its robustness with respect to different channel combination and interferences. In comparison with other clustering-based granulation methods, AGrM achieved comparable pinch recognition accuracy but was of lowest computational cost and highest force grand prediction accuracy.

Simultaneous sEMG Classification of Hand/Wrist Gestures and Forces.

Surface electromyography (sEMG) signals represent a promising approach for decoding the motor intention of amputees to control a multifunctional prosthetic hand in a non-invasive way. Several approaches based on proportional amplitude methods or simple thresholds on sEMG signals have been proposed to control a single degree of freedom at time, without the possibility of increasing the number of controllable multiple DoFs in a natural manner. Myoelectric control based on PR techniques have been introduced to add multiple DoFs by keeping low the number of electrodes and allowing the discrimination of different muscular patterns for each class of motion. However, the use of PR algorithms to simultaneously decode both gestures and forces has never been studied deeply. This paper introduces a hierarchical classification approach with the aim to assess the desired hand/wrist gestures, as well as the desired force levels to exert during grasping tasks. A Finite State Machine was introduced to manage and coordinate three classifiers based on the Non-Linear Logistic Regression algorithm. The classification architecture was evaluated across 31 healthy subjects. The “hand/wrist gestures classifier,” introduced for the discrimination of seven hand/wrist gestures, presented a mean classification accuracy of 98.78%, while the “Spherical and Tip force classifier,” created for the identification of three force levels, reached an average accuracy of 98.80 and 96.09%, respectively. These results were confirmed by Linear Discriminant Analysis (LDA) with time domain features extraction, considered as ground truth for the final validation of the performed analysis. A Wilcoxon Signed-Rank test was carried out for the statistical analysis of comparison between NLR and LDA and statistical significance was considered at p < 0.05. The comparative analysis reports not statistically significant differences in terms of F1Score performance between NLR and LDA. Thus, this study reveals that the use of non-linear classification algorithm, as NLR, is as much suitable as the benchmark LDA classifier for implementing an EMG pattern recognition system, able both to decode hand/wrist gestures and to associate different performed force levels to grasping actions.

Real-time classification of shoulder girdle motions for multifunctional prosthetic hand control: A preliminary study.

In every country in the world, there are a number of amputees who have been exposed to some accidents that led to the loss of their upper limbs. The aim of this study is to suggest a system for real-time classification of five classes of shoulder girdle motions for high-level upper limb amputees using a pattern recognition system. In the suggested system, the wavelet transform was utilized for feature extraction, and the extreme learning machine was used as a classifier. The system was tested on four intact-limbed subjects and one amputee, with eight channels involving five electromyography channels and three-axis accelerometer sensor. The study shows that the suggested pattern recognition system has the ability to classify the shoulder girdle motions for high-level upper limb motions with 88.4% average classification accuracy for four intact-limbed subjects and 92.8% classification accuracy for one amputee by combining electromyography and accelerometer channels. The outcomes of this study may suggest that the proposed pattern recognition system can help to provide control signals to drive a prosthetic arm for high-level upper limb amputees.

An Alternative Myoelectric Pattern Recognition Approach for the Control of Hand Prostheses: A Case Study of Use in Daily Life by a Dysmelia Subject

The functionality of upper limb prostheses can be improved by intuitive control strategies that use bioelectric signals measured at the stump level. One such strategy is the decoding of motor volition via myoelectric pattern recognition (MPR), which has shown promising results in controlled environments and more recently in clinical practice. Moreover, not much has been reported about daily life implementation and real-time accuracy of these decoding algorithms. This paper introduces an alternative approach in which MPR allows intuitive control of four different grips and open/close in a multifunctional prosthetic hand. We conducted a clinical proof-of-concept in activities of daily life by constructing a self-contained, MPR-controlled, transradial prosthetic system provided with a novel user interface meant to log errors during real-time operation. The system was used for five days by a unilateral dysmelia subject whose hand had never developed, and who nevertheless learned to generate patterns of myoelectric activity, reported as intuitive, for multi-functional prosthetic control. The subject was instructed to manually log errors when they occurred via the user interface mounted on the prosthesis. This allowed the collection of information about prosthesis usage and real-time classification accuracy. The assessment of capacity for myoelectric control test was used to compare the proposed approach to the conventional prosthetic control approach, direct control. Regarding the MPR approach, the subject reported a more intuitive control when selecting the different grips, but also a higher uncertainty during proportional continuous movements. This paper represents an alternative to the conventional use of MPR, and this alternative may be particularly suitable for a certain type of amputee patients. Moreover, it represents a further validation of MPR with dysmelia cases.

Vector Autoregressive Hierarchical Hidden Markov Models for Extracting Finger Movements Using Multichannel Surface EMG Signals

We present a novel computational technique intended for the robust and adaptable control of a multifunctional prosthetic hand using multichannel surface electromyography. The initial processing of the input data was oriented towards extracting relevant time domain features of the EMG signal. Following the feature calculation, a piecewise modeling of the multidimensional EMG feature dynamics using vector autoregressive models was performed. The next step included the implementation of hierarchical hidden semi‐Markov models to capture transitions between piecewise segments of movements and between different movements. Lastly, inversion of the model using an approximate Bayesian inference scheme served as the classifier. The effectiveness of the novel algorithms was assessed versus methods commonly used for real‐time classification of EMGs in a prosthesis control application. The obtained results show that using hidden semi‐Markov models as the top layer, instead of the hidden Markov models, ranks top in all the relevant metrics among the tested combinations. The choice of the presented methodology for the control of prosthetic hand is also supported by the equal or lower computational complexity required, compared to other algorithms, which enables the implementation on low‐power microcontrollers, and the ability to adapt to user preferences of executing individual movements during activities of daily living.

Decoding of individual finger movements from surface EMG signals using vector autoregressive hierarchical hidden Markov models (VARHHMM).

In this paper we present a novel method for predicting individual fingers movements from surface electromyography (EMG). The method is intended for real-time dexterous control of a multifunctional prosthetic hand device. The EMG data was recorded using 16 single-ended channels positioned on the forearm of healthy participants. Synchronously with the EMG recording, the subjects performed consecutive finger movements based on the visual cues. Our algorithm could be described in following steps: extracting mean average value (MAV) of the EMG to be used as the feature for classification, piece-wise linear modeling of EMG feature dynamics, implementation of hierarchical hidden Markov models (HHMM) to capture transitions between linear models, and implementation of Bayesian inference as the classifier. The performance of our classifier was evaluated against commonly used real-time classifiers. The results show that the current algorithm setup classifies EMG data similarly to the best among tested classifiers but with equal or less computational complexity.

Mechanomyography Assisted Myoeletric Sensing for Upper-Extremity Prostheses: A Hybrid Approach

The myoelectric upper-limb prosthetic manipulation is inherently limited by the unreliable sensor-skin interface. This paper presents a hybrid approach to overcome the limitation of electromyography (EMG) through mechanomyography (MMG) assisted myoelectric sensing. An integrated hybrid sensor system was developed for simultaneous EMG and MMG measurement. The hybrid system formed a platform to capture muscular activations in different frequencies. To evaluate the effectiveness of hybrid EMG-MMG sensing, hand motion experiments have been carried out on seven able-bodied and two transradial amputee subjects. It convincingly demonstrated, a significantly ( ${p} ) improved classification accuracy (CA). Furthermore, the CA was compensated by 8.7% ~ 33.7% in the presence of 2 ~ 3 fault EMG channels. These results suggest that MMG assisted myoelectric sensing can improve the control performance and robustness. It has great potential to promote the clinical application of multi-functional prosthetic hand with hybrid EMG-MMG sensor system.

Deep learning-based artificial vision for grasp classification in myoelectric hands

Objective. Computer vision-based assistive technology solutions can revolutionise the quality of care for people with sensorimotor disorders. The goal of this work was to enable trans-radial amputees to use a simple, yet efficient, computer vision system to grasp and move common household objects with a two-channel myoelectric prosthetic hand. Approach. We developed a deep learning-based artificial vision system to augment the grasp functionality of a commercial prosthesis. Our main conceptual novelty is that we classify objects with regards to the grasp pattern without explicitly identifying them or measuring their dimensions. A convolutional neural network (CNN) structure was trained with images of over 500 graspable objects. For each object, 72 images, at intervals, were available. Objects were categorised into four grasp classes, namely: pinch, tripod, palmar wrist neutral and palmar wrist pronated. The CNN setting was first tuned and tested offline and then in realtime with objects or object views that were not included in the training set. Main results. The classification accuracy in the offline tests reached for the seen and for the novel objects; reflecting the generalisability of grasp classification. We then implemented the proposed framework in realtime on a standard laptop computer and achieved an overall score of in classifying a set of novel as well as seen but randomly-rotated objects. Finally, the system was tested with two trans-radial amputee volunteers controlling an i-limb UltraTM prosthetic hand and a motion controlTM prosthetic wrist; augmented with a webcam. After training, subjects successfully picked up and moved the target objects with an overall success of up to . In addition, we show that with training, subjects’ performance improved in terms of time required to accomplish a block of 24 trials despite a decreasing level of visual feedback. Significance. The proposed design constitutes a substantial conceptual improvement for the control of multi-functional prosthetic hands. We show for the first time that deep-learning based computer vision systems can enhance the grip functionality of myoelectric hands considerably.

Development of a mathematical model of a biomechatronic finger prosthesis as part of a multifunctional prosthetic hand

A mathematical model of a biomechanical finger prosthesis is proposed, which makes it possible to study its dynamics under the condition of an increased degree of mobility of the structure under consideration. Increased degree of mobility avoids the disadvantages of creating a multifunctional biomechanical hand prosthetic.

Control of Multifunctional Prosthetic Hands by Processing the Electromyographic Signal.

The human hand is a complex system, with a large number of degrees of freedom (DoFs), sensors embedded in its structure, actuators and tendons, and a complex hierarchical control. Despite this complexity, the efforts required to the user to carry out the different movements is quite small (albeit after an appropriate and lengthy training). On the contrary, prosthetic hands are just a pale replication of the natural hand, with significantly reduced grasping capabilities and no sensory information delivered back to the user. Several attempts have been carried out to develop multifunctional prosthetic devices controlled by electromyographic (EMG) signals (myoelectric hands), harness (kinematic hands), dimensional changes in residual muscles, and so forth, but none of these methods permits the "natural" control of more than two DoFs. This article presents a review of the traditional methods used to control artificial hands by means of EMG signal, in both the clinical and research contexts, and introduces what could be the future developments in the control strategy of these devices.

Novel postural control algorithm for control of multifunctional myoelectric prosthetic hands.

The myoelectric controller (MEC) remains a technological bottleneck in the development of multifunctional prosthetic hands. Current MECs require physiologically inappropriate commands to indicate intent and lack effectiveness in a clinical setting. Postural control schemes use surface electromyography signals to drive a cursor in a continuous two-dimensional domain that is then transformed into a hand posture. Here, we present a novel algorithm for a postural controller and test the efficacy of the system during two experiments with 11 total subjects. In the first experiment, we found that performance increased when a velocity cursor-control technique versus a position cursor-control technique was used. Also, performance did not change when using 3, 4, or 12 surface electrodes. In the second experiment, subjects commanded a six degree-of-freedom virtual hand into seven functional postures without training, with completion rates of 82 +/- 4%, movement times of 3.5 +/- 0.2 s, and path efficiencies of 45 +/- 3%. Subjects retained the ability to use the postural controller at a high level across days after a single 1 hr training session. Our results substantiate the novel algorithm for a postural controller as a robust and advantageous design for a MEC of multifunction prosthetic hands.

Development and Control of a Multifunctional Prosthetic Hand with Shape Memory Alloy Actuators

In this research paper, non-conventional actuation technology, based on shape memory alloys, is employed for the development of an innovative low-cost five-fingered prosthetic hand. By exploiting t...