Control Of Multifunctional Prostheses Research Articles

Multi-modality deep forest for hand motion recognition via fusing sEMG and acceleration signals.

Bio-signal based hand motion recognition plays a critical role in the tasks of human-machine interaction, such as the natural control of multifunctional prostheses. Although a large number of classification technologies have been taken to improve the motion recognition accuracy, it is still a challenge to achieve acceptable performance for multiple modality input. This study proposes a multi-modality deep forest (MMDF) framework to identify hand motions, in which surface electromyographic signals (sEMG) and acceleration signals (ACC) are fused at the input level. The proposed MMDF framework constitutes of three main stages, sEMG and ACC feature extraction, feature dimension reduction, and a cascade structure deep forest for classification. A public database "Ninapro DB7" is used to evaluate the performance of the proposed framework, and the experimental results show that it can achieve a significantly higher accuracy than that of competitors. Besides, our experimental results also show that MMDF outperforms other traditional classifiers with the input of the single modality of sEMG signals. In sum, this study verifies that ACC signals can be an excellent supplementary for sEMG, and MMDF is a plausible solution to fuse mulit-modality bio-signals for human motion recognition.

A Novel Motion Recognition Method Based on Force Myography of Dynamic Muscle Contractions.

Surface electromyogram-based pattern recognition (sEMG-PR) has been considered as the most promising method to control multifunctional prostheses for decades. However, the commercial applications of sEMG-PR in prosthetic control is still limited due to the ambient noise and impedance variation between electrodes and skin surface. In order to reduce these issues, a force-myography-based pattern recognition method was proposed. In this method, a type of polymer-based flexible film sensors, the piezoelectrets, were used to record the rate of stress change (RSC) signals on the muscle surface of eight able-bodied subjects for six hand motions. Thirteen time domain features and four classification algorithms of linear discriminant analysis (LDA), K-nearest neighbor (KNN), artificial neural network (ANN), and support vector machine (SVM) were adopted to decode the RSC signals of different motion classes. In addition, the optimal feature set, classifier, and analysis window length were investigated systematically. Results showed that the average classification accuracy was 95.5 ± 2.2% by using the feature combination of root mean square (RMS) and waveform length (WL) for the classifier of KNN, and the analysis window length of 300 ms was found to obtain the best classification performance. Moreover, the robustness of the proposed method was investigated, and the classification accuracies were observed above 90% even when the white noise ratio increased to 50%. The work of this study demonstrated the effectiveness of RSC-based pattern recognition method for motion classification, and it would provide an alternative approach for the control of multifunctional prostheses.

Multifunctional Prosthesis Control with Simulation of Myoelectric Signals

The skeletal muscle activation generates electric signals called myoelectric signals. In recent years a strong scientific activity has been developed in the recognition of limb movements from electromyography (EMG) signals recorded from non-invasive (surface) electrodes, in order to design systems for prosthetic control. Surface EMG acquire the activation of surrounding muscles and for that reason the obtained signal needs to be conditioned and processed, with pattern recognition techniques for extraction and classification. In this work EMG signals were acquired for two hand movements, “hand close” and “hand open”. The EMG electrodes were placed on the forearm and positioned over the extensor digitorum muscle, for the “hand open” and flexor digitorum muscle, for the “hand close”. Using MATLAB software the signal conditioning, feature extraction and classification were performed. The feature extraction process was carried with the Wavelet Packet Transform (WPT) technique and the classification process was done with two different techniques for comparison purposes, Neural Networks (NN) and Support Vector Machines (SVM). The results show that the SVM classifier used presented better classification performance compared to NN classifier used.
 Keywords: EMG, Signal conditioning, Wavelet Packet Transform (WPT), Neural Networks (NN), Support Vector Machines (SVM)

Ultrasonography and electromyography based hand motion intention recognition for a trans-radial amputee: A case study

Surface electromyography (sEMG) has dominated upper-limb prosthesis control for decades due to its simplicity and effectiveness [1–3]. However, the inherent variability of EMG signal hinders the flexible and accurate control of advanced multi-functional prosthesis. This study is an attempt to use ultrasonography (US) as an alternative for prosthetic hand control. A type of multi-sensory module, comprising a single-element ultrasound channel and one sEMG bipolar channel, is customised to ensure a fair comparison between these two modalities. Three machine-learning-oriented approaches were adopted to evaluate the performance in motion classification based on datasets captured from a trans-radial amputee. The experimental results demonstrated that the ultrasound outperformed the sEMG in random (98.9% vs 70.4%) and enhanced-trial-wise (74.10% vs 61.83%) cross-validation, but fell behind the sEMG in trial-wise (39.47% vs 58.04%) validation that is the closest comparison to a real life prosthetic control. This study preliminarily implies that 1) A-mode ultrasound signal can be more stable than the sEMG with minimum electrode shift, but more sensitive to external interference than the sEMG; and 2) to maintain high classification accuracy, US approach may require harsher electrode fixing mechanism or advanced on-line calibration approach.

A novel feature extraction method for machine learning based on surface electromyography from healthy brain

Feature extraction is one of most important steps in the control of multifunctional prosthesis based on surface electromyography (sEMG) pattern recognition. In this paper, a new sEMG feature extraction method based on muscle active region is proposed. This paper designs an experiment to classify four hand motions using different features. This experiment is used to prove that new features have better classification performance. The experimental results show that the new feature, active muscle regions (AMR), has better classification performance than other traditional features, mean absolute value (MAV), waveform length (WL), zero crossing (ZC) and slope sign changes (SSC). The average classification errors of AMR, MAV, WL, ZC and SSC are 13%, 19%, 26%, 24% and 22%, respectively. The new EMG features are based on the mapping relationship between hand movements and forearm active muscle regions. This mapping relationship has been confirmed in medicine. We obtain the active muscle regions data from the original EMG signal by the new feature extraction algorithm. The results obtained from this algorithm can well represent hand motions. On the other hand, the new feature vector size is much smaller than other features. The new feature can narrow the computational cost. This proves that the AMR can improve sEMG pattern recognition accuracy rate.

Robust simultaneous myoelectric control of multiple degrees of freedom in wrist-hand prostheses by real-time neuromusculoskeletal modeling

Objective. Robotic prosthetic limbs promise to replace mechanical function of lost biological extremities and restore amputees’ capacity of moving and interacting with the environment. Despite recent advances in biocompatible electrodes, surgical procedures, and mechatronics, the impact of current solutions is hampered by the lack of intuitive and robust man-machine interfaces. Approach. This work presents a biomimetic interface that synthetizes the musculoskeletal function of an individual’s phantom limb as controlled by neural surrogates, i.e. electromyography-derived neural activations. With respect to current approaches based on machine learning, our method employs explicit representations of the musculoskeletal system to reduce the space of feasible solutions in the translation of electromyograms into prosthesis control commands. Electromyograms are mapped onto mechanical forces that belong to a subspace contained within the broader operational space of an individual’s musculoskeletal system. Main results. Our results show that this constraint makes the approach applicable to real-world scenarios and robust to movement artefacts. This stems from the fact that any control command must always exist within the musculoskeletal model operational space and be therefore physiologically plausible. The approach was effective both on intact-limbed individuals and a transradial amputee displaying robust online control of multi-functional prostheses across a large repertoire of challenging tasks. Significance. The development and translation of man-machine interfaces that account for an individual’s neuromusculoskeletal system creates unprecedented opportunities to understand how disrupted neuro-mechanical processes can be restored or replaced via biomimetic wearable assistive technologies.

Two ways to improve myoelectric control for a transhumeral amputee after targeted muscle reinnervation: a case study

BackgroundMyoelectric control of multifunctional prostheses is challenging for individuals with high-level amputations due to insufficient surface electromyography (sEMG) signals. A surgical technique called targeted muscle reinnervation (TMR) has achieved impressive improvements in myoelectric control by providing more sEMG control signals. In this case, some channels of sEMG signals are coupled after TMR, which limits the performance of conventional amplitude-based control for upper-limb prostheses.MethodsIn this paper, two different ways (training and algorithms) were attempted to solve the problem in a transhumeral amputee after TMR. Firstly, effect of rehabilitation training on generating independent sEMG signals was investigated. The results indicated that some sEMG signals recorded were still coupled over the targeted muscles after rehabilitation training for about two months. Secondly, pattern recognition (PR) algorithm was then applied to classify the sEMG signals. In the second way, to further improve the real-time performance of prosthetic control, a post-processing method named as mean absolute value-based (MAV-based) threshold switches was utilized.ResultsUsing the improved algorithms, substantial improvement was shown in a subset of the modified Action Research Arm Test (ARAT). Compared with common PR control without post-processing method, the total scores increased more than 18% with majority vote and more than 58% with MAV-based threshold switches. The amputee was able to finish all the tasks within the allotted time with the standard MAV-based threshold switches. Subjectively the amputee preferred the PR control with MAV-based threshold switches and reported it to be more accurate and much smoother both in experiment and practical use.ConclusionsAlthough the sEMG signals were still coupled after rehabilitation training on the TMR patient, the online performance of the prosthetic operation was improved through application of PR control with combination of the MAV-based threshold switches.Trial registrationRetrospectively registered http://www.chictr.org.cn/showproj.aspx?proj=22058.

The clinical relevance of advanced artificial feedback in the control of a multi-functional myoelectric prosthesis

BackgroundTo effectively replace the human hand, a prosthesis should seamlessly respond to user intentions but also convey sensory information back to the user. Restoration of sensory feedback is rated highly by the prosthesis users, and feedback is critical for grasping in able-bodied subjects. Nonetheless, the benefits of feedback in prosthetics are still debated. The lack of consensus is likely due to the complex nature of sensory feedback during prosthesis control, so that its effectiveness depends on multiple factors (e.g., task complexity, user learning).MethodsWe evaluated the impact of these factors with a longitudinal assessment in six amputee subjects, using a clinical setup (socket, embedded control) and a range of tasks (box and blocks, block turn, clothespin and cups relocation). To provide feedback, we have proposed a novel vibrotactile stimulation scheme capable of transmitting multiple variables from a multifunction prosthesis. The subjects wore a bracelet with four by two uniformly placed vibro-tactors providing information on contact, prosthesis state (active function), and grasping force. The subjects also completed a questionnaire for the subjective evaluation of the feedback.ResultsThe tests demonstrated that feedback was beneficial only in the complex tasks (block turn, clothespin and cups relocation), and that the training had an important, task-dependent impact. In the clothespin relocation and block turn tasks, training allowed the subjects to establish successful feedforward control, and therefore, the feedback became redundant. In the cups relocation task, however, the subjects needed some training to learn how to properly exploit the feedback. The subjective evaluation of the feedback was consistently positive, regardless of the objective benefits. These results underline the multifaceted nature of closed-loop prosthesis control as, depending on the context, the same feedback interface can have different impact on performance. Finally, even if the closed-loop control does not improve the performance, it could be beneficial as it seems to improve the subjective experience.ConclusionsTherefore, in this study we demonstrate, for the first time, the relevance of an advanced, multi-variable feedback interface for dexterous, multi-functional prosthesis control in a clinically relevant setting.

Improving the Robustness of Real-Time Myoelectric Pattern Recognition against Arm Position Changes in Transradial Amputees.

Previous studies have showed that arm position variations would significantly degrade the classification performance of myoelectric pattern-recognition-based prosthetic control, and the cascade classifier (CC) and multiposition classifier (MPC) have been proposed to minimize such degradation in offline scenarios. However, it remains unknown whether these proposed approaches could also perform well in the clinical use of a multifunctional prosthesis control. In this study, the online effect of arm position variation on motion identification was evaluated by using a motion-test environment (MTE) developed to mimic the real-time control of myoelectric prostheses. The performance of different classifier configurations in reducing the impact of arm position variation was investigated using four real-time metrics based on dataset obtained from transradial amputees. The results of this study showed that, compared to the commonly used motion classification method, the CC and MPC configurations improved the real-time performance across seven classes of movements in five different arm positions (8.7% and 12.7% increments of motion completion rate, resp.). The results also indicated that high offline classification accuracy might not ensure good real-time performance under variable arm positions, which necessitated the investigation of the real-time control performance to gain proper insight on the clinical implementation of EMG-pattern-recognition-based controllers for limb amputees.

Exoprosthetic Replacement of the Upper Extremity

During the last years, the prosthetic replacement in upper limb amputees has undergone different developments. The use of new nerve surgical concepts improved the control strategies tremendously, especially for high-level amputees. Technological innovation in the field of pattern recognition enables the control of multifunctional myoelectric hand prostheses in a natural and intuitive manner. However, the different levels of amputation pose different challenges for the therapeutic team which concern not only the prosthetic attachment; also the expected functional outcome of prosthetic limb replacement differs greatly between the individual levels of amputation. Therefore, especially in partial hand amputations the indication for prosthetic fitting has to be evaluated critically, as these patients may benefit more from biologic reconstructive concepts. The value of the upper extremity, in particular of the hand, is undisputable and, as such represents the driving force for the technological and surgical developments within the exoprosthetic replacement. This article discusses the possibilities and limitations of exoprosthetic limb replacement on the different amputation levels and explores new developments.

Towards reducing the impacts of unwanted movements on identification of motion intentions

Surface electromyogram (sEMG) has been extensively used as a control signal in prosthesis devices. However, it is still a great challenge to make multifunctional myoelectric prostheses clinically available due to a number of critical issues associated with existing EMG based control strategy. One such issue would be the effect of unwanted movements (UMs) that are inadvertently done by users on the performance of movement classification in EMG pattern recognition based algorithms. Since UMs are not considered in training a classifier, they would decay the performance of a trained classifier in identifying the target movements (TMs), which would cause some undesired actions in control of multifunctional prostheses. In this study, the impact of UMs was systemically investigated in both able-bodied subjects and transradial amputees. Our results showed that the UMs would be unevenly classified into all classes of the TMs. To reduce the impact of the UMs on the performance of a classifier, a new training strategy that would categorize all possible UMs into a new movement class was proposed and a metric called Reject Ratio that is a measure of how many UMs is rejected by a trained classifier was adopted. The results showed that the average Reject Ratio across all the participants was greater than 91%, meanwhile the average classification accuracy of TMs was above 99% when UMs occurred. This suggests that the proposed training strategy could greatly reduce the impact of UMs on the performance of the trained classifier in identifying the TMs and may enhance the robustness of myoelectric control in clinical applications.

New Control Strategies for Multifunctional Prostheses that Combine Electromyographic and Speech Signals

The control of multifunctional myoelectric prostheses is commonly limited due to the lack of sufficient electromyography (EMG) signals after amputation. With a goal of developing easy-to-use and flexibly controlled multifunctional prostheses, the authors propose two control strategies that fuse EMG and speech signals. In the first, speech signals switch the joint-mode, and EMG signals determine a motion class to actuate the target movement. In the second, speech signals select a motion class directly, and EMG signals actuated the movement. Experimental results showed that the proposed strategies could improve control performance and enhance the operational efficiency significantly, suggesting that signal fusion is a feasible way to effectively strengthen interactions between humans and machines.

Adaptive myoelectric pattern recognition toward improved multifunctional prosthesis control

The non-stationary property of electromyography (EMG) signals in real life settings usually hinders the clinical application of the myoelectric pattern recognition for prosthesis control. The classical EMG pattern recognition approach consists of two separate steps: training and testing, without considering the changes between training and testing data induced by electrode shift, fatigue, impedance changes and psychological factors, and often results in performance degradation. The aim of this study was to develop an adaptive myoelectric pattern recognition system, aiming to retrain the classifier online with the testing data without supervision, providing a self-correction mechanism for suppressing misclassifications. This paper presents an adaptive unsupervised classifier based on support vector machine (SVM) to improve the classification performance. Experimental data from 15 healthy subjects were used to evaluate performance. Preliminary study on intra-session and inter-session EMG data was conducted to verify the performance of the unsupervised adaptive SVM classifier. The unsupervised adaptive SVM classifier outperformed the conventional SVM by 3.3% and 8.0% for the combination of time-domain and autoregressive features in the intra-session and inter-session tests, respectively. The proposed approach is capable of incorporating the useful information in testing data to the classification model by taking into account the overtime changes in the testing data with respect to the training data to retrain the original classifier, therefore providing a self-correction mechanism for suppressing misclassifications.

Exploring the mechanism of neural-function reconstruction by reinnervated nerves in targeted muscles

A lack of myoelectric sources after limb amputation is a critical challenge in the control of multifunctional motorized prostheses. To reconstruct myoelectric sources physiologically related to lost limbs, a newly proposed neural-function construction method, targeted muscle reinnervation (TMR), appears promising. Recent advances in the TMR technique suggest that TMR could provide additional motor command information for the control of multifunctional myoelectric prostheses. However, little is known about the nature of the physiological functional recovery of the reinnervated muscles. More understanding of the underlying mechanism of TMR could help us fine tune the technique to maximize its capability to achieve a much higher performance in the control of multifunctional prostheses. In this study, rats were used as an animal model for TMR surgery involving transferring a median nerve into the pectoralis major, which served as the target muscle. Intramuscular myoelectric signals reconstructed following TMR were recorded by implanted wire electrodes and analyzed to explore the nature of the neural-function reconstruction achieved by reinnervation of targeted muscles. Our results showed that the active myoelectric signal reconstructed in the targeted muscle was acquired one week after TMR surgery, and its amplitude gradually became stronger over time. These preliminary results from rats may serve as a basis for exploring the mechanism of neural-function reconstruction by the TMR technique in human subjects.

Effects of non-training movements on the performance of motion classification in electromyography pattern recognition.

In electromyography pattern-recognition-based control of a multifunctional prosthesis, it would be inevitable for the users to unintentionally perform some classes of movements that are excluded from the training motion classes of a classifier, which might decay the performance of a trained classifier. It remains unknown how these untrained movements, designated as non-target movements (NTMs) in the study, would affect the performance of a trained classifier in the control of multifunctional prostheses. The goal of the current study was to evaluate the effects of NTMs on the performance of movement classification. Five classes of target movements (TMs) and four classes of NTMs were considered in this pilot study. A classifier based on a linear discriminant analysis (LDA) was trained with the electromyography (EMG) signals from the five TMs and the effects of the four NTMs were examined by feeding the EMG signals of the four NTMs to the trained classifier. Our results showed that these NTMs were classified into one or more classes of the TMs, which would cause the unexpected movements of prostheses. A method to reduce the effects of NTMs has been proposed in the study and our results showed that the averaged classification accuracies of the corrected classifiers were above 99% for the healthy subjects.

Can transcranial direct current stimulation enhance performance of myoelectric control for multifunctional prosthesis?

Pattern recognition based myoelectric control has been studied by many researchers. However, the classification accuracy was pretty low for amputees towards multifunctional prosthesis control in practice. In this work, a novel method of transcranial direct current stimulation (tDCS) which can modulate brain activity was used to enhance performance for myoelectric prosthesis control. The pilot study was conducted on three able-bodied subjects and one transradial amputee. Surface electromyography (EMG) signals were acquired from both arms when performing eleven hand and wrist motions in pre-tDCS and post-tDCS sessions. Time domain (TD) features and linear discriminant analysis (LDA) classifier were adopted to process EMG. For the non-dominant hand of the healthy subjects, active anodal tDCS of the contralateral primary motor cortex was able to significantly improve average classification accuracy by 3.82% (p <; 0.05), while sham tDCS could not have such effect (p > 0.05). For amputated (phantom) hand of the amputee, active anodal tDCS was able to significantly improve average classification accuracy by 12.56%, while sham tDCS could not have such effect. For the dominant hand and intact hand, the average classification accuracies were stable and not significantly improved using either active tDCS or sham tDCS. The results show that tDCS is a powerful noninvasive method to modulate brain function and enhance EMG classification performance especially for the amputated hand towards multifunctional prosthesis control. The method proposed has a huge potential to promote EMG pattern recognition based control scheme to clinical application.

Application of a self-enhancing classification method to electromyography pattern recognition for multifunctional prosthesis control

BackgroundThe nonstationary property of electromyography (EMG) signals usually makes the pattern recognition (PR) based methods ineffective after some time in practical application for multinational prosthesis. The conventional EMG PR, which is accomplished in two separate steps: training and testing, ignores the mismatch between training and testing conditions and often discards the useful information in testing dataset.MethodThis paper presents a novel self-enhancing approach to improve the classification performance of the electromyography (EMG) pattern recognition (PR). The proposed self-enhancing method incorporates the knowledge beyond the training condition to the classifiers from the testing data. The widely-used linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA) are extended to self-enhancing LDA (SELDA) and self-enhancing QDA (SEQDA) by continuously updating their model parameters such as the class mean vectors, the class covariances and the pooled covariance. Autoregressive (AR) and Fourier-derived cepstral (FC) features are adopted. Experimental data in two different protocols are used to evaluate performance of the proposed methods in short-term and long-term application respectively.ResultsIn protocol of short-term EMG, based on AR and FC, the recognition accuracy of SEQDA and SELDA is 2.2% and 1.6% higher than conventional that of QDA and LDA respectively. The mean results of SEQDA(C) and SEQDA (M) are improved by 2.2% and 0.75% for AR, and 1.99% and 1.1% for FC respectively when compared to QDA. The mean results of SELDA(C) and SELDA (M) are improved by 0.48% and 1.55% for AR, and 0.67% and 1.22% for FC when compared to LDA. In protocol of long-term EMG, the mean result of SEQDA is 3.15% better than that of QDA.ConclusionThe experimental results show that the self-enhancing classifiers significantly outperform the original versions using both AR and FC coefficient feature sets. The performance of SEQDA is superior to SELDA. In addition, preliminary study on long-term EMG data is conducted to verify the performance of SEQDA.

SEMG-based hand motion recognition using cumulative residual entropy and extreme learning machine

This paper proposes a scheme consisting of two novel components to recognize multiple hand motions from surface electromyography (SEMG). First, we use the cumulative residual entropy (CREn), a measure of uncertainty in a random variable, as the feature. Second, we employ the extreme learning machine (ELM), a fast and effective classifier using single-hidden layer feedforward neural network with additive neurons, to distinguish different motions. To evaluate performance of the proposed system, we compare CREn with fuzzy entropy, sample entropy, and approximate entropy, and a state-of-the-art time-domain feature; and ELM with linear discriminant analysis and support vector machine. They are tested on four channel SEMG signals acquired from ten normal subjects. Experimental results indicate that the classification accuracies of CREn are not only better than those of other entropies with all the classifiers, but also comparable to the time-domain feature for all the segment lengths of 200, 250 and 1,000 ms with all classifiers that are evaluated. Furthermore, the computational complexity of CREn is lower than those of other features, and ELM performs significantly faster than other classifiers without sacrificing any performance. It suggests that the proposed CREn-ELM scheme has the potential to be applied to real-time control of SEMG-based multifunctional prosthesis.

Evaluation of data mining approaches for the control of multifunctional arm prostheses

Commercially available hand prostheses are controlled by myoelectric signals, biosignals that must be accurately interpreted if a prosthesis is to work effectively. We propose the use of standardized muscle contractions to select grasp types; these a

Conditioning and Sampling Issues of EMG Signals in Motion Recognition of Multifunctional Myoelectric Prostheses

Historically, the investigations of electromyography (EMG) pattern recognition-based classification of intentional movements for control of multifunctional prostheses have adopted the filter cut-off frequency and sampling rate that are commonly used in EMG research fields. In practical implementation of a multifunctional prosthesis control, it is desired to have a higher high-pass cut-off frequency to reduce more motion artifacts and to use a lower sampling rate to save the data processing time and memory of the prosthesis controller. However, it remains unclear whether a high high-pass cut-off frequency and a low-sampling rate still preserve sufficient neural control information for accurate classification of movements. In this study, we investigated the effects of high-pass cut-off frequency and sampling rate on accuracy in identifying 11 classes of arm and hand movements in both able-bodied subjects and arm amputees. Compared to a 5-Hz high-pass cut-off frequency, excluding the EMG components below 60Hz decreased the average accuracy of 0.1% in classifying the 11 movements across able-bodied subjects and increased the average accuracy of 0.1 and 0.4% among the transradial (TR) and shoulder disarticulation (SD) amputees, respectively. Using a 500 Hz instead of a 1-kHz sampling rate, the average classification accuracy only dropped about 2.0% in arm amputees. The combination of sampling rate and high-pass cut-off frequency of 500 and 60Hz only resulted in about 2.3% decrease in average accuracy for TR amputees and 0.4% decrease for SD amputees in comparison to the generally used values of 1kHz and 5Hz. These results suggest that the combination of sampling rate of 500Hz and high-pass cut-off frequency of 60Hz should be an optimal selection in EMG recordings for recognition of different arm movements without sacrificing too much of classification accuracy which can also remove most of motion artifacts and power-line interferences for improving the performance of myoelectric prosthesis control.