Limb Movement Classification Research Articles

Multimodal Fusion Convolutional Neural Network Based on sEMG and Accelerometer Signals for Intersubject Upper Limb Movement Classification

The variation in the distributions of recorded data between individuals leads to low classification accuracy. To address this issue, we introduce a multimodal fusion convolutional neural network (MFCNN). This network extracts common information from surface electromyography (sEMG) and accelerometer signals of different subjects using a two-stream CNN. To enhance the classification accuracy of a particular subject, a fine-tuning approach was implemented. The performance of the proposed method was assessed in four different scenarios, which include inter-subject classification, inter-subject classification when training data from multiple subjects, fine-tuned inter-subject classification, and fine-tuned inter-subject classification when training data from multiple subjects. The results demonstrate that in the inter-subject scenario, when multiple subjects are available for training, the MFCNN achieves higher classification accuracy (p < 0.05) than other neural networks and support vector machines (SVMs) that use sEMG signals (NN and SVM), accelerometer signals (accNN and accSVM), sEMG and accelerometer signals (MFNN and MFSVM) as inputs, as well as a CNN that uses sEMG signals as input after fine-tuning. Furthermore, compared with an MFCNN model trained with data from a single subject and an accCNN model trained with data from a single subject or multiple subjects, an MFCNN trained with multiple subjects demonstrated better performance on new subjects after fine-tuning (p < 0.05). This method can learn common features among different subjects and improve the performance of classification among subjects. Our proposed method demonstrates the innovation of using a multimodal fusion approach and two-stream CNN to improve inter-subject classification accuracy in upper limb movements.

Lower Limb Motion Classification Using Energy Density Features of Surface Electromyography Signals’ Activation Region and Dynamic Ensemble Model

Accurate classification of lower-limb movements based on surface electromyography (sEMG) signals is vital for wearable auxiliary devices in military and medical rehabilitation. It is difficult to accurately obtain the effective features of the active region and establish a reliable classification model of complex lower-limb motion modes due to the strong dynamic time variability of the sEMG signals. Thus, an onset point detection algorithm, namely the maximum interclass variance (MIV) algorithm, was proposed to determine an accurate activation signal region. Then, a variational modal correlation dimension (VMCD) algorithm was proposed for feature construction to obtain the energy density feature (EDF) of the activation region, which reflects the activation signal’s amplitude energy distribution state. Finally, a random sampling multi-classifier dynamic ensemble (RSDE) algorithm was proposed to rapidly construct an ensemble model by adapting to the feature changes of the sEMG signals. A total of 25 subjects were recruited to participate in the experiment. Compared with the other methods, the proposed method could accurately classify five lower-limb motions by collecting only the data from three lower-limb muscles. The average accuracy and response time were better than those of many existing classification methods. This indicates that the proposed method can improve the man-machine cooperative response speed, which is beneficial to enhance the application effect of sEMG auxiliary equipment.

Lower limb motion recognition based on surface electromyography signals and its experimental verification on a novel multi-posture lower limb rehabilitation robots☆

The construction of motion decoder based on surface Electromyography (sEMG) signals is an important part in the clinical trial of rehabilitation robot. Its performance directly determines the success of clinical trial. However, feature extraction is essential in motion decoder. A single feature such as time domain and frequency domain can achieve good classification results, but it is only suitable for a single posture. Hence, for the lying and sitting postures, different feature analysis and their combination are used in this study to improve the sEMG-based lower limb motion classification performance. Nine participants in the clinical trial performed four different movements respectively. Through feature extraction and pattern recognition of sEMG, the trained motion decoders were obtained. The control commands are sent to the robot through labels retrieval to drive the lower limbs for corresponding rehabilitation training. The effectiveness of the based on sEMG signals control method is verified through real-time experimental analysis.

Hybridized neural network for upper limb movement detection using EEG signals

PurposeThis paper aims to propose a new upper limb movement classification with two phases like pre-processing and classification. Investigation of human limb movements is a significant topic in biomedical engineering, particularly for treating patients. Usually, the limb movement is examined by analyzing the signals that occurred by the movements. However, only few attempts were made to explore the correlations among the movements that are recognized by the human brain.Design/methodology/approachThe initial process is the pre-processing that is performed for detecting and removing noisy channels. The artifacts are marked by band-pass filtering that discovers the values below and above thresholds of 200 and –200 µV, correspondingly. It also discovers the trials with unusual joint probabilities, and the trials with unusual kurtosis are also determined using this method. After this, the pre-processed signals are subjected to a classification process, where the neural network (NN) model is used. The model finally classifies six movements like “elbow extension, elbow flexion, forearm pronation, forearm supination, hand open, and hand close,” respectively. To make the classification more accurate, this paper intends to optimize the weights of NN by a new hybrid algorithm known as bypass integrated jaya algorithm (BI-JA) that hybrids the concept of rider optimization algorithm (ROA) and JA. Finally, the performance of the proposed model is proved over other conventional models concerning certain measures like accuracy, sensitivity, specificity, and precision, false positive rate, false negative rate, false discovery rate, F1-score and Matthews correlation coefficient.FindingsFrom the analysis, the adopted BI-JA-NN model in terms of accuracy was high at 80th population size was 7.85%, 3.66%, 7.53%, 2.09% and 0.52% better than Levenberg–Marquardt (LM)-NN, firefly (FF)-NN, JA-NN, whale optimization algorithm (WOA)-NN and ROA-NN algorithms. On considering sensitivity, the proposed method was 2%, 0.2%, 5.01%, 0.29% and 0.3% better than LM-NN, FF-NN, JA-NN, WOA-NN and ROA-NN algorithms at 50th population size. Also, the specificity of the implemented BI-JA-NN model at 80th population size was 7.47%, 4%, 7.05%, 2.1% and 0.5% better than LM-NN, FF-NN, JA-NN, WOA-NN and ROA-NN algorithms. Thus, the betterment of the presented scheme was proved.Originality/valueThis paper adopts the latest optimization algorithm called BI-JA to introduce a new upper limb movement classification with two phases like pre-processing and classification. This is the first work that uses BI-JA based optimization for improving the upper limb movement detection using electroencephalography signals.

A Review of the Role of Machine Learning Techniques towards Brain–Computer Interface Applications

This review article provides a deep insight into the Brain–Computer Interface (BCI) and the application of Machine Learning (ML) technology in BCIs. It investigates the various types of research undertaken in this realm and discusses the role played by ML in performing different BCI tasks. It also reviews the ML methods used for mental state detection, mental task categorization, emotion classification, electroencephalogram (EEG) signal classification, event-related potential (ERP) signal classification, motor imagery categorization, and limb movement classification. This work explores the various methods employed in BCI mechanisms for feature extraction, selection, and classification and provides a comparative study of reviewed methods. This paper assists the readers to gain information regarding the developments made in BCI and ML domains and future improvements needed for improving and designing better BCI applications.

Upper Limb Movement Classification Via Electromyographic Signals and an Enhanced Probabilistic Network.

Few studies in the literature have researched the use of surface electromyography (sEMG) for motor assessment post-stroke due to the complexity of this type of signal. However, recent advances in signal processing and machine learning have provided fresh opportunities for analyzing complex, non-linear, non-stationary signals, such as sEMG. This paper presents a method for identification of the upper limb movements from sEMG signals using a combination of digital signal processing, that is discrete wavelet transform, and the enhanced probabilistic neural network (EPNN). To explore the potential of sEMG signals for monitoring motor rehabilitation progress, this study used sEMG signals from a subset of movements of the Arm Motor Ability Test (AMAT) as inputs into a movement classification algorithm. The importance of a particular frequency domain feature, that is the ratio of the mean absolute values between sub-bands, was discovered in this work. An average classification accuracy of 75.5% was achieved using the proposed approach with a maximum accuracy of 100%. The performance of the proposed method was compared with results obtained using three other classification algorithms: support vector machine (SVM), k-Nearest Neighbors (k-NN), and probabilistic neural network (PNN) in terms of sEMG movement classification. The study demonstrated the capability of using upper limb sEMG signals to identify and distinguish between functional movements used in standard upper limb motor assessments for stroke patients. The classification algorithm used in the proposed method, EPNN, outperformed SVM, k-NN, and PNN.

Characterizing Multiple Patterns of Motor Intent Using Spatial-Temporal Information for Intuitively Active Motor Training in Stroke Survivors.

Upper extremity motor function loss severely affects stroke survivors during daily life activities. Different rehabilitation robotic systems have been developed to allow stroke survivors regain their motor function. Meanwhile, most of the robots only operate in a passive mode and restrict the users to navigate predefined trajectories which may not align with their motion intent, thus limiting motor recovery. One way to resolve this issue would be to utilize a decoded movement intent to trigger intuitively active motor training for patients. In this direction, this study proposed and investigated the use of spatial-temporal neuromuscular descriptor (STD) for optimal decoding of multiple patterns of movement intents in patient to provide inputs for active motor training in the rehabilitation robotic systems. The STD performance was validated using High-Density surface electromyogram recordings from five stroke survivors who performed 21 limb movements. Experimental results show that the STD achieved a significant reduction in limb movement classification error (13.36%) even in the presence of the inevitable White Gaussian Noise compared to other methods (p<0.05). The STD also showed obvious class separability for individual movement. Findings from this study suggest that the STD may provide potential inputs for intuitively active motor training in stroke rehabilitation robotic systems.Clinical Relevance- This study showed that spatial-temporal neuromuscular information could aid adequate decoding of movement intents upon which intuitively active motor training could be achieved in stroke rehabilitation robotic systems.

Feature Extraction and Classification of Lower Limb Motion Based on sEMG Signals

Surface electromyography (sEMG) signals can reflect the body motion information and are widely used in military, medical rehabilitation, industrial production. The lower limb motion classification mainly includes feature extraction and classification model establishment. Firstly, we proposed a feature extraction method based on the wavelet packet transform (WPT) and principal component analysis (PCA). We used the wavelet packet method to decompose the sEMG signals of three muscles in the lower limb and got the 24-dimensional eigenvector. To reduce the calculation and improve the speed of the classification model, we used the PCA method to reduce the dimension of the feature vector and got the 3-dimensional eigenvector. Then, we proposed a method based on the scale unscented Kalman filter (SUKF) and neural network (NN) for lower limb motion classification. Through the scale correction unscented transform (SCUT) could optimize the neural network weight and improve lower limb motion classification accuracy. Finally, the experimental results showed that the average accuracy was 93.7%. Compared with the backpropagation neural network (BPNN) and wavelet neural network (WNN), this method could improve the accuracy and reliability of the lower limb motion classification.

VMD-based denoising methods for surface electromyography signals

Objective. Since noise is inevitably introduced during the measurement process of surface electromyographic (sEMG) signals, two novel methods for denoising based on the variational mode decomposition (VMD) method were proposed in this work. Prior to this study, there has been no literature relating to how VMD is applied to sEMG denoising. Approach. The first proposed method uses the VMD method to decompose the signal into multiple variational mode functions (VMFs), each of which has its own center frequency and narrow band, and then the wavelet soft thresholding (WST) method is applied to each VMF. This method is termed the VMD-WST. The second proposed method uses the VMD method to decompose the signal into multiple VMFs, and then the soft interval thresholding (SIT) method is performed on each VMF, which is abbreviated as VMD-SIT. Ten healthy subjects and ten stroke patients participated in the experiment, and the sEMG signals of bicep brachii were measured and analyzed. In this paper, three methods are used for quantitative evaluation of the filtering performance: the signal-to-noise ratio (SNR), root mean square error and R-squared value. The proposed two methods (VMD-WST, VMD-SIT) are compared with the empirical mode decomposition (EMD) method and the wavelet method. Main results. The experimental results showed that the VMD-WST and VMD-SIT methods can effectively filter the noise effect, and the denoising effects were better than the EMD method and the wavelet method. The VMD-SIT method has the best performance. Significance. This study provides a new means of eliminating the noise of sEMG signals based on the VMD method, and it can be applied in the fields of limb movement classification, disease diagnosis, human-machine interaction and so on.

Time Derivative Moments Based Feature Extraction Approach for Recognition of Upper Limb Motions Using EMG

Electromyography pattern recognition (EMG-PR) is extensively recognized in human–machine interactive applications such as prosthesis control and rehabilitation devices. Conventional time-domain (TD) features have been shown to produce a decent performance in upper limb movements’ classification. However, performance limitation exists in terms of classification accuracy. Hence, a novel feature set extraction on time derivative moments is proposed to improve the performance of EMG-PR in upper limb motion classification. A standardized and benchmark data base NinaPro (subdatabase2) has been used for examination of the proposed feature set. A total of eight intact subjects (sub1–sub8) have been selected for eight grasping motion's EMG signal collection. For classification of features, three classification techniques (linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), and support vector machine (SVM)) have been applied. The average improvisation with respect to the conventional TD features that was achieved for LDA, QDA, and SVM are 8.08%, 7.95%, and 6.41%, respectively.

Investigating Upper Limb Movement Classification on Users with Tetraplegia as a Possible Neuroprosthesis Interface.

Spinal cord injury (SCI), stroke and other nervous system conditions can result in partial or total paralysis of individual's limbs. Numerous technologies have been proposed to assist neurorehabilitation or movement restoration, e.g. robotics or neuroprosthesis. However, individuals with tetraplegia often find difficult to pilot these devices. We developed a system based on a single inertial measurement unit located on the upper limb that is able to classify performed movements using principal component analysis. We analyzed three calibration algorithms: unsupervised learning, supervised learning and adaptive learning. Eight participants with tetraplegia (C4C7) piloted three different postures in a robotic hand. We achieved 89% accuracy using the supervised learning algorithm. Through offline simulation, we found accuracies of 76% on the unsupervised learning, and 88% on the adaptive one.

Pattern recognition of electromyography signals based on novel time domain features for amputees' limb motion classification

Feature extraction is essential in Electromyography pattern recognition (EMG-PR) based prostheses control method. Time-domain features have been shown to have good performance in upper limb movement classification. However, the performance of EMG-PR prostheses driven by the existing time-domain features is still unsatisfactory. Hence, this study proposed three new time-domain features to improve the performance of EMG-PR based strategy in arm movement classification. EMG signals were recorded from the residual arms of eight amputees while performing different upper limb movements. Then, the newly proposed features were extracted and used to classify their limb movements. Experimental results showed that the proposed features could achieved an average classification accuracy of 92.00% ± 3.11% which was 6.49% higher than that of the commonly used time-domain features (p < 0.05). With three additional metrics, the proposed features also performed better, which suggest that the new features may be potential for improving the clinical performance of EMG-PR prostheses.

3D hand motion trajectory prediction from EEG mu and beta bandpower.

A motion trajectory prediction (MTP) - based brain-computer interface (BCI) aims to reconstruct the three-dimensional (3D) trajectory of upper limb movement using electroencephalography (EEG). The most common MTP BCI employs a time series of bandpass-filtered EEG potentials (referred to here as the potential time-series, PTS, model) for reconstructing the trajectory of a 3D limb movement using multiple linear regression. These studies report the best accuracy when a 0.5-2Hz bandpass filter is applied to the EEG. In the present study, we show that spatiotemporal power distribution of theta (4-8Hz), mu (8-12Hz), and beta (12-28Hz) bands are more robust for movement trajectory decoding when the standard PTS approach is replaced with time-varying bandpower values of a specified EEG band, ie, with a bandpower time-series (BTS) model. A comprehensive analysis comprising of three subjects performing pointing movements with the dominant right arm toward six targets is presented. Our results show that the BTS model produces significantly higher MTP accuracy (R~0.45) compared to the standard PTS model (R~0.2). In the case of the BTS model, the highest accuracy was achieved across the three subjects typically in the mu (8-12Hz) and low-beta (12-18Hz) bands. Additionally, we highlight a limitation of the commonly used PTS model and illustrate how this model may be suboptimal for decoding motion trajectory relevant information. Although our results, showing that the mu and beta bands are prominent for MTP, are not in line with other MTP studies, they are consistent with the extensive literature on classical multiclass sensorimotor rhythm-based BCI studies (classification of limbs as opposed to motion trajectory prediction), which report the best accuracy of imagined limb movement classification using power values of mu and beta frequency bands. The methods proposed here provide a positive step toward noninvasive decoding of imagined 3D hand movements for movement-free BCIs.

Bispectrum-based sEMG multi-domain joint feature extraction for upper limb motion classification

sEMG based motion pattern recognition is the focus in the rehabilitation medical engineering area. In order to get more information to characterize the sEMG signal of different upper limb motions, the signal data acquisition program is designed and the non-Gaussian characteristic of the sEMG signal is analyzed, the result shows that the sEMG signal collected is non-Gaussian signal. Bispectrum as a third-order statistics contains non-Gaussian information and the integral of bispectrum slice is extracted as feature. After that, the PCA method is adopted to reduce the bispectrum feature dimension. Then, the integral of bispectrum slice after PCA and the integral value of sEMG are combined as a multi-domain joint feature called BisIE. Finally, the experiment is executed to validate the effectiveness of the feature extraction method proposed by the SVM classifier compared with power spectrum-based multi-domain joint feature MMIE. The average classification accuracy of BisIE is about 97% and that of MMIE is about 93%. Besides, for the same subject, the classification accuracy of BisIE is higher than that of MMIE. The result shows that the proposed feature BisIE is effective in promoting sEMG-based upper limb motion recognition accuracy.

Toward attenuating the impact of arm positions on electromyography pattern-recognition based motion classification in transradial amputees

BackgroundElectromyography (EMG) pattern-recognition based control strategies for multifunctional myoelectric prosthesis systems have been studied commonly in a controlled laboratory setting. Before these myoelectric prosthesis systems are clinically viable, it will be necessary to assess the effect of some disparities between the ideal laboratory setting and practical use on the control performance. One important obstacle is the impact of arm position variation that causes the changes of EMG pattern when performing identical motions in different arm positions. This study aimed to investigate the impacts of arm position variation on EMG pattern-recognition based motion classification in upper-limb amputees and the solutions for reducing these impacts.MethodsWith five unilateral transradial (TR) amputees, the EMG signals and tri-axial accelerometer mechanomyography (ACC-MMG) signals were simultaneously collected from both amputated and intact arms when performing six classes of arm and hand movements in each of five arm positions that were considered in the study. The effect of the arm position changes was estimated in terms of motion classification error and compared between amputated and intact arms. Then the performance of three proposed methods in attenuating the impact of arm positions was evaluated.ResultsWith EMG signals, the average intra-position and inter-position classification errors across all five arm positions and five subjects were around 7.3% and 29.9% from amputated arms, respectively, about 1.0% and 10% low in comparison with those from intact arms. While ACC-MMG signals could yield a similar intra-position classification error (9.9%) as EMG, they had much higher inter-position classification error with an average value of 81.1% over the arm positions and the subjects. When the EMG data from all five arm positions were involved in the training set, the average classification error reached a value of around 10.8% for amputated arms. Using a two-stage cascade classifier, the average classification error was around 9.0% over all five arm positions. Reducing ACC-MMG channels from 8 to 2 only increased the average position classification error across all five arm positions from 0.7% to 1.0% in amputated arms.ConclusionsThe performance of EMG pattern-recognition based method in classifying movements strongly depends on arm positions. This dependency is a little stronger in intact arm than in amputated arm, which suggests that the investigations associated with practical use of a myoelectric prosthesis should use the limb amputees as subjects instead of using able-body subjects. The two-stage cascade classifier mode with ACC-MMG for limb position identification and EMG for limb motion classification may be a promising way to reduce the effect of limb position variation on classification performance.

A Gaussian Mixture Model Based Classification Scheme for Myoelectric Control of Powered Upper Limb Prostheses

This paper introduces and evaluates the use of Gaussian mixture models (GMMs) for multiple limb motion classification using continuous myoelectric signals. The focus of this work is to optimize the configuration of this classification scheme. To that end, a complete experimental evaluation of this system is conducted on a 12 subject database. The experiments examine the GMMs algorithmic issues including the model order selection and variance limiting, the segmentation of the data, and various feature sets including time-domain features and autoregressive features. The benefits of postprocessing the results using a majority vote rule are demonstrated. The performance of the GMM is compared to three commonly used classifiers: a linear discriminant analysis, a linear perceptron network, and a multilayer perceptron neural network. The GMM-based limb motion classification system demonstrates exceptional classification accuracy and results in a robust method of motion classification with low computational load.