Hand Motion Recognition Research Articles

ML-Based Approach for Enhancing Sewing Operator Training in the Apparel Industry Using Hand Movement Recognition

ML-Based Approach for Enhancing Sewing Operator Training in the Apparel Industry Using Hand Movement Recognition

A Novel Transformer-Based Approach for Simultaneous Recognition of Hand Movements and Force Levels in Amputees Using Flexible Ultrasound Transducers.

Accurate hand motion intention recognition is essential for the intuitive control of intelligent prosthetic hands and other human-machine interaction systems. Sonomyography, which can detect the changes in muscle morphology and structure precisely, is a promising signal source for fine hand movement recognition. However, sonomyography measured by traditional rigid ultrasound probes may suffer from poor acoustic coupling because the rigid probe surfaces cannot accommodate the curvilinear shape of the human body, particularly in the case of small and irregular residual limbs in amputees. In this study, we used a self-designed lightweight, flexible, and wearable ultrasound transducer to acquire muscle ultrasound images, and proposed a sonomyography transformer (SMGT) model for simultaneous recognition of hand movements and force levels. The performance of SMGT was systematically compared to two commonly used image processing methods, HOG and Gray Gradient, as well as a deep CNN model, in simultaneously recognizing ten classes of hand/finger movements and three force levels. Additionally, ten subjects including seven non-disabled subjects and three trans-radial amputees who are the end users of prosthetic hands were recruited to evaluate the effectiveness of SMGT. Results showed that our proposed method achieved average classification accuracies of 98.4% ± 0.6% and 96.2% ± 3.0% in non-disabled subjects and amputee subjects, respectively, which are much higher than those of other methods. This study provided a valuable approach for ultrasound-based hand motion recognition that may promote the applications of intelligent prosthetic hands.

Fusing sEMG and EEG to Increase the Robustness of Hand Motion Recognition Using Functional Connectivity and GCN

Surface electromyogram (sEMG) is widely used in active rehabilitation control for stroke patients. However, the accuracy of movement recognition using sEMG signals is affected by abnormal states such as muscular fatigue and muscle weakness. In this article, a multimodal fusion strategy of electroencephalogram (EEG) and sEMG is proposed to improve the accuracy and robustness of hand motion recognition. It is an end-to-end approach based on a graph theory, in which the temporal signals of EEG and sEMG are considered as the features of nodes, and the functional connectivity is considered as the weights of edges. Four topologies, namely, 2EnMe, 2EwMe, 5EnMe, and 5EwMe, are proposed, and two standardization methods are tested for each topology. Then, three functional connectivity methods are investigated, namely, Pearson coefficient, mutual information, and coherence. Ten rounds of fivefold cross validation show that graph convolutional network (GCN)-2EnMe with the Pearson coefficient and min–max standardization is the best fusion model. At the fatigue levels of 0% and 30%, the achieved average accuracies are, respectively, 93.86% and 91.23%, which are significantly higher than those when using a parallel fusion method and a single-modality model. Moreover, the accuracy decrease ratio (ADR) of GCN-2EnMe is 2.80%, which is considerably better than that of a convolutional neural network (CNN) with parallel fusion (7.87%) and a CNN with single-modality sEMG (11.15%). The results show that the proposed novel EEG and sEMG fusion method has the potential to improve the accuracy and reliability of active control for stroke rehabilitation.

A Novel Method for Hand Movement Recognition Based on Wavelet Packet Transform and Principal Component Analysis with Surface Electromyogram.

As an input method of signal language, the hand movement classification technology has developed into one of the ways of natural human-computer interaction. The surface electromyogram (sEMG) signal contains abundant human movement information and has significant advantages as the input signal of human-computer interaction. However, how to effectively extract components from sEMG signals to improve the accuracy of hand motion classification is a difficult problem. Therefore, this work proposes a novel method based on wavelet packet transform (WPT) and principal component analysis (PCA) to classify six kinds of hand motions. The method applies WPT to decompose the sEMG signal into multiple sub-band signals. To efficiently extract the intrinsic components of the sEMG signal, the classification performance of different wavelet packet basis functions is evaluated. The PCA algorithm is used to reduce the dimension of the feature space composed of the features reflecting hand motions extracted from each sub-band signal. Besides, to ensure higher classification performance while reducing the dimension of the feature space by the PCA algorithm, the classification performance of different dimensions of the feature space is compared. In addition, the effects of the variability of the sEMG signal and the size of the window on the proposed method are further analyzed. The proposed method was tested on the sEMG for Basic Hand Movements Data Set and achieved an average accuracy of 96.03%. Compared with the existing research, the proposed method has better classification performance, which indicates that the research results can be applied to the fields of exoskeleton robot, rehabilitation training, and intelligent prosthesis.

Multi-modality deep forest for hand motion recognition via fusing sEMG and acceleration signals.

Bio-signal based hand motion recognition plays a critical role in the tasks of human-machine interaction, such as the natural control of multifunctional prostheses. Although a large number of classification technologies have been taken to improve the motion recognition accuracy, it is still a challenge to achieve acceptable performance for multiple modality input. This study proposes a multi-modality deep forest (MMDF) framework to identify hand motions, in which surface electromyographic signals (sEMG) and acceleration signals (ACC) are fused at the input level. The proposed MMDF framework constitutes of three main stages, sEMG and ACC feature extraction, feature dimension reduction, and a cascade structure deep forest for classification. A public database "Ninapro DB7" is used to evaluate the performance of the proposed framework, and the experimental results show that it can achieve a significantly higher accuracy than that of competitors. Besides, our experimental results also show that MMDF outperforms other traditional classifiers with the input of the single modality of sEMG signals. In sum, this study verifies that ACC signals can be an excellent supplementary for sEMG, and MMDF is a plausible solution to fuse mulit-modality bio-signals for human motion recognition.

A transfer fusion framework for body sensor networks (BSNs): Dynamic domain adaptation from distribution evaluation to domain evaluation

Information fusion is a scenario-driven, long-term, challenging task in body sensor networks (BSNs). Recent strategies, such as supervised or semi-supervised learning, rely heavily on fusion-decision models that are supported by prior knowledge or experience. Such techniques typically need a large amount of labeled physiological data and require the target data to have the same distribution as the prior data, which is challenging to achieve in practice. Domain adaptation (DA) is considered a viable solution for the cross-domain problem in BSNs. However, the correlation between the fusion rules and DA was rarely considered in existing studies. The independence of the domain deviation for each sensing source was ignored. This study aims to provide a universal solution for cross-domain information fusion scenarios in BSNs. Firstly, a transfer fusion framework (TF) was proposed to simultaneously solve the problems of the information fusion and multidomain deviation in BSNs by adjusting the correlation between the DA and fusion. Secondly, a DA algorithm based on dynamic domain evaluation (DDE) was proposed for the DA stage of the TF framework. In addition, we provided a feasible solution for quantitatively evaluating the domain weights. Extensive experiments on emotion recognition, fall detection, daily activity recognition, and hand movement recognition have verified the adaptability and promising performance of TF and DDE for cross-domain information fusion in BSNs. The average classification accuracy of the TF is 8.39% and 6.71% higher than that of the conventional fusion transfer framework and the baseline method, respectively. In all transfer tasks, DDE shows the best performance, which is 4.66% higher than the classification accuracy of the optimal comparison method. This study can provide technical support for BSN applications based on cross-domain information fusion.

A stacked triboelectric nanogenerator coupled with elastomer and non-elastomer for mechanical energy harvesting and hand motion recognition

It is common to improve the adaptability and contact effect by using elastomer materials to fabricate triboelectric nanogenerator (TENG). However, compared with the traditional TENG made of non-elastomer, the existing design still has the disadvantages of low output and cumbersome preparation. This study proposes a TENG based on stacking teflon (PTFE), fluoro rubber (FKM), and polyurethane sponge (PU), the TENG not only combines the advantages of an elastomer and non-elastomer but also integrates the triboelectric characteristics of three materials to achieve higher performance. The stacked PTFE/FKM/PU TENG in single-electrode mode achieves a maximum instantaneous open-circuit voltage of 1250 V and short-circuit current of 108.9 μA, driving more than 1000 LEDs. It can also effectively recognize motion of flapping, wiping, squeezing, scratching, clicking, or poking by specific output feedback. Therefore, the stacked PTFE/FKM/PU TENG is expected to contribute to collection of discrete energy and development of human sensing technology.

A Review of Hand Function Rehabilitation Systems Based on Hand Motion Recognition Devices and Artificial Intelligence.

The incidence of stroke and the burden on health care and society are expected to increase significantly in the coming years, due to the increasing aging of the population. Various sensory, motor, cognitive and psychological disorders may remain in the patient after survival from a stroke. In hemiplegic patients with movement disorders, the impairment of upper limb function, especially hand function, dramatically limits the ability of patients to perform activities of daily living (ADL). Therefore, one of the essential goals of post-stroke rehabilitation is to restore hand function. The recovery of motor function is achieved chiefly through compensatory strategies, such as hand rehabilitation robots, which have been available since the end of the last century. This paper reviews the current research status of hand function rehabilitation devices based on various types of hand motion recognition technologies and analyzes their advantages and disadvantages, reviews the application of artificial intelligence in hand rehabilitation robots, and summarizes the current research limitations and discusses future research directions.

Deep Convolutional Generative Adversarial Network-Based EMG Data Enhancement for Hand Motion Classification.

The acquisition of bio-signal from the human body requires a strict experimental setup and ethical approvements, which leads to limited data for the training of classifiers in the era of big data. It will change the situation if synthetic data can be generated based on real data. This article proposes such a kind of multiple channel electromyography (EMG) data enhancement method using a deep convolutional generative adversarial network (DCGAN). The generation procedure is as follows: First, the multiple channels of EMG signals within sliding windows are converted to grayscale images through matrix transformation, normalization, and histogram equalization. Second, the grayscale images of each class are used to train DCGAN so that synthetic grayscale images of each class can be generated with the input of random noises. To evaluate whether the synthetic data own the similarity and diversity with the real data, the classification accuracy index is adopted in this article. A public EMG dataset (that is, ISR Myo-I) for hand motion recognition is used to prove the usability of the proposed method. The experimental results show that adding synthetic data to the training data has little effect on the classification performance, indicating the similarity between real data and synthetic data. Moreover, it is also noted that the average accuracy (five classes) is slightly increased by 1%–2% for support vector machine (SVM) and random forest (RF), respectively, with additional synthetic data for training. Although the improvement is not statistically significant, it implies that the generated data by DCGAN own its new characteristics, and it is possible to enrich the diversity of the training dataset. In addition, cross-validation analysis shows that the synthetic samples have large inter-class distance, reflected by higher cross-validation accuracy of pure synthetic sample classification. Furthermore, this article also demonstrates that histogram equalization can significantly improve the performance of EMG-based hand motion recognition.

Toward Precision Recognition of Complex Hand Motions: Wearable Thermoelectrics by Synergistic 2D Nanostructure Confinement and Controlled Reduction

AbstractSmart wearable electronics have attracted increasing attention because of their great prospects in motion monitoring. To date, a specific design of thermoelectric wearable devices aiming at precision monitoring is still challenging, although thermoelectric materials and devices have witnessed significant developments. In this work, intercalated composites of reduced graphene oxide/reduced poly(3,4‐ethylenedioxythiophene):poly(styrene sulfonate) (rGO/rPEDOT:PSS) are reported through the insertion of PEDOT:PSS into 2D graphene oxide layers followed by precise optimization of oxidation‐level of composite, exhibiting an outstanding thermoelectric performance along with excellent thermoelectric stability and mechanical flexibility. Furthermore, a novel‐concept, simple‐structural, self‐powered thermoelectric wearable sensor is demonstrated with rGO/rPEDOT:PSS composite as the active sensing component of the thermoelectric device. Combined with the excellent thermoelectric performance, ingenious device design, and optimized algorithm, the thermoelectric wearable device achieves precision recognition of various hand motions. This study provides a new way to design wearable sensors toward the precision recognition of human motions.

Analysis of tissue electrical properties on bio-impedance variation of upper limps

Upper limb loss has a significant impact on individual socioeconomic life. Human-machine interface (HMI) using surface electromyography (sEMG) establishes a link between the user and a hand prosthesis to recognize hand gestures and motions which allows the control of robotic machines and prostheses to perform dexterous tasks. Numerous methods aimed to enhance hand gesture and motion recognition toward an HMI. Bio-impedance analysis (BIA) is a noninvasive way of assessing body compositions and has been recently used for hand motion interpretation using `brute force? pattern recognition. The impedance variation in the body mostly depends on the precise stimulation using appropriate electrical features of the associated tissue layers. It has been reported that the electrical properties of these layers varied significantly. Thus, it is essential to investigate the influence of these variations on the stimulator design for the hand motion interpretation. This may not be possible using experimental approaches. Alternatively, using highly advanced computational models, this can be readily investigated by attaining the available range of the electrical properties of each tissue layer and applying appropriate boundary conditions and simulation settings. The computational models are composed of a volume conductor of the human arm model and electrode settings. Also, two different computational study methods were used to determine the influence of the tissues? dielectric properties on the results. The quasistatic approximation was used by only considering the resistivity of the anatomical layers and the transient simulation was used to analyze the capacitive impact on the results. Finite element (FE) models were developed to simulate the potential distribution inside the skin, muscle, and bone layers of the upper arm for given electrode settings. Then, simulation results were recorded for various electrical properties and different study types. It was shown that the capacitive influence of the tissue may not be ignored for certain conditions due to significant variation in the induced electrical potential variation along with the target muscle. Also, the influence of the individual tissue?s electrical properties was investigated using a set of dielectric parameters. The results showed that the skin and muscle layers have a significant impact on the electrical potential variation across the muscle length.

User independent hand motion recognition for robot arm manipulation

User independent hand motion recognition for robot arm manipulation

An approach to continuous hand movement recognition using SEMG based on features fusion

An approach to continuous hand movement recognition using SEMG based on features fusion

SEMG Based Recognition of Hand Motions for Lower Limb Prostheses

Aim: On multiple muscle locations, surface electromyography (sEMG) signals were recorded to predict the effect of different hand movements. Background: Myoelectric information is a non-stationary signal, so extracting correct features is important to boost any myoelectric control devices' performance. Myoelectric signal is an electrical activity recorded by a surface electrode at various movements of the muscles. Objective: The study presented pattern recognition classification methods to select an excellent algorithm for controlling the SEMG signal. Method: Various time domain and frequency domain parameters were extracted prior to conducting the classifier test. Result: For the evaluation of the results for the recorded data (of all six movements), confusion matrix for neural network, support vector machine (SVM), DT, and linear discriminant analysis (LDA) classifiers is presented. Conclusion: This present study will be a step ahead in analyzing different problems for developing lower limb prosthesis.

SEMG Signals Characterization and Identification of Hand Movements by Machine Learning Considering Sex Differences

Developing a robust machine-learning algorithm to detect hand motion is one of the most challenging aspects of prosthetic hands and exoskeleton design. Machine-learning methods that considered sex differences were used to identify and describe hand movement patterns in healthy individuals. To this purpose, surface Electromyographic (sEMG) signals have been acquired from muscles in the forearm and hand. The results of statistical analysis indicated that most of the same muscle pairs in the right hand (females and males) showed significant differences during the six hand movements. Time features were used an as input to machine-learning algorithms for the recognition of six gestures. Specifically, two types of hand-gesture recognition methods that considered sex differences(differentiating sex datasets and adding a sex label)were proposed and applied to the k-nearest neighbor (k-NN), support vector machine (SVM) and artificial neural network (ANN) algorithms for comparison. In addition, a t-test statistical analysis approach and 5-fold cross validation were used as complements to verify whether considering sex differences could significantly improve classification performance. It was demonstrated that considering sex differences can significantly improve classification performance. The ANN algorithm with the addition of a sex label performed best in movement classification (98.4% accuracy). In the future, hand movement recognition algorithms considering sex differences could be applied to control systems for prosthetic hands or exoskeletons.

Recognition of Hand Movements for Specially-Abled

Abstract: The procedure through a user can make gestures to provide or deliver information is called Recognizing gestures. In daily life our physical gestures have proved to be an incredible tool of communication. An entire set of physical gesture man contribute to a whole language like that of the sign language. The have the ability to smoothly dispatch information, facts, emotions and feelings. Rightly so a combination of physical behaviour and emotional expression. Broadly speaking, hand gesture can be broken down into two categories: static gestures and dynamic gestures. For the initial type, the gesture the hand makes denotes a sign. For the type that follows which is dynamic gestures, there is a sequence of actions that takes shape of gestures, taking place which communicates certain message. Recognizing gestures can predict the intent of the user via a model without human interference. Recent times has seen tremendous amount of work done on designing, building, and upgrading existing Neural Network model into a different form of structure which carries the original purpose of classification but does so with even more promising accuracy and precision. The EfficientNet Model is one that has recently resurfaced. A model constructed by scaling the parameters of an existing Convolutional Neural Network. In this research, we suggest employing an EfficientNet Convolutional Neural Network to identify static hand motions. The process would involve extracting static hand gestures, converting them to gray scale to lessen the computational cost. Segregating the dataset obtained into training, testing, and validating datasets. Later in the upcoming Major Project, a CNN will be built with EfficientNet, and K-fold cross validation will be used for training and testing. We will later in the upcoming Major Project, compare our findings to earlier research and work done on the architecture of Neural Networks. Keywords: Hand Gestures, CNN, EfficientNet, ResNet, disability

Discrete Hand Motion Recognition Method Based on Transfer Learning

Discrete Hand Motion Recognition Method Based on Transfer Learning

A Noble Classification Framework for Data Glove Classification of a Large Number of Hand Movements

The recognition of hand movements is an important method for human-computer interaction (HCI) technology, and it is widely used in virtual reality and other HCI areas. While many valuable efforts have been made, efficient ways to capture over 20 types of hand movements with high accuracy by one data glove are still lacking. This paper addresses a new classification framework for 52 hand movements. This classification framework includes the following two parts: the movement detection algorithm and the movement classification algorithm. The fine K-nearest neighbor (Fine KNN) is the core of the movement detection algorithm. The movement classification algorithm is composed of downsampling in data preparation and a new deep learning network named the DBDF network. Bidirectional Long Short-Term Memory (BiLSTM) is the main part of the DBDF network. The results of experiments using the Ninapro DB1 dataset demonstrate that our work can classify more types of hand movements than related algorithms with a precision of 93.15%.

Dynamic Gesture Recognition Using Surface EMG Signals Based on Multi-Stream Residual Network.

Gesture recognition technology is widely used in the flexible and precise control of manipulators in the assisted medical field. Our MResLSTM algorithm can effectively perform dynamic gesture recognition. The result of surface EMG signal decoding is applied to the controller, which can improve the fluency of artificial hand control. Much current gesture recognition research using sEMG has focused on static gestures. In addition, the accuracy of recognition depends on the extraction and selection of features. However, Static gesture research cannot meet the requirements of natural human-computer interaction and dexterous control of manipulators. Therefore, a multi-stream residual network (MResLSTM) is proposed for dynamic hand movement recognition. This study aims to improve the accuracy and stability of dynamic gesture recognition. Simultaneously, it can also advance the research on the smooth control of the Manipulator. We combine the residual model and the convolutional short-term memory model into a unified framework. The architecture extracts spatiotemporal features from two aspects: global and deep, and combines feature fusion to retain essential information. The strategy of pointwise group convolution and channel shuffle is used to reduce the number of network calculations. A dataset is constructed containing six dynamic gestures for model training. The experimental results show that on the same recognition model, the gesture recognition effect of fusion of sEMG signal and acceleration signal is better than that of only using sEMG signal. The proposed approach obtains competitive performance on our dataset with the recognition accuracies of 93.52%, achieving state-of-the-art performance with 89.65% precision on the Ninapro DB1 dataset. Our bionic calculation method is applied to the controller, which can realize the continuity of human-computer interaction and the flexibility of manipulator control.

Robust Continuous Hand Motion Recognition Using Wearable Array Myoelectric Sensor

With the advantages of comfortable wearing and outdoor usage, the myoelectric gesture recognition techniques have gained much attention in the field of human-machine interaction (HMI). The purpose of this study is to optimize model structure and transfer generalized features to improve the robustness of myoelectric hand motion decoding. We derived the hand motion recognition framework from the muscle synergy theory, which is formulated as a temporal convolutional (TC) model of array sEMG signals, then a hierarchical myoelectric decoding model was proposed to predict simultaneous and continuous hand motion. The model was trained by the methods of unsupervised low-level feature learning and automated data labeling to minimize training supervision. Extensive experiments on the public sEMG database (17 subjects in Biopatrec) show that the TC model can extract muscle synergy features with higher fidelity ( R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> = 0.85±0.23) than the traditional instantaneous mixture model, the results of online test demonstrate robust myoelectric decoding on multiple simultaneous and continuous hand motions. More importantly, the analysis of weights visualization shows that the low-level feature representation layer of TC model can be migrated across the individuals, which provides a transferrable feature extraction layer for generalized hand motion decoding.