Control Of Robotic Hand Research Articles

A Multimodal Bracelet to Acquire Muscular Activity and Gyroscopic Data to Study Sensor Fusion for Intent Detection.

Researchers have attempted to control robotic hands and prostheses through biosignals but could not match the human hand. Surface electromyography records electrical muscle activity using non-invasive electrodes and has been the primary method in most studies. While surface electromyography-based hand motion decoding shows promise, it has not yet met the requirements for reliable use. Combining different sensing modalities has been shown to improve hand gesture classification accuracy. This work introduces a multimodal bracelet that integrates a 24-channel force myography system with six commercial surface electromyography sensors, each containing a six-axis inertial measurement unit. The device's functionality was tested by acquiring muscular activity with the proposed device from five participants performing five different gestures in a random order. A random forest model was then used to classify the performed gestures from the acquired signal. The results confirmed the device's functionality, making it suitable to study sensor fusion for intent detection in future studies. The results showed that combining all modalities yielded the highest classification accuracies across all participants, reaching 92.3±2.6% on average, effectively reducing misclassifications by 37% and 22% compared to using surface electromyography and force myography individually as input signals, respectively. This demonstrates the potential benefits of sensor fusion for more robust and accurate hand gesture classification and paves the way for advanced control of robotic and prosthetic hands.

Large measurement range curvature sensor based on all-fiber Mach-Zehnder interferometer using triple-ring-core few-mode fiber.

This study experimentally demonstrates a large measurement range curvature sensor based on a Mach-Zehnder interferometer (MZI) in a triple-ring-core fiber (TRCF). The sensor is fabricated by fusion splicing a segment of TRCF between two pieces of single-mode fiber (SMF), forming the SMF-TRCF-SMF sandwich structure. Since the TRCF can support the propagation of a few guided modes, the fundamental mode interferes with high-order modes in the sensing part to produce a periodic interference spectrum. Bending causes the effective refractive index of the mode to change, which shifts the interference spectrum. The bending measurement is achieved by monitoring the wavelength variation of interference dips. The proposed sensors can realize curvature measurements up to 100 m-1, with a bending sensitivity of -225.9 pm/m-1. It is believed that the proposed MZI sensor may have potential applications in robot hand gesture recognition and control.

Decomposing Task-Relevant Information From Surface Electromyogram for User-Generic Dexterous Finger Force Decoding.

Existing electromyographic (EMG) based motor intent detection algorithms are typically user-specific, and a generic model that can quickly adapt to new users is highly desirable. However, establishing such a model remains a challenge due to high inter-person variability and external interference with EMG signals. In this study, we present a feature disentanglement approach, implemented by an autoencoder-like architecture, designed to decompose user-invariant, motor-task-sensitive high-level representations from user-sensitive, task-irrelevant representations in EMG amplitude features. Our method is user-generic and can be applied to unseen users for continuous multi-finger force predictions. We evaluated our approach on eight subjects, predicting the force of three fingers (index, middle, and ring-pinky) concurrently. We assessed the decoder's performance through a rigorous leave-one-subject-out validation. Our developed approach consistently outperformed both the conventional EMG amplitude method and a commonly used feature projection approach, principal component analysis (PCA), with a lower force prediction error (RMSE: 6.91 ± 0.45 % MVC; R2: 0.835 ± 0.026) and a higher finger classification accuracy (83.0 ± 4.5%). The comparison with the state-of-the-art neural networks further demonstrated the superior performance of our method in user-generic force predictions. Overall, our methods provide novel insights into the development of user-generic and accurate neural decoding for myoelectric control of assistive robotic hands.

Towards Efficient Neural Decoder for Dexterous Finger Force Predictions.

Dexterous control of robot hands requires a robust neural-machine interface capable of accurately decoding multiple finger movements. Existing studies primarily focus on single-finger movement or rely heavily on multi-finger data for decoder training, which requires large datasets and high computation demand. In this study, we investigated the feasibility of using limited single-finger surface electromyogram (sEMG) data to train a neural decoder capable of predicting the forces of unseen multi-finger combinations. We developed a deep forest-based neural decoder to concurrently predict the extension and flexion forces of three fingers (index, middle, and ring-pinky). We trained the model using varying amounts of high-density EMG data in a limited condition (i.e., single-finger data). We showed that the deep forest decoder could achieve consistently commendable performance with 7.0% of force prediction errors and R2 value of 0.874, significantly surpassing the conventional EMG amplitude method and convolutional neural network approach. However, the deep forest decoder accuracy degraded when a smaller amount of data was used for training and when the testing data became noisy. The deep forest decoder shows accurate performance in multi-finger force prediction tasks. The efficiency aspect of the deep forest lies in the short training time and small volume of training data, which are two critical factors in current neural decoding applications. This study offers insights into efficient and accurate neural decoder training for advanced robotic hand control, which has the potential for real-life applications during human-machine interactions.

Towards humanlike grasp in robotic hands: mechanical implementation of force synergies

In the field of robotic hands, finger force coordination is usually achieved by complex mechanical structures and control systems. This study presents the design of a novel transmission system inspired from the physiological concept of force synergies, aiming to simplify the control of multifingered robotic hands. To this end, we collected human finger force data during six isometric grasping tasks, and force synergies (i.e. the synergy weightings and the corresponding activation coefficients) were extracted from the concatenated force data to explore their potential for force modulation. We then implemented two force synergies with a cable-driven transmission mechanism consisting of two spring-loaded sliders and five V-shaped bars. Specifically, we used fixed synergy weightings to determine the stiffness of the compression springs, and the displacements of sliders were determined by time-varying activation coefficients. The derived transmission system was then used to drive a five-finger robotic hand named SYN hand. We also designed a motion encoder to selectively activate desired fingers, making it possible for two motors to empower a variety of hand postures. Experiments on the prototype demonstrate successful grasp of a wide range of objects in everyday life, and the finger force distribution of SYN hand can approximate that of human hand during six typical tasks. To our best knowledge, this study shows the first attempt to mechanically implement force synergies for finger force modulation in a robotic hand. In comparison to state-of-the-art robotic hands with similar functionality, the proposed hand can distribute humanlike force ratios on the fingers by simple position control, rather than resorting to additional force sensors or complex control strategies. The outcome of this study may provide alternatives for the design of novel anthropomorphic robotic hands, and thus show application prospects in the field of hand prostheses and exoskeletons.

A rotary transformer cross-subject model for continuous estimation of finger joints kinematics and a transfer learning approach for new subjects.

Surface Electromyographic (sEMG) signals are widely utilized for estimating finger kinematics continuously in human-machine interfaces (HMI), and deep learning approaches are crucial in constructing the models. At present, most models are extracted on specific subjects and do not have cross-subject generalizability. Considering the erratic nature of sEMG signals, a model trained on a specific subject cannot be directly applied to other subjects. Therefore, in this study, we proposed a cross-subject model based on the Rotary Transformer (RoFormer) to extract features of multiple subjects for continuous estimation kinematics and extend it to new subjects by adversarial transfer learning (ATL) approach. We utilized the new subject's training data and an ATL approach to calibrate the cross-subject model. To improve the performance of the classic transformer network, we compare the impact of different position embeddings on model performance, including learnable absolute position embedding, Sinusoidal absolute position embedding, and Rotary Position Embedding (RoPE), and eventually selected RoPE. We conducted experiments on 10 randomly selected subjects from the NinaproDB2 dataset, using Pearson correlation coefficient (CC), normalized root mean square error (NRMSE), and coefficient of determination (R2) as performance metrics. The proposed model was compared with four other models including LSTM, TCN, Transformer, and CNN-Attention. The results demonstrated that both in cross-subject and subject-specific cases the performance of RoFormer was significantly better than the other four models. Additionally, the ATL approach improves the generalization performance of the cross-subject model better than the fine-tuning (FT) transfer learning approach. The findings indicate that the proposed RoFormer-based method with an ATL approach has the potential for practical applications in robot hand control and other HMI settings. The model's superior performance suggests its suitability for continuous estimation of finger kinematics across different subjects, addressing the limitations of subject-specific models.

Motion intention recognition of the affected hand based on the sEMG and improved DenseNet network

The key to sEMG (surface electromyography)-based control of robotic hands is the utilization of sEMG signals from the affected hand of amputees to infer their motion intentions. With the advancements in deep learning, researchers have successfully developed viable solutions for CNN (Convolutional Neural Network)-based gesture recognition. However, most studies have primarily concentrated on utilizing sEMG data from the hands of healthy subjects, often relying on high-dimensional feature vectors obtained from a substantial number of electrodes. This approach has yielded high-performing sEMG recognition systems but has failed to consider the considerable inconvenience that the abundance of electrodes poses to the daily lives and work of patients. In this paper, we focused on transradial amputees and used sEMG data from the Ninapro DB3 database as our dataset. Firstly, we introduce a STFT (Short-Time Fourier Transform)-based time-frequency feature fusion map for sEMG. This map includes both time-frequency features and the time-frequency localization of sEMG signals. Secondly, we propose an Improved DenseNet (Dense Convolutional Network) model for recognizing motion intentions in the affected hand of amputees based on their sEMG signals. Finally, addressing the issue of optimizing the number of electrodes carried by amputees, we introduce the PCMIRR (Pearson Correlation and Motion Intention Recognition Rate) algorithm. This algorithm optimizes the number of channels by considering the Pearson correlation between the sEMG channels of amputees and the recognition rate of motion intentions in the affected hand based on single-channel sEMG data. The experimental results reveal that the recognition accuracy, recall, and F1 score achieved by the Improved DenseNet model were 93.82%, 93.61%, and 93.65%, respectively. When the number of electrodes was optimized to 8, the recognition accuracy reached 94.50%. In summary, this paper ultimately attained precise recognition of motion intentions in amputees' affected hands while utilizing the minimum number of sEMG channels. This method offers a novel approach to sEMG-based control of bionic robotic hands.

Low hysteresis, water retention, anti-freeze multifunctional hydrogel strain sensor for human-machine interfacing and real-time sign language translation.

Hydrogel strain sensors have received increasing attention due to their potential applications in human-machine interfaces and flexible electronics. However, they usually suffer from both mechanical and electrical hysteresis and poor water retention, which limit their practical applications. To address this challenge, a poly(acrylic acid-co-acrylamide) hydrogel crosslinked by silica nanoparticles is fabricated via photo polymerization and salting-out of hydrophilic ions for the strain sensor. The resulting hydrogel strain sensor possessed low electrical hysteresis (1.6%), low mechanical hysteresis (<7%), high cycle stability (>10 000 cycles), high durability, water retention and anti-freezing ability. Moreover, this strain sensor can be used as a wearable sensor for real-time control of robotic hands and hand gesture recognition. Finally, a sign language translation system has been demonstrated with the aid of machine learning, achieving recognition rates of over 98% for 15 different sign languages. This work offers a promising prospect for human-machine interfaces, smart wearable devices, and the Internet of Things.

GRASPING FORCE ESTIMATION FROM TWO-CHANNEL ELECTROMYOGRAPHY SIGNALS USING EXTREME LEARNING MACHINE

This research presents a method to select the best parameters of an extreme learning machine (ELM) for estimating grasping force from two-channel surface electromyography (sEMG) signals. Its advantages compared to the use of multi-channel sEMG signals include faster computing, lower electrode costs, and easier usage. The proposed method is appropriate for certain applications where time is very important but accuracy can be slightly compromised. The study recorded sEMG signals from 20 healthy volunteers, 10 males and 10 females, aged 22-52 years. The recorded sEMG signals were used in testing the proposed optimization method for grasping force estimation. It was found that the proposed method for optimizing the number of nodes, gain, and factor values would make the force estimation more efficient. The results show that the optimum number of nodes, gain, and factor values is 4, 0.05, and 1.0045, respectively. The resulting root mean square error and correlation coefficient values in force estimation from the ELM optimization method were 2.5642 and 0.9287, respectively, when the computing time was only 0.0038 s. These results show the feasibility of the proposed method for estimating force using only two-channel sEMG signals. As a result, real-time implementation, such as force estimation in robotic hand control for rehabilitation using sEMG signals, can be achieved efficiently.

Dexterous manipulation of construction tools using anthropomorphic robotic hand

Emerging studies are utilizing reinforcement learning (RL) and imitation learning (IL) to control large-scale robots in heavy construction tasks. There is limited attention given to the automation of delicate tasks typically performed manually. This paper proposes the control of an anthropomorphic robotic hand with a high degree of freedom for the manipulation of construction tools in a learning-based approach. For controlling the robotic hand, a simulation-based policy learning framework based on pretraining policies through IL is presented, subsequently fine-tuning them for construction tool manipulation using RL. In experimental trials, six policies are trained for the robotic hand to grasp six different construction tools. The results indicate that each of the learned policies enables the robotic hand to manipulate the corresponding tool with an almost 100% success rate, demonstrating resilience when confronted with tools of different scales. Additionally, the paper showcases the potential for scaling up the fundamental policy for downstream applications.

Highly Stable Ladder‐Type Conjugated Polymer Based Organic Electrochemical Transistors for Low Power and Signal Processing‐Free Surface Electromyogram Triggered Robotic Hand Control

AbstractOrganic electrochemical transistors (OECTs) based complementary inverters have been considered as promising candidates in electrophysiological amplification, owing to their low power consumption, and high gain. To create complementary inverters, it is important to use highly stable p‐type and n‐type polymers with well‐balanced current. In this study, the electrochemical stability of p‐type ladder‐conjugated polymer‐based OECT is improved through an annealing process that maintains its doped‐state drain current from 76% to 105% after 4,500 cycles in ambient environment. Next an OECT‐based complementary inverter made from p‐type and n‐type ladder‐conjugated polymers (PBBTL and BBL) that possess ultra‐low power consumption (≈170 nW), high gain (67 V/V), and high noise margin (92%) with full rail‐to‐rail swing, is presented. Furthermore, its potential for amplifying the envelope of surface electromyography (EMG) for robotic hand control is demonstrated. The high variation in the output (0.35 V) allows the amplified EMG signals to be directly captured by a commercial analog‐to‐digital converter, which in turn controls the robot hand to grasp different objects with low delay and low noise. These results demonstrate the capability of OECT inverter‐based amplifier in future signal processing‐free human‐machine interface, particularly useful for prosthetic control and gesture control applications.

Multi-Grasp Classification for the Control of Robot Hands Employing Transformers and Lightmyography Signals.

The increasing use of smart technical devices in our everyday lives has necessitated the use of muscle-machine interfaces (MuMI) that are intuitive and that can facilitate immersive interactions with these devices. The most common method to develop MuMIs is using Electromyography (EMG) based signals. However, due to several drawbacks of EMG-based interfaces, alternative methods to develop MuMI are being explored. In our previous work, we presented a new MuMI called Lightmyography (LMG), which achieved outstanding results compared to a classic EMG-based interface in a five-gesture classification task. In this study, we extend our previous work experimentally validating the efficiency of the LMG armband in classifying thirty-two different gestures from six participants using a deep learning technique called Temporal Multi-Channel Vision Transformers (TMC-ViT). The efficiency of the proposed model was assessed using accuracy. Moreover, two different undersampling techniques are compared. The proposed thirty-two-gesture classifiers achieve accuracies as high as 92%. Finally, we employ the LMG interface in the real-time control of a robotic hand using ten different gestures, successfully reproducing several grasp types from taxonomy grasps presented in the literature.

Hand Grasp Pose Prediction Based on Motion Prior Field.

Shared control of bionic robot hands has recently attracted much research attention. However, few studies have performed predictive analysis for grasp pose, which is vital for the pre-shape planning of robotic wrists and hands. Aiming at shared control of dexterous hand grasp planning, this paper proposes a framework for grasp pose prediction based on the motion prior field. To map the hand-object pose to the final grasp pose, an object-centered motion prior field is established to learn the prediction model. The results of motion capture reconstruction show that, with the input of a 7-dimensional pose and cluster manifolds of dimension 100, the model performs best in terms of prediction accuracy (90.2%) and error distance (1.27 cm) in the sequence. The model makes correct predictions in the first 50% of the sequence during hand approach to the object. The outcomes of this study enable prediction of the grasp pose in advance as the hand approaches the object, which is very important for enabling the shared control of bionic and prosthetic hands.

Robust neural decoding for dexterous control of robotic hand kinematics

BackgroundManual dexterity is a fundamental motor skill that allows us to perform complex daily tasks. Neuromuscular injuries, however, can lead to the loss of hand dexterity. Although numerous advanced assistive robotic hands have been developed, we still lack dexterous and continuous control of multiple degrees of freedom in real-time. In this study, we developed an efficient and robust neural decoding approach that can continuously decode intended finger dynamic movements for real-time control of a prosthetic hand. MethodsHigh-density electromyogram (HD-EMG) signals were obtained from the extrinsic finger flexor and extensor muscles, while participants performed either single-finger or multi-finger flexion-extension movements. We implemented a deep learning-based neural network approach to learn the mapping from HD-EMG features to finger-specific population motoneuron firing frequency (i.e., neural-drive signals). The neural-drive signals reflected motor commands specific to individual fingers. The predicted neural-drive signals were then used to continuously control the fingers (index, middle, and ring) of a prosthetic hand in real-time. ResultsOur developed neural-drive decoder could consistently and accurately predict joint angles with significantly lower prediction errors across single-finger and multi-finger tasks, compared with a deep learning model directly trained on finger force signals and the conventional EMG-amplitude estimate. The decoder performance was stable over time and was robust to variations of the EMG signals. The decoder also demonstrated a substantially better finger separation with minimal predicted error of joint angle in the unintended fingers. ConclusionsThis neural decoding technique offers a novel and efficient neural-machine interface that can consistently predict robotic finger kinematics with high accuracy, which can enable dexterous control of assistive robotic hands.

A Transfer Learning based Cross-subject Generic Model for Continuous Estimation of Finger Joint Angles from a New User.

Continuous estimation of finger joints based on surface electromyography (sEMG) has attracted much attention in the field of human-machine interface (HMI). A couple of deep learning models were proposed to estimate the finger joint angles for specific subject. When applied onto a new subject, however, the performance of the subject-specific model would degrade significantly due to the inter-subject differences. Therefore, a novel cross-subject generic (CSG) model was proposed in this study to estimate continuous kinematics of finger joints for new users. Firstly, a multi-subject model based on the LSTA-Conv network was built by using sEMG and finger joint angles data from multiple subjects. Then, the subjects adversarial knowledge (SAK) transfer learning strategy was adopted to calibrate the multi-subject model with the training data from a new user. With the updated model parameters and the testing data from the new user, multiple finger joint angles could be estimated afterwards. The overall performance of the CSG model for new users was validated on three public datasets from Ninapro. The results showed that the newly proposed CSG model significantly outperformed five subject-specific models and two transfer learning models in terms of Pearson correlation coefficient, root mean square error, and coefficient of determination. Comparison analysis showed that both the long short-term feature aggregation (LSTA) module and the SAK transfer learning strategy contributed to the CSG model. Moreover, increasing number of subjects in training set improved the generalization capability of the CSG model. The novel CSG model would facilitate the application of robotic hand control and other HMI settings.

Suppression of Packet Losses and Disturbances in Networked Control Systems Based on Equivalent-Input-Disturbance Approach.

This article presents a method of suppressing packet losses and exogenous disturbances for a networked control system (NCS) subject to network-introduced delays. The NCS has two feedback loops: 1) a local one and 2) a main one. The local feedback loop contains a state observer, an equivalent-input-disturbance (EID) estimator, and state feedback. It is used to ensure prompt disturbance suppression. The controller in the main feedback loop contains an internal model to track a reference input. The system is divided into two subsystems for the design of controllers. The state-observer gain is designed for one subsystem using the concept of perfect regulation to ensure disturbance estimation performance. The state-feedback gains of the other subsystem are designed based on a stability condition in the form of a linear matrix inequality (LMI). A tracking specification is embedded in the LMI-based stability condition to ensure satisfactory tracking performance. A case study on a two-finger robot hand control system and a comparison with a Smith-EID and H∞ controller approach validate the effectiveness and superiority of the presented method.

Ultrastretchable, Self-Healing Conductive Hydrogel-Based Triboelectric Nanogenerators for Human–Computer Interaction

The rapid development of wearable electronic devices and virtual reality technology has revived interest in flexible sensing and control devices. Here, we report an ionic hydrogel (PTSM) prepared from polypropylene amine (PAM), tannic acid (TA), sodium alginate (SA), and MXene. Based on the multiple weak H-bonds, this hydrogel exhibits excellent stretchability (strain >4600%), adhesion, and self-healing. The introduction of MXene nanosheets endows the hydrogel sensor with a high gauge factor (GF) of 6.6. Meanwhile, it also enables triboelectric nanogenerators (PTSM-TENGs) fabricated from silicone rubber-encapsulated hydrogels to have excellent energy harvesting efficiency, with an instantaneous output power density of 54.24 mW/m2. We build a glove-based human-computer interaction (HMI) system using PTSM-TENGs. The multidimensional signal features of PTSM-TENG are extracted and analyzed by the HMI system, and the functions of gesture visualization and robot hand control are realized. In addition, triboelectric signals can be used for object recognition with the help of machine learning techniques. The glove based on PTSM-TENG achieves the classification and recognition of five objects through contact, with an accuracy rate of 98.7%. Therefore, strain sensors and triboelectric nanogenerators based on hydrogels have broad application prospects in man-machine interface, intelligent recognition systems, auxiliary control systems, and other fields due to their excellent stretchable and high self-healing performance.

Integrated Design Fabrication and Control of a Bioinspired Multimaterial Soft Robotic Hand.

Machines that mimic humans have inspired scientists for centuries. Bioinspired soft robotic hands are a good example of such an endeavor, featuring intrinsic material compliance and continuous motion to deal with uncertainty and adapt to unstructured environments. Recent research led to impactful achievements in functional designs, modeling, fabrication, and control of soft robots. Nevertheless, the full realization of life-like movements is still challenging to achieve, often based on trial-and-error considerations from design to fabrication, consuming time and resources. In this study, a soft robotic hand is proposed, composed of soft actuator cores and an exoskeleton, featuring a multimaterial design aided by finite element analysis (FEA) to define the hand geometry and promote finger's bendability. The actuators are fabricated using molding, and the exoskeleton is 3D-printed in a single step. An ON-OFF controller keeps the set fingers' inner pressures related to specific bending angles, even in the presence of leaks. The FEA numerical results were validated by experimental tests, as well as the ability of the hand to grasp objects with different shapes, weights, and sizes. This integrated solution will make soft robotic hands more available to people, at a reduced cost, avoiding the time-consuming design-fabrication trial-and-error processes.

Slave Robotic Hand Control by Master Glove with Haptic Feedback

Slave Robotic Hand Control by Master Glove with Haptic Feedback

Improving robotic hand control via adaptive Fuzzy-PI controller using classification of EMG signals

Robotic or prosthetic organs are designed to have the maximum similarity to human organs. This paper aims to improve robotic hand control via an adaptive Fuzzy-PI controller using EMG signals. The data is collected from the FDS and FPL muscles of the forearm of five individuals who performed eight movements. Then, appropriate filters are used to eliminate the noise of the signals, and MAV, VAR, and SE features are extracted. Based on MAV and VAR, classification is carried out using DA, KNN, and SVM. With an average accuracy, specificity, and sensitivity of 90.69%, 94.64%, and 62.10%, SVM is a better choice for movement detection. Following the movement detection by SVM, an appropriate reference signal is sent to the controller. The reference signal is the angle change of the fingers during the movement. All the eight gestures are modeled in a new way through these angles. The adaptive fuzzy-PI controller is used to control a robotic hand model with fifteen degrees of freedom. It has the advantages of learning from human experiences and adapting to environmental changes. The performance of the controller is evaluated in two ways. One is the comparison of the fuzzy-PI with the PI by RMSE. The average RMSE for eight movements using the fuzzy-PI is 1.6067, and for the PI, 5.0082. These results show that the fuzzy-PI controller performs better than the PI. Another new evaluation way presented in this paper is comparing the EMG signal features with the robotic hand movement signal features in terms of RMSE. The small RMSE values indicate that the EMG signal and robotic hand movement data features are significantly similar. Therefore, it can be concluded that the robotic hand controlled by the proposed controller is notably identical to the human hand.