Dexterous Control Research Articles

Reorganization of functional and directed corticomuscular connectivity during precision grip from childhood to adulthood

How does the neural control of fine movements develop from childhood to adulthood? Here, we investigated developmental differences in functional corticomuscular connectivity using coherence analyses in 111 individuals from four different age groups covering the age range 8–30 y. EEG and EMG were recorded while participants performed a uni-manual force-tracing task requiring fine control of force in a precision grip with both the dominant and non-dominant hand. Using beamforming methods, we located and reconstructed source activity from EEG data displaying peak coherence with the EMG activity of an intrinsic hand muscle during the task. Coherent cortical sources were found anterior and posterior to the central sulcus in the contralateral hemisphere. Undirected and directed corticomuscular coherence was quantified and compared between age groups. Our results revealed that coherence was greater in adults (20–30 yo) than in children (8–10 yo) and that this difference was driven by greater magnitudes of descending (cortex-to-muscle), rather than ascending (muscle-to-cortex), coherence. We speculate that the age-related differences reflect maturation of corticomuscular networks leading to increased functional connectivity with age. We interpret the greater magnitude of descending oscillatory coupling as reflecting a greater degree of feedforward control in adults compared to children. The findings provide a detailed characterization of differences in functional sensorimotor connectivity for individuals at different stages of typical ontogenetic development that may be related to the maturational refinement of dexterous motor control.

A Recurrent Neural Network Provides Stable Across-Day Prosthetic Control for a Human Amputee with Implanted Intramuscular Electromyographic Recording Leads.

Upper-limb prosthetic control is often challenging and non-intuitive, leading to up to 50% of prostheses users abandoning their prostheses. Convolutional neural networks (CNN) and recurrent long short-term memory (LSTM) networks have shown promise in extracting high-degree-of-freedom motor intent from myoelectric signals, thereby providing more intuitive and dexterous prosthetic control. An important next consideration for these algorithms is if performance remains stable over multiple days. Here we introduce a new LSTM network and compare its performance to previously established state-of-the-art algorithms-a CNN and a modified Kalman filter (MKF)-in offline analyses using 76 days of intramuscular recordings from one amputee participant collected over 425 calendar days. Specifically, we assessed the robustness of each algorithm over time by training on data from the first (one, five, ten, 30, or 60) days and then testing on myoelectric signals on the last 16 days. Results indicate that training on additional datasets from prior days generally decreases the Root Mean Squared Error (RMSE) of intended and unintended movements for all algorithms. Across all algorithms trained with 60 days of data, the lowest RMSE for unintended movements was achieved with the LSTM. The LSTM also showed less across-day variance in RMSE of unintended movements relative to the other algorithms. Altogether this work suggests that the LSTM algorithm introduced here can provide more intuitive and dexterous control for prosthetic users, and that training on multiple days of data improves overall performance on subsequent days, at least for offline analyses.

A Bionic Hand for Semi-Autonomous Fragile Object Manipulation via Proximity and Pressure Sensors.

Multiarticulate bionic hands are now capable of recreating the endogenous movements and grip patterns of the human hand, yet amputees continue to be dissatisfied with existing control strategies. One approach towards more dexterous and intuitive control is to create a semi-autonomous bionic hand that can synergistically aid a human with complex tasks. To that end, we have developed a bionic hand that can automatically detect and grasp nearby objects with minimal force using multi-modal fingertip sensors. We evaluated performance using a fragile-object task in which participants must move an object over a barrier without applying pressure above specified thresholds. Participants completed the task under three conditions: 1) with their native hand, 2) with the bionic hand using surface electromyography control, and 3) using the semi-autonomous bionic hand. We show that the semi-autonomous hand is extremely capable of completing this dexterous task and significantly outperforms a more traditional surface-electromyography controller. Furthermore, we show that the semi-autonomous bionic hand significantly increased users' grip precision and reduced users' perceived task workload. This work constitutes an important step towards more dexterous and intuitive bionic hands and serves as a foundation for future work on shared human-machine control for intelligent bionic systems.

Intrafascicular peripheral nerve stimulation produces fine functional hand movements in primates.

Restoring dexterous hand control is critical for people with paralysis. Approaches based on surface or intramuscular stimulation provide limited finger control, generate insufficient force to recover functional movements, and require numerous electrodes. Here, we show that intrafascicular peripheral electrodes could produce functional grasps and sustained forces in three monkeys. We designed an intrafascicular implantable electrode targeting the motor fibers of the median and radial nerves. Our interface selectively and reliably activated extrinsic and intrinsic hand muscles, generating multiple functional grips, hand opening, and sustained contraction forces for up to 2 months. We extended those results to a behaving monkey with transient hand paralysis and used intracortical signals to control simple stimulation protocols that enabled this animal to perform a functional grasping task. Our findings show that just two intrafascicular electrodes can generate a rich portfolio of dexterous and functional hand movements with important implications for clinical applicability.

Dynamic Gesture Recognition Using Surface EMG Signals Based on Multi-Stream Residual Network.

Gesture recognition technology is widely used in the flexible and precise control of manipulators in the assisted medical field. Our MResLSTM algorithm can effectively perform dynamic gesture recognition. The result of surface EMG signal decoding is applied to the controller, which can improve the fluency of artificial hand control. Much current gesture recognition research using sEMG has focused on static gestures. In addition, the accuracy of recognition depends on the extraction and selection of features. However, Static gesture research cannot meet the requirements of natural human-computer interaction and dexterous control of manipulators. Therefore, a multi-stream residual network (MResLSTM) is proposed for dynamic hand movement recognition. This study aims to improve the accuracy and stability of dynamic gesture recognition. Simultaneously, it can also advance the research on the smooth control of the Manipulator. We combine the residual model and the convolutional short-term memory model into a unified framework. The architecture extracts spatiotemporal features from two aspects: global and deep, and combines feature fusion to retain essential information. The strategy of pointwise group convolution and channel shuffle is used to reduce the number of network calculations. A dataset is constructed containing six dynamic gestures for model training. The experimental results show that on the same recognition model, the gesture recognition effect of fusion of sEMG signal and acceleration signal is better than that of only using sEMG signal. The proposed approach obtains competitive performance on our dataset with the recognition accuracies of 93.52%, achieving state-of-the-art performance with 89.65% precision on the Ninapro DB1 dataset. Our bionic calculation method is applied to the controller, which can realize the continuity of human-computer interaction and the flexibility of manipulator control.

A Meta-analysis of the Relationship between Motor Skills and Executive Functions in Typically-developing Children

ABSTRACT The relationship between motor skills and executive functions (EFs) is gaining prominence in the field of developmental psychology. However, evidence of the motor skills–EFs link in children with typical development is somewhat inconsistent and there has been no adequate attempt to evaluate it systematically across studies. In view of this, the present meta-analysis was carried out to investigate the relationship between the global domains of motor skills and EFs; and to explore specific associations between different components of motor skills (balance, manual dexterity, locomotor skills, and object control skills) and EFs (response inhibition, working memory, and cognitive flexibility) in typically developing children. The analysis involved data on 4,866 children between the ages of 3 and 12 years, taken from 32 studies. The results revealed a significant positive association between motor skills and EFs at the global level, as well as at the specific level of analysis. At the specific level of analysis, balance and manual dexterity were found to have the strongest independent associations with all EF components. Moderator analysis revealed an age effect between balance and response inhibition only. In summary, the present meta-analysis provides evidence for the theoretical assumption of a link between motor skills and EFs and emphasizes the importance of including cognitively engaging motor tasks in intervention programs designed to promote motor skills and higher-order cognitive skills in children.

Towards Multi-Finger Dexterous Hand Mechanics Control and Tactile Feedback

Abstract：The ever-changing demands of industrial automation and space technology have promoted the rapid development of robotics. Traditional robotic end effectors are difficult to perform smart operations, so there is an urgent need for a robotic hand to perform complex operations instead of humans. In this article, we will focus our attention on mechanical control and haptic feedback. Mechanical control and haptic feedback are necessary conditions for the stable and accurate grasping of multi-finger dexterous hands. Tactile perception can provide stiffness and temperature to multi-finger dexterous hands. Important information makes the function of the dexterous hand more perfect. This article introduces the kinematics and dynamics of dexterous hand fingers, as well as the kinematics and dynamics solving equations, then reviews the current sensors and various control driving methods used in dexterous hands, discusses drive control, and compares each method Pros and cons. Finally, the future development of dexterous hands is predicted.

A portable, self-contained neuroprosthetic hand with deep learning-based finger control

Objective. Deep learning-based neural decoders have emerged as the prominent approach to enable dexterous and intuitive control of neuroprosthetic hands. Yet few studies have materialized the use of deep learning in clinical settings due to its high computational requirements. Approach. Recent advancements of edge computing devices bring the potential to alleviate this problem. Here we present the implementation of a neuroprosthetic hand with embedded deep learning-based control. The neural decoder is designed based on the recurrent neural network architecture and deployed on the NVIDIA Jetson Nano—a compacted yet powerful edge computing platform for deep learning inference. This enables the implementation of the neuroprosthetic hand as a portable and self-contained unit with real-time control of individual finger movements. Main results. A pilot study with a transradial amputee is conducted to evaluate the proposed system using peripheral nerve signals acquired from implanted intrafascicular microelectrodes. The preliminary experiment results show the system’s capabilities of providing robust, high-accuracy (95%–99%) and low-latency (50–120 ms) control of individual finger movements in various laboratory and real-world environments. Conclusion. This work is a technological demonstration of modern edge computing platforms to enable the effective use of deep learning-based neural decoders for neuroprosthesis control as an autonomous system. Significance. The proposed system helps pioneer the deployment of deep neural networks in clinical applications underlying a new class of wearable biomedical devices with embedded artificial intelligence.Clinical trial registration: DExterous Hand Control Through Fascicular Targeting (DEFT). Identifier: NCT02994160.

Assessment of the training program for Versius, a new innovative robotic system for use in minimal access surgery

ObjectivesThe Versius surgical system has been developed for use in robot-assisted minimal access surgery (MAS). This study aimed to evaluate the effectiveness of the Versius training program.DesignA 3.5-day program following...

Joint Modeling and Closed-Loop Control of a Robotic Hand Driven by the Tendon-Sheath

Robotic dexterous operation is a long-term challenging problem. Although recent advances in robotic hand have successfully realized some complex operations, its high dexterity and high precision control are still challenges for robotic hands to replace the human hand. Here, this letter developed a tendon-sheath transmission-based robotic hand with decoupling joint design and high-precision closed-loop control strategy. Firstly, the designed hand is introduced, and the nonlinear joint transmission model is established. Then, the transfer function of the joint is established based on the dynamic model, and an adaptive controller with neural network compensation is designed to realize the high-precision joint control. The simulation and experiment results show that it is feasible to apply the tendon-sheath transmission method to the robotic hand, and the proposed closed-loop control strategy can significantly improve the joint control accuracy. This research opens new prospects for the novel tendon-sheath driving method applied in the design of robotic hand, which is expected to have a long-term impact for the design and control of dexterous hand.

Adaptation of the Corticomuscular and Biomechanical Systems of Pianists.

Independent control of movements between the fingers plays a role in hand dexterity characterizing skilled individuals. However, it remains unknown whether and in what manner neuromuscular and biomechanical constraints on the movement independence of the fingers depend on motor expertise. Here, we compared motor dexterity, corticospinal excitability of multiple muscles, muscular activation, and anatomical features of the fingers between the pianists and nonpianists. When the ring finger was passively moved by a robot, passive motions produced at the adjacent fingers were smaller for the pianists than the nonpianists, indicating reduced biomechanical constraint of fingers in the pianists. In contrast, when the ring finger moved actively, we found no group difference in passive motions produced at the adjacent fingers; however, reduced inhibition of corticospinal excitability of the adjacent fingers in the pianists compared with the nonpianists. This suggests strengthened neuromuscular coupling between the fingers of the pianists, enhancing the production of coordinated finger movements. These group differences were not evident during the index and little finger movements. Together, pianists show expertise-dependent biomechanical and neurophysiological adaptations, specifically at the finger with innately low movement independence. Such contrasting adaptations of pianists may subserve dexterous control of both the individuated and coordinated finger movements.

REVIEW ON DEEP LEARNING APPROACHES TO THE CONTROL OF PROSTHETIC HANDS WITH ELECTROMYOGRAPHY SIGNALS

Natural control methods based on surface electromyography (sEMG) and pattern recognition are promising for hand prosthetics. Several efforts have been carried out to enhance dexterous hand prosthesis control by impaired individuals. However, the control robustness offered by scientic research is still not sufcient for many real life applications, and commercial prostheses are capable of offering natural control for only a few movements. This paper reviews various papers on deep learning approaches to the control of prosthetic hands with EMG signals and made a comparison on their accuracy.

Artificial Perception and Semiautonomous Control in Myoelectric Hand Prostheses Increases Performance and Decreases Effort

Dexterous control of upper limb prostheses with multiarticulated wrists/hands is still a challenge due to the limitations of myoelectric man-machine interfaces. Multiple factors limit the overall performance and usability of these interfaces, such as the need to control degrees of freedom sequentially and not concurrently, and the inaccuracies in decoding the user intent from weak or fatigued muscles. In this article, we developed a novel man-machine interface that endows a myoelectric prosthesis (MYO) with artificial perception, estimation of user intention, and intelligent control (MYO-PACE) to continuously support the user with automation while preparing the prosthesis for grasping. We compared the MYO-PACE against state-of-the-art myoelectric control (pattern recognition) in laboratory and clinical tests. For this purpose, eight able-bodied and two amputee individuals performed a standard clinical test consisting of a series of manipulation tasks (portion of the SHAP test), as well as a more complex sequence of transfer tasks in a cluttered scene. In all tests, the subjects not only completed the trials faster using the MYO-PACE but also achieved more efficient myoelectric control. These results demonstrate that the implementation of advanced perception, context interpretation, and autonomous decision-making into active prostheses improves control dexterity. Moreover, it also effectively supports the user by speeding up the preshaping phase of the movement and decreasing muscle use.

Cortical signatures of precision grip force control in children, adolescents, and adults.

Human dexterous motor control improves from childhood to adulthood, but little is known about the changes in cortico-cortical communication that support such ontogenetic refinement of motor skills. To investigate age-related differences in connectivity between cortical regions involved in dexterous control, we analyzed electroencephalographic data from 88 individuals (range 8-30 years) performing a visually guided precision grip task using dynamic causal modelling and parametric empirical Bayes. Our results demonstrate that bidirectional coupling in a canonical 'grasping network' is associated with precision grip performance across age groups. We further demonstrate greater backward coupling from higher-order to lower-order sensorimotor regions from late adolescence in addition to differential associations between connectivity strength in a premotor-prefrontal network and motor performance for different age groups. We interpret these findings as reflecting greater use of top-down and executive control processes with development. These results expand our understanding of the cortical mechanisms that support dexterous abilities through development.

Perception of Static Position and Kinesthesia of the Finger using Vibratory Stimulation.

Proprioception provides information regarding the state of an individual's limb in terms of static position and kinesthesia (dynamic movement). When such feedback is lost or impaired, the performance of dexterous control of our biological limbs or assistive devices tends to deteriorate. In this study, we determined if external vibratory stimulation patterns could allow for the perception of a finger's static position and kinesthesia. Using four tactors and two stimulus levels, eight vibratory settings corresponded to eight discrete finger positions. The transition patterns between these eight settings corresponded to kinesthesia. Three experimental blocks assessed the perception of a finger's static position, speed, and movement (amplitude and direction). Our results demonstrated that both position and kinesthesia could be recognized with over 93% accuracy. The outcomes suggest that vibratory stimulus can inform subjects of static and dynamic aspects of finger proprioception. This sensory stimulation approach can be implemented to improve outcomes in clinical populations with sensory deficits, and to enhance user experience when users interact with assistive devices.

Transcutaneous Spinal Cord Stimulation Enhances Walking Performance and Reduces Spasticity in Individuals with Multiple Sclerosis.

Gait dysfunction and spasticity are common debilitating consequences of multiple sclerosis (MS). Improvements of these motor impairments by lumbar transcutaneous spinal cord stimulation (tSCS) have been demonstrated in spinal cord injury. Here, we explored for the first time the motor effects of lumbar tSCS applied at 50 Hz for 30 min in 16 individuals with MS and investigated their temporal persistence post-intervention. We used a comprehensive protocol assessing walking ability, different presentations of spasticity, standing ability, manual dexterity, and trunk control. Walking ability, including walking speed and endurance, was significantly improved for two hours beyond the intervention and returned to baseline after 24 h. Muscle spasms, clonus duration, and exaggerated stretch reflexes were reduced for two hours, and clinically assessed lower-extremity muscle hypertonia remained at improved levels for 24 h post-intervention. Further, postural sway during normal standing with eyes open was decreased for two hours. No changes were detected in manual dexterity and trunk control. Our results suggest that transcutaneous lumbar SCS can serve as a clinically accessible method without known side effects that holds the potential for substantial clinical benefit across the disability spectrum of MS.

Optimal strategy of sEMG feature and measurement position for grasp force estimation.

Grasp force estimation based on surface electromyography (sEMG) is essential for the dexterous control of a prosthetic hand. Nowadays, although increasing the number of sEMG measurement positions and extracting more features are common methods to increase the accuracy of grasp force estimation, it will increase the computational burden. In this paper, an approach based on analysis of variance (ANOVA) and generalized regression neural network (GRNN) for optimal measurement positions and features is proposed, with the purpose of using fewer measurement positions or features to achieve higher estimation accuracy. Firstly, we captured six channels of sEMG from subjects’ forearm and grasp force synchronously. Then, four kinds of features in time domain are extracted from each channel of sEMG. By combining different measurement position sets (MPSs) and feature set (FSs), we construct 945 data sets. These data sets are fed to GRNN to realize grasp force estimation. Normalized root mean square error (NRMS), normalized mean of absolute error (NMAE), and correlation coefficient (CC) between estimated grasp force and actual force are introduced to evaluate the performance of grasp force estimation. Finally, ANOVA and Tukey HSD testing are introduced to analyze grasp force estimation results so as to obtain the optimal measurement positions and features. We obtain the optimal MPSs for grasp force estimation when different FSs are employed, and the optimal FSs when different MPSs are utilized.

Novel intraoperative online functional mapping of somatosensory finger representations for targeted stimulating electrode placement: technical note.

Defining eloquent cortex intraoperatively, traditionally performed by neurosurgeons to preserve patient function, can now help target electrode implantation for restoring function. Brain-machine interfaces (BMIs) have the potential to restore upper-limb motor control to paralyzed patients but require accurate placement of recording and stimulating electrodes to enable functional control of a prosthetic limb. Beyond motor decoding from recording arrays, precise placement of stimulating electrodes in cortical areas associated with finger and fingertip sensations allows for the delivery of sensory feedback that could improve dexterous control of prosthetic hands. In this study, the authors demonstrated the use of a novel intraoperative online functional mapping (OFM) technique with high-density electrocorticography to localize finger representations in human primary somatosensory cortex. In conjunction with traditional pre- and intraoperative targeting approaches, this technique enabled accurate implantation of stimulating microelectrodes, which was confirmed by postimplantation intracortical stimulation of finger and fingertip sensations. This work demonstrates the utility of intraoperative OFM and will inform future studies of closed-loop BMIs in humans.

Segmentation and Removal of Surgical Instruments for Background Scene Visualization from Endoscopic / Laparoscopic Video.

Surgical tool segmentation is becoming imperative to provide detailed information during intra-operative execution. These tools can obscure surgeons' dexterity control due to narrow working space and visual field-of-view, which increases the risk of complications resulting from tissue injuries (e.g. tissue scars and tears). This paper demonstrates a novel application of segmenting and removing surgical instruments from laparoscopic/endoscopic video using digital inpainting algorithms. To segment the surgical instruments, we use a modified U-Net architecture (U-NetPlus) composed of a pre-trained VGG11 or VGG16 encoder and redesigned decoder. The decoder is modified by replacing the transposed convolution operation with an up-sampling operation based on nearest-neighbor (NN) interpolation. This modification removes the artifacts generated by the transposed convolution, and, furthermore, these new interpolation weights require no learning for the upsampling operation. The tool removal algorithms use the tool segmentation mask and either the instrument-free reference frames or previous instrument-containing frames to fill-in (i.e., inpaint) the instrument segmentation mask with the background tissue underneath. We have demonstrated the performance of the proposed surgical tool segmentation/removal algorithms on a robotic instrument dataset from the MICCAI 2015 EndoVis Challenge. We also showed successful performance of the tool removal algorithm from synthetically generated surgical instruments-containing videos obtained by embedding a moving surgical tool into surgical tool-free videos. Our application successfully segments and removes the surgical tool to unveil the background tissue view otherwise obstructed by the tool, producing visually comparable results to the ground truth.

Synergistic Activation Patterns of Hand Muscles in Left-and Right-Hand Dominant Individuals.

Handedness has been associated with behavioral asymmetries between limbs that suggest specialized function of dominant and non-dominant hand. Whether patterns of muscle co-activation, representing muscle synergies, also differ between the limbs remains an open question. Previous investigations of proximal upper limb muscle synergies have reported little evidence of limb asymmetry; however, whether the same is true of the distal upper limb and hand remains unknown. This study compared forearm and hand muscle synergies between the dominant and non-dominant limb of left-handed and right-handed participants. Participants formed their hands into the postures of the American Sign Language (ASL) alphabet, while EMG was recorded from hand and forearm muscles. Muscle synergies were extracted for each limb individually by applying non-negative-matrix-factorization (NMF). Extracted synergies were compared between limbs for each individual, and between individuals to assess within and across participant differences. Results indicate no difference between the limbs for individuals, but differences in limb synergies at the population level. Left limb synergies were found to be more similar than right limb synergies across left- and right-handed individuals. Synergies of the left hand of left dominant individuals were found to have greater population level similarity than the other limbs tested. Results are interpreted with respect to known differences in the neuroanatomy and neurophysiology of proximal and distal upper limb motor control. Implications for skill training in sports requiring dexterous control of the hand are discussed.