Dexterous Robot Hand Research Articles

Restoration of grasping in an upper limb amputee using the myokinetic prosthesis with implanted magnets.

The loss of a hand disrupts the sophisticated neural pathways between the brain and the hand, severely affecting the level of independence of the patient and the ability to carry out daily work and social activities. Recent years have witnessed a rapid evolution of surgical techniques and technologies aimed at restoring dexterous motor functions akin to those of the human hand through bionic solutions, mainly relying on probing of electrical signals from the residual nerves and muscles. Here, we report the clinical implementation of an interface aimed at achieving this goal by exploiting muscle deformation, sensed through passive magnetic implants: the myokinetic interface. One participant with a transradial amputation received an implantation of six permanent magnets in three muscles of the residual limb. A truly self-contained myokinetic prosthetic arm embedding all hardware components and the battery within the prosthetic socket was developed. By retrieving muscle deformation caused by voluntary contraction through magnet localization, we were able to control in real time a dexterous robotic hand following both a direct control strategy and a pattern recognition approach. In just 6 weeks, the participant successfully completed a series of functional tests, achieving scores similar to those achieved when using myoelectric controllers, a standard-of-care solution, with comparable physical and mental workloads. This experience raised conceptual and technical limits of the interface, which nevertheless pave the way for further investigations in a partially unexplored field. This study also demonstrates a viable possibility for intuitively interfacing humans with robotic technologies.

Conceptual Design of 17 DOF Dexterous Robotic Hand for grasping and Manipulation Task

Conceptual Design of 17 DOF Dexterous Robotic Hand for grasping and Manipulation Task

DexFuncGrasp: A Robotic Dexterous Functional Grasp Dataset Constructed from a Cost-Effective Real-Simulation Annotation System

Robot grasp dataset is the basis of designing the robot's grasp generation model. Compared with the building grasp dataset for Low-DOF grippers, it is harder for High-DOF dexterous robot hand. Most current datasets meet the needs of generating stable grasps, but they are not suitable for dexterous hands to complete human-like functional grasp, such as grasp the handle of a cup or pressing the button of a flashlight, so as to enable robots to complete subsequent functional manipulation action autonomously, and there is no dataset with functional grasp pose annotations at present. This paper develops a unique Cost-Effective Real-Simulation Annotation System by leveraging natural hand's actions. The system is able to capture a functional grasp of a dexterous hand in a simulated environment assisted by human demonstration in real world. By using this system, dexterous grasp data can be collected efficiently as well as cost-effective. Finally, we construct the first dexterous functional grasp dataset with rich pose annotations. A Functional Grasp Synthesis Model is also provided to validate the effectiveness of the proposed system and dataset. Our project page is: https://hjlllll.github.io/DFG/.

Disposable Nitrile Glove Resistance to Limonene: Dextrous Robot Hand Versus ASTM F739 Comparison.

The current technique to assess glove resistance to chemicals for worker protection relies on challenging a flat, 2.54 cm diameter glove piece at or near room temperature. This does not simulate a donned whole glove near the skin temperature subjected to work activity forces. Four different types of disposable nonpowdered unlined/unsupported nitrile gloves in triplicate were measured for thickness, porosity, and for the acrylonitrile content (A) of the challenge and collection sides. Limonene permeation at 35 °C through a whole glove on a clenching and nonclenching dextrous robot hand and with the standard ASTM F739 technique were facilitated by taking samples from the collection sides for GC-MS analysis. The standardized breakthrough time (SBT) when permeation reached 100 ng/cm2/min and the steady state permeation rate (SSPR) depended on A, thickness, and porosity. Only the thinnest glove (Lavender) showed statistically significant (p ≤ 0.05) increased average SSPR for the clenching hand relative to the nonclenching hand and for the ASTM technique. The ASTM test data for the three thickest gloves were not statistically different from those of the robot hand, but differed from the manufacturer's. More research with different chemicals and higher clenching forces is needed. Clenching forces can enhance the permeation. Workers wearing ultrathin disposable nitrile gloves have a higher potential for chemical penetration/permeation. Company glove permeation data obtained near room temperature may have a longer SBT and lower SSPR than in practice. Double gloving may be advisible in emergencies and for unknown chemicals when no appropriate thicker Chemical Protective glove is available.

3-D Tactile-Based Object Recognition for Robot Hands Using Force-Sensitive and Bend Sensor Arrays

Tactile sensing is a particularly important and challenging task for a modern robot to safely manipulate objects, interact with humans in a shared space, and provide various services. This paper presents a 3D tactile glove for robots with the combination of a piezoresistive-based force sensor array (412 sensors) covering the full hand and a resistive bend sensor array (5 sensors) on the back of five fingers. Deep learning-based CNN (Convolutional Neural Network) methods using the designed tactile glove are proposed for object recognition. In the experiment for recognizing 15 objects with a dexterous robot hand, an average classification accuracy of 93.67% has been achieved. Comparison experiments with three other typical classifiers (the Quadratic SVM, Weighted KNN, and Bagged Trees) and our CNN methods show an average recognition accuracy of 91.67% with the 3D tactile glove, revealing an accuracy improvement of 4.17% over only using the force sensor array. We further apply our 3D tactile glove and the multi-modal CNN to identify three other objects and demonstrate their generalization ability of tactile object recognition with an average success accuracy of 78.33%. The proposed 3D tactile glove can be further used in human-robot interactions, the design of prosthetics and humanoid robots, and for improving the intelligence level in brain-computer collaborative systems.

Toward Human-Like Grasp: Functional Grasp by Dexterous Robotic Hand Via Object-Hand Semantic Representation.

Intelligent robotic manipulation is a challenging study of machine intelligence. Although many dexterous robotic hands have been designed to assist or replace human hands in executing various tasks, how to teach them to perform dexterous operations like human hands is still a challenge. This motivates us to conduct an in-depth analysis of human behavior in manipulating objects and propose an object-hand manipulation representation. This representation provides an intuitive and clear semantic indication of how the dexterous hand should touch and manipulate an object based on the object's own functional areas. At the same time, we propose a functional grasp synthesis framework, which does not require real grasp label supervision, but relies on the guidance of our object-hand manipulation representation. In addition, in order to obtain better functional grasp synthesis results, we propose a network pre-training method that can make full use of easily obtained stable grasp data, and a network training strategy to coordinate the loss functions. We conduct object manipulation experiments on a real robot platform, and evaluate the performance and generalization of our object-hand manipulation representation and grasp synthesis framework. The project website is: https://github.com/zhutq-github/Toward-Human-Like-Grasp-V2-.

A Mechanical Hand-Tracking System With Tactile Feedback Designed for Telemanipulation.

In this paper, we present a mechanical hand-tracking system with tactile feedback designed for fine manipulation in teleoperation scenarios. Alternative tracking methods based on artificial vision and data gloves have become an asset for virtual reality interaction. Yet, occlusions, lack of precision, and the absence of effective haptic feedback beyond vibrotactile still appear as a limit for teleoperation applications. In this work, we propose a methodology to design a linkage mechanism for hand pose tracking purposes, preserving complete finger mobility. Presentation of the method is followed by design and implementation of a working prototype, and by evaluation of the tracking accuracy using optical markers. Moreover, a teleoperation experiment involving a dexterous robotic arm and hand was proposed to ten participants. It investigated the effectiveness and repeatability of the hand tracking with combined haptic feedback during a proposed pick and place manipulation tasks.

Large Curvature Bending Measurable Fiber-Optic Neurons for Multi-Joint Bending Perception

In recent years, dexterous robotic hands have become an indispensable part of intelligent equipment. The introduction of bionic neurons that provide finger motion perception will improve the accuracy of joint control. Here, series- and parallel-integrated inner cladding fiber Bragg gratings (ICFBG) act as bionic neurons to realize multi-joint bending perception in a robotic finger by measuring the reflective intensity changes of the ICFBGs. The fiber-optic neurons (FON) were fabricated using femtosecond laser point-by-point technology in the inner cladding of bend-insensitive single-mode fiber (BIF). The proposed FONs can measure large curvature bending up to 250 m <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-1</sup> , with a bending sensitivity of ∼0.07 dB/m <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-1</sup> and a measuring resolution of 0.4 m <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-1</sup> . Additionally, vector bending sensing can be achieved using the two pairs of ICFBGs that are orthogonally distributed to each other. The FONs are insensitive to temperature and axial strain. Furthermore, the proposed FONs were fixed to the knuckles of a robotic finger for multi-joint bending perception. The results show that FONs can potentially be used in robotic gesture recognition and control systems.

Soft Magnetic Fingertip With Particle-Jamming Structure for Tactile Perception and Grasping

Typically, the fingertips of a dexterous robotic hand are designed to be rigid and equipped with a variety of sensors to provide tactile perception. However, this design scheme renders the grasping of small objects difficult because rigid fingertips usually have a small contact area with the objects. In this paper, we propose a novel fingertip design that combines soft tactile skin and a particle jamming structure. Benefiting from the particle jamming structure, the soft skin and fluid-state jamming particle material ensure that the fingertip encloses the object as much as possible when in contact with it. Then, the application of a vacuum turns the particle material into a high stiffness solid state, allowing it to hold objects. Soft tactile skin integrated with permanent magnetic particles estimates contact information between the fingertip and objects. Thus, utilizing the soft tactile skin and particle jamming structure the fingertip can perceive and grasp objects simultaneously. The proposed soft fingertip module is fabricated by molding, which reduces the processing cycle time and cost. Additionally, a series of experiments were conducted to demonstrate the performance of this novel fingertip design. The results validate that the proposed robotic fingertip can grasp tiny objects while perceiving contact position and force information.

Grasping Living Objects With Adversarial Behaviors Using Inverse Reinforcement Learning

Living objects are difficult to grasp since they can actively elude capture by adopting adversarial behaviors that are extremely hard to model or predict. In this case, an inappropriately strong contact force may hurt the struggling living objects and a grasping algorithm that can minimize the contact force whenever possible is required. To solve this challenging task, in this article, we present a reinforcement-learning (RL)-based algorithm with two stages: the pregrasp stage and the in-hand stage. In the pregrasp stage, the robot focuses on the living object's adversarial behavior and approaches it in a reliable manner. In particular, we use inverse RL to encode the living object's adversarial behavior into a reward function. The negative value of the learned reward function is then used to train a high-quality grasping policy that can compete with the living object's adversarial behavior with the RL framework. In the in-hand stage, we use RL to train a grasp policy such that the dexterous hand can grab the living object with the minimal force. A set of dense rewards are also specifically designed to encourage the robot to grasp and hold the living object persistently. To further improve the grasp performance, we explicitly take into account the structure of the dexterous robot hand by treating the hand as a graph and adopting graph convolutional network to formulate the grasping policy. We conduct a set of experiments to demonstrate the performance of our proposed method, in which the robot can grasp living objects with the success rate of 90% and 95% in the pregrasp and in-hand stages, respectively. The contact force applied by the robotic hand to the living object is dramatically reduced in comparison with the baseline grasping policy.

Formalization of the inverse kinematics of three-fingered dexterous hand

As an effective extension of human limbs, dexterous robotic hands based on cyber-physical systems (CPS) are used to perform diverse tasks in industrial production, medical operation, outer-space exploration, etc. The reliable functioning of dexterous robotic hands depends on the proper modeling and solving of inverse kinematics problems in the design phase. The traditional D-H parameter method for modeling and solving inverse kinematics problems suffers from the singularity problem, which is avoided by methods based on the screw theory. However, the manual or computer-simulation-based development of these methods could suffer from human errors and software defects. In this paper, we address this problem by using formal techniques and tools to develop a screw-theory-based method. Firstly, we formalize theories related to the Paden-Kahan sub-problem in an interactive theorem prover. Secondly, we construct a formal model for solving the inverse kinematics problem for robotic systems. Lastly, taking the three-fingered dexterous hand as a case study, we formally verified its inverse kinematics solutions. The results presented in this article help guarantee a level of safety for robotic systems that is beyond achievable by manual and computer-simulation-based development methods.

Mechatronic Design of a Self-Contained Dexterous Robotic Hand for Gestural Communication

Mechatronic Design of a Self-Contained Dexterous Robotic Hand for Gestural Communication

A Signal Flow Graph Approach to the Resolution of Spherical Triangles Using CORDIC

Jack Volder’s original motivation for the COordinate DIgital Computer (CORDIC) was the real-time digital solution of spherical triangle equations employed in airborne navigation, for which he presented a solution flow diagram without detailing its construction. In fact, without a strong guidance, it is not easy to express the solutions of linear algebraic problems such as those involved when solving spherical triangles as cascades of CORDIC operations—called by Volder CORDIC solution-flow diagrams—and thus to devise a system solution on CORDIC processing units. As it gives a bird’s eye view of the system design problem, a signal flow graph representation of the underlying system of linear equations provides such guidance; the operations leading to the problem solution are uncovered by performing a sequence of partial flow reversals on the graph. The approach is illustrated by the problem where two sides and the included angle of a spherical triangle are given, called SAS problem, that is encountered in applications as diverse as air navigation, lattice filters for adaptive processing, and dexterous robotic hands. The solutions thus obtained are at least as efficient as existing ones, whenever available.

A Prediction of Fingertip Force and Joint Kinematics of Individual Fingers using Motoneuron Firing Activities

<h3>Research Objectives</h3> Advanced assistive robotic hands can help individuals with hand impairment to perform daily activities. The detection of intended motor output of individual fingers, however, remains a challenge. The objective of this study was to develop a decoding method to continuously and simultaneously predict fingertip forces and joint kinematics of individual fingers using motoneuron firing frequency. <h3>Design</h3> Blind source separation method was applied to EMG signals to obtain motor unit (MU) firing activities. MU refinement procedure was used to determine finger- and task-specific MU pools. Obtained MU pools were tested on different datasets involving both isometric and dynamic tasks (combined). Populational MU firing frequencies were calculated to estimate the fingertip forces and joint angles concurrently using regression models. Model performance was compared with conventional EMG amplitude-based approach. <h3>Setting</h3> The study was carried out at the NC State/UNC Joint Department of Biomedical Engineering. <h3>Participants</h3> Seven healthy volunteers (Age: 28±7) participated in this study. No participant had any prior neuro-muscular ailments. All participants provided an informed consent. <h3>Interventions</h3> Not applicable. <h3>Main Outcome Measure(s)</h3> Concurrent and continuous prediction of finger kinetics and kinematics via motoneuron activities. <h3>Results</h3> MU approach led to better performance (higher correlation between the predicted and measured output and lower prediction error) in predicting both joint angles and fingertip forces than the EMG amplitude approach. <h3>Conclusion(s)</h3> MU pools can be identified for specific finger-task combinations, which allows concurrent control of fingertip forces and joint angles with assistive robot hands. Separation matrix derived from single finger trials was used applied to combined trials to avoid the computationally intensive nature of HD-EMG decomposition and to determine its usability in real-time. Further development of this method could provide concurrent and continuous control of individual finger movements of highly dexterous robotic hands.

Electrostatic Adhesion Clutch with Superhigh Force Density Achieved by MXene-Poly(Vinylidene Fluoride-Trifluoroethylene-Chlorotrifluoroethylene) Composites.

Electrostatic adhesion (EA) clutches are widely applied in robots, wearable devices, and virtual reality, due to their compliance, lightweight, ultrathin profile, and low power consumption. Higher force density has been constantly perpetuated in the past decades since EA was initially proposed. In this study, by composing terpolymer poly(vinylidene fluoride-trifluoroethylene-chlorotrifluoroethylene) [P(VDF-TrFE-CTFE)] and two-dimensional Ti3C2Tx nanosheets (MXene), nanocomposite films with high dielectric constant ( > 2300) and low loss tangent are achieved. The force representative index (the relative dielectric constant times the square of breakdown electric field) is enhanced by 5.91 times due to the charge accumulation at matrix-filler interfaces. Superhigh shear stress (85.61 N cm-2) is generated, 408% higher than the previous maximum value. One of the EA clutches fabricated in this study is only 160 μm thin and 0.4 g heavy. Owing to the low current (<1 μA), the power consumption is <60 mW/cm2. It can hold a 2.5 kg weight by only 0.32 cm2 area and support an adult (45 kg) (Clinical Trial Registration number: 20210090). With this technology, a dexterous robotic hand is displayed to grasp and release a ball, showing extensive applications of this technique.

Using Adaptive Directed Acyclic Graph for Human In-Hand Motion Identification with Hybrid Surface Electromyography and Kinect

The multi-fingered dexterous robotic hand is increasingly used to achieve more complex and sophisticated human-like manipulation tasks on various occasions. This paper proposes a hybrid Surface Electromyography (SEMG) and Kinect-based human in-hand motion (HIM) capture system architecture for recognizing complex motions of the humans by observing the state information between an object and the human hand, then transferring the manipulation skills into bionic multi-fingered robotic hand realizing dexterous in-hand manipulation. First, an Adaptive Directed Acyclic Graph (ADAG) algorithm for recognizing HIMs is proposed and optimized based on the comparison of multi-class support vector machines; second, ten representative complex in-hand motions are demonstrated by ten subjects, and SEMG and Kinect signals are obtained based on a multi-modal data acquisition platform; then, combined with the proposed algorithm framework, a series of data preprocessing algorithms are realized. There is statistical symmetry in similar types of SEMG signals and images, and asymmetry in different types of SEMG signals and images. A detailed analysis and an in-depth discussion are given from the results of the ADAG recognizing HIMs, motion recognition rates of different perceptrons, motion recognition rates of different subjects, motion recognition rates of different multi-class SVM methods, and motion recognition rates of different machine learning methods. The results of this experiment confirm the feasibility of the proposed method, with a recognition rate of 95.10%.

Peg-in-Hole Assembly With Dual-Arm Robot and Dexterous Robot Hands

This study focuses on implementing robotic peg-in-hole in a more human-like approach, namely using dual-arms and dexterous robotic hands. The peg-in-hole strategy in this study mainly consists of two parts: grasping strategy (hand part) and assembly strategy (arm part). The grasping strategy explains the fundamental grasping method called “advanced blind grasping” and the in-hand manipulation method that is required for the reorientation of the workpiece. A feed-forward task space force control scheme is proposed for the actual implementation. The assembly strategy presents fundamental unit motions called “perturbation pattern” and proposes four assembly stages that are constructed by a combination of the unit motions. A force-position hybrid control for the implementation of the assembly strategy was also addressed. For evaluation, a peg-in-hole assembly demonstration with a keyhole-like shape was conducted using a human-sized 50-DOF upper-body robot.

End-to-End From Human Hand Synergies to Robot Hand Tendon Routing

The human hand capabilities are paramount for highly dexterous manipulation interactions. Unfortunately, the limitations of current technologies make replicating such capabilities unfeasible. Although several works have focused on directly attempting to create robot hands able to mimic human ones closely, few of them have attempted to create generalizable platforms, where robotic hand mechanisms can be iteratively selected and customized to different tasks. In order to build highly dexterous robotic hands in the future, it is crucial to understand not only human manipulation, but also develop methods to leverage robotic mechanisms limitations to mimic human hand interactions accurately. In this letter, we propose an end-to-end framework capable of generating underactuated tendon routings that allow a generic robot hand model to reproduce desired observed human grasp motion synergies accurately. Our contributions are threefold: (1) an end to end framework to generate task-oriented robot hand tendon routings, with the potential to implement desired synergies, (2) a novel grammar based representation of robot hand tendon routings, and (3) a schematic visualization of robot hand tendon routings. The latter two contributions have the potential to embed and compare properties among robot hands. Our results in simulation show that the proposed method produces tendon routing mechanisms that are able to closely mimic the joint trajectories of human subjects performing the same experimental tasks, while achieving dynamically stable grasping postures.

Modelling and Finite Element Based Analysis of a Five Fingered Underactuated Robotic Hand

Abstract: Imparting the dexterity and autonomous competence to a robotic system is a significant burden in humanoid robotics, especially in the fields of industrial manufacturing, prosthetics, orthopedic rehabilitation, etc. Operating a humanoid hand requires a very innovative actuator and transmission system. The under-actuated concepts are proving to be a possible means of achieving extremely dexterous robotic hands without the need for diverse mechanical design. The main characteristics of an under-actuated robotic hand are that fewer actuators are required to operate it than the degrees of freedom. The under-actuated equivalent hand is significantly less expensive than the fully-actuated equivalent hand and remarkably reduces the complexity of the control system. The existing work dealt with the modeling and finite element-based analysis of an anthropomorphic underactuated robotic hand using five fingers including the thumb and palm with dexterity and with a total of twenty-one degrees of freedom.

Robot Operating System controlled anthropomorphic robot hand

This paper presents a new design of a dexterous robot hand by incorporating human hand factors. The robotic hand is a Robot Operating System (ROS) controlled standalone unit that can perform key tasks and work independently. Hardware such as actuators, electronics, sensors, pulleys etc. are embedded within or on the hand itself. Raspberry Pi, a single board computer which runs ROS and is used to control the hand movements as well as process the sensor signals is placed outside of the hand. It supports peripheral devices such as screen display, keyboard and mouse. The hand prototype is designed in Solid Works and 3D printed/built using aluminum sheet. The prototype is similar to the human hand in terms of shape and possesses key functionalities and abilities of the human hand, especially to imitate key movements of the human hand and be as dexterous as possible whilst keeping a low cost. Other important factors considered while prototyping the model were that the hand should be reliable, have a durable const­­ruction, and should be built using widely available off-the-shelf components and an open-source software. Though the prototype hand only has 6 degrees-of-freedom (DOF) compared to the 22 DOF of the human hand, it is able to perform most grasps effectively. The proposed model will allow other researchers to build similar robotic hands and perform specialized research.