Multi-fingered Dexterous Hand Research Articles

Interactive learning for multi-finger dexterous hand: A model-free hierarchical deep reinforcement learning approach

When a multi-fingered dexterous hand interacts with the external environment, it encounters various challenges, including the utilization of complex control techniques and the intricate coordination of finger motion sequences. Previous studies have primarily concentrated on investigating the interaction between multi-fingered dexterous hands and external objects, usually using model-based control or model-free reinforcement learning techniques. However, during practical implementation, various constraining factors are encountered, such as intricate modeling and limited interaction capabilities. In practical scenarios, the utilization of multi-fingered dexterous hands is imperative for the swift and efficient execution of a wide range of interactive tasks, including but not limited to throwing a ball and playing rock-paper-scissors. These tasks require skilled manual dexterity to demonstrate both precise control and quick responsiveness. To tackle this issue, we propose a hierarchical control approach for multi-fingered dexterous hands with interactive functionalities, utilizing model-free deep reinforcement learning. The complex interaction task is decomposed into simple sub-tasks using hierarchical strategy and action primitive decomposition, which effectively reduces the complexity of the action space, and achieves the motion planning and end finger trajectory control of dexterous hand. In a simulated environment, the aforementioned method has successfully executed interactive tasks, including ball throwing and playing rock-paper-scissors. It achieved a maximum normalized reward of 0.83 and an 84 % success rate. These results are noteworthy in terms of both control accuracy and response speed. This study offers novel insights into the effective resolution of the intricate challenges associated with interactions involving multi-fingered dexterous hands and human-computer interaction.

Research on two-stage grasping planning method for multifingered dexterous hand

Purpose The purpose of this paper is to design a multifingered dexterous hand grasping planning method that can efficiently perform grasping tasks on multiple dexterous hand platforms. Design/methodology/approach The grasping process is divided into two stages: offline and online. In the offline stage, the grasping solution form is improved based on the forward kinematic model of the dexterous hand. A comprehensive evaluation method of grasping quality is designed to obtain the optimal grasping solution offline data set. In the online stage, a safe and efficient selection strategy of the optimal grasping solution is proposed, which can quickly obtain the optimal grasping solution without collision. Findings The experiments verified that the method can be applied to different multifingered dexterous hands, and the average grasping success rate for objects with different structures is 91.7%, indicating a good grasping effect. Originality/value Using a forward kinematic model to generate initial grasping points can improve the generality of grasping planning methods and the quality of initial grasping solutions. The offline data set of optimized grasping solutions can be generated faster by the comprehensive evaluation method of grasping quality. Through the simple and fast obstacle avoidance strategy, the safe optimal grasping solution can be quickly obtained when performing a grasping task. The proposed method can be applied to automatic assembly scenarios where the end effector is a multifingered dexterous hand, which provides a technical solution for the promotion of multifingered dexterous hands in industrial scenarios.

Using Adaptive Directed Acyclic Graph for Human In-Hand Motion Identification with Hybrid Surface Electromyography and Kinect

The multi-fingered dexterous robotic hand is increasingly used to achieve more complex and sophisticated human-like manipulation tasks on various occasions. This paper proposes a hybrid Surface Electromyography (SEMG) and Kinect-based human in-hand motion (HIM) capture system architecture for recognizing complex motions of the humans by observing the state information between an object and the human hand, then transferring the manipulation skills into bionic multi-fingered robotic hand realizing dexterous in-hand manipulation. First, an Adaptive Directed Acyclic Graph (ADAG) algorithm for recognizing HIMs is proposed and optimized based on the comparison of multi-class support vector machines; second, ten representative complex in-hand motions are demonstrated by ten subjects, and SEMG and Kinect signals are obtained based on a multi-modal data acquisition platform; then, combined with the proposed algorithm framework, a series of data preprocessing algorithms are realized. There is statistical symmetry in similar types of SEMG signals and images, and asymmetry in different types of SEMG signals and images. A detailed analysis and an in-depth discussion are given from the results of the ADAG recognizing HIMs, motion recognition rates of different perceptrons, motion recognition rates of different subjects, motion recognition rates of different multi-class SVM methods, and motion recognition rates of different machine learning methods. The results of this experiment confirm the feasibility of the proposed method, with a recognition rate of 95.10%.

A Survey of Multifingered Robotic Manipulation: Biological Results, Structural Evolvements, and Learning Methods.

Multifingered robotic hands (usually referred to as dexterous hands) are designed to achieve human-level or human-like manipulations for robots or as prostheses for the disabled. The research dates back 30 years ago, yet, there remain great challenges to effectively design and control them due to their high dimensionality of configuration, frequently switched interaction modes, and various task generalization requirements. This article aims to give a brief overview of multifingered robotic manipulation from three aspects: a) the biological results, b) the structural evolvements, and c) the learning methods, and discuss potential future directions. First, we investigate the structure and principle of hand-centered visual sensing, tactile sensing, and motor control and related behavioral results. Then, we review several typical multifingered dexterous hands from task scenarios, actuation mechanisms, and in-hand sensors points. Third, we report the recent progress of various learning-based multifingered manipulation methods, including but not limited to reinforcement learning, imitation learning, and other sub-class methods. The article concludes with open issues and our thoughts on future directions.

Multi-fingered Dexterous Hands: From Simplicity to Complexity and Simplifying Complex Applications

Multi-fingered Dexterous Hands: From Simplicity to Complexity and Simplifying Complex Applications

Robotic Grasping of Unknown Objects Using Novel Multilevel Convolutional Neural Networks: From Parallel Gripper to Dexterous Hand

To achieve high-accuracy grasping of unknown objects, we present novel multilevel convolutional neural networks (CNNs) for robotic grasping with a parallel gripper or multifingered dexterous hand. The multilevel CNNs include four levels with different structures and functions. The first level is constructed to get the approximate position of the grasped object. The second level aims to obtain the preselected grasping rectangles. The third level is constructed to re-evaluate the preselected grasping rectangles and obtain substantially detailed features with quite a large network, so as to assess each preselected grasping rectangle exactly. By using a selection algorithm, the optimal grasping rectangle can be determined and unknown object grasping can be achieved with a parallel gripper. The purpose of the fourth level is to obtain the finger position distribution to complete the accurate grasping of unknown objects with a multifingered dexterous hand. The test results indicate that, compared to state-of-the-art methods, the proposed multilevel CNNs can greatly increase the precision of the grasping rectangle. Grasping experiments were implemented on a Youbot arm with five degrees of freedom and a Shadow four-fingered dexterous hand. The results show that the multilevel CNNs can determine the optimal grasping rectangle and finger position distribution, thereby achieving high-accuracy grasping of various unknown objects, even under several complex environmental conditions. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners—Robot grasping of objects lags far behind human experiences and poses a significant challenge in the robotics area. To solve it, we present new multilevel convolutional neural networks (CNNs) to process red green blue-depth (RGB-D) images and realize optimal grasping detection of unknown objects. Moreover, we provide details of the network structure, network training, and network testing. The testing results obtained from the open grasping data set show that the multilevel CNNs can significantly increase the accuracy of the grasping rectangle compared to state-of-the-art methods. Experiments were implemented on different robotic platforms, including a five-degrees-of-freedom Youbot arm with a parallel gripper and a UR5 robot arm with a Shadow multifingered dexterous hand. The results validate that the multilevel CNNs offer excellent generalization and robustness for handling different sizes and shapes of unknown objects, as well as background disturbances, which are key problems in robotic manipulation.

Grasp prediction and evaluation of multi-fingered dexterous hands using deep learning

Learning from human skills has become one of the popular inspirations in grasp prediction and evaluation, but lack of effective methods on groups of grasp points for multi-fingered dexterous hands yields an open challenge. When facing an object, humans firstly predict a variety of options for grasps, which can be concerned as a complex multi-valued problem. After prediction, humans evaluate grasps and then choose the optimal one. Inspired by human skills, we propose Grasp Prediction Networks (GPNs) based on Convolutional Neural Networks (CNNs) and Mixture Density Networks (MDNs). The proposed GPNs map from a depth image to a set of parameters for Gaussian Mixture Model (GMM), from which candidate groups of grasp points can be sampled for prediction. Besides, we also propose Grasp Evaluation Networks (GENs) to evaluate candidate groups and then choose the optimal group of grasp points. The proposed GENs consider force-closure metric as grasp quality for evaluation. Different from other related work, our method (1) utilizes a probabilistic model to predict multiple groups of grasp points from a monocular depth image and (2) evaluates grasp quality with force-closure metric given a monocular depth image and a group of grasp points. Furthermore, we built a grasp dataset which consists of depth images, groups of grasp points and each group’s grasp quality. Herein, three different experiments were designed to validate our approach. The first one was a comparative experiment and revealed that GPNs show equivalent performance as GraspIt! in terms of high-quality grasp planning. The second one was also a comparative experiment, which validated that GENs can evaluate grasps as precisely as GraspIt!. Moreover, the last one was an actual experiment implemented on Shadow Hand Lite, and experimental results indicated that our approach achieved finely grasp of novel objects.

Development of multi-fingered dexterous hand for grasping manipulation

In this paper, a developed multi-fingered dexterous hand with flexible tactile skin is described. The dexterous hand has 5-fingers with 6-DOFs and each finger is equipped with a small harmonic drive gear and a fine high-power mini actuator. To achieve the goal of grasping with high accuracy, each fingertip is covered with the tactile array sensors for determination of the force between the finger and the grasped object. Some preliminary experiments are conducted to illustrate the performance of the grasping of the developed dexterous hand.

EXPERIMENTAL ANALYSIS ON SPATIAL AND CARTESIAN IMPEDANCE CONTROL FOR THE DEXTEROUS DLR/HIT II HAND

This paper presents an experimental study on impedance control in both Cartesian and object level with adaptive friction compensation for dexterous robot hand based on joint torque feedback. To adaptively decrease the effects of high friction caused by complex transmission systems and joint coupling, a friction observer is proposed based on the extended Kalman filter (EKF) in this paper. A Cartesian impedance controller is implemented on a multi-fingered dexterous robot hand with identical fingers, based on the modelling of each modular finger. In addition, a flexible n-fingered object frame is proposed in this paper, applicable to any finger configuration with three or more fingers (n ≥ 3). This enables the design of a 6-DoF spatial impedance controller. Stability of the closedloop system with friction observer is analysed. A position error of less than 0.16 ◦ is achieved using joint impedance control with adaptive friction compensation, which shows significant improvement in performance, as compared to 1.5 ◦ without compensation, and 0.5 ◦ with fixed-parameters friction compensation. Experimental results confirm the improvement in performance for the robot hand with Cartesian impedance control and adaptive joint friction compensation, demonstrating the effectiveness of spatial impedance controller with the proposed object frame and estimation strategy.

Control System Design of the YWZ Multi-Fingered Dexterous Hand

The manipulation abilities of a multi-fingered dexterous hand, such as motion in real-time, flexibility, grasp stability etc., are largely dependent on its control system. This paper developed a control system for the YWZ dexterous hand, which had five fingers and twenty degrees of freedom (DOFs). All of the finger joints of the YWZ dexterous handwere active joints driven by twenty micro-stepper motors respectively. The main contribution of this paper was that we were able to use stepper motor control to actuate the hand's fingers, thus, increasing the hands feasibility. Based the actuators of the YWZ dexterous hand, we firstly developed an integrated circuit board (ICB), which was the communication hardware between the personal computer (PC) and the YWZ dexterous hand. The ICB included a centre controller, twenty driver chips, a USB port and other electrical parts. Then, a communication procedure between the PC and the ICB was developed to send the control commands to actuate the YWZ dexterous hand. Experiment results showed that under this control system, the motion of the YWZ dexterous hand was real-time; both the motion accuracy and the motion stability of the YWZ dexterous hand were reliable. Compared with other types of actuators related to dexterous hands, such as pneumatic servo cylinder, DC servo motor, shape memory alloy etc., experiment results verified that the stepper motors as actuators for the dexterous handswere effective, economical, controllable and stable.

Design and control of a cable-free and single-motor-driven dexterous robot hand

This paper presents a new design for a multi-fingered dexterous robot hand. Based on an analysis of currently available hands, two improvements are proposed for better mechanism, actuation, and functionality. We designed a new cable-free, rigid connection, fifteen DOFs robot hand mechanism having one DC-motor, fifteen bidirectional clutches, and the capability for holding cylinder object. The finger mechanism and motion transmission chain of this robot hand are described. Hand control structure and finger control structure are analyzed. In addition, simulation is perfonned in Matlab® SimMechanics toolbox. Two design problems are also discussed and summarized. Copyright © 2003 IFAC.