EP-DDPG: A deep reinforcement learning framework for visual-tactile fusion in grasping control of humanoid robotic hand

Existing open-source robotic hand designs rarely achieve full human-like functionality, limiting their accessibility and utility. Ale-HAND-ra addresses this gap with a cost-effective, open-source humanoid robotic hand featuring 20 degrees of freedom, developed using filament- and resin-based 3D printing for affordable production (total cost: \$161.54). Grounded in biomechanical analysis, its kinematic model enables precise finger movements and versatile grasping, validated through simulations and tests against human benchmarks. Employing a modular design with servomotors and low-cost materials, Ale-HAND-ra balances functionality and simplicity, achieving a 350-gram grip capacity. Released on GitHub, this prototype fosters collaboration for educational, prosthetic, and robotic applications.

The article presents the development of an automated smart humanoid robot hand on the Arduino platform using 3D printing, servo drives, and a sensor glove with flexible sensors. An electrical circuit and a software module for reading and processing signals have been developed, which provides real-time reproduction of the operator's finger movements. Modelling, assembly of mechanical and electronic parts, calibration of sensors and actuators have been carried out. Experimental studies have confirmed the accuracy, speed, and stability of the system, which makes it suitable for educational and demonstration robotics applications.

The operation accuracy of humanoid robot hands driven by the tendon sheath will be reduced due to the influence of friction torque during rotation, which is not conducive to the dexterous operations of humanoid robot hands. In order to improve the control accuracy of humanoid robot hands, this paper proposes a control strategy based on the disturbance observer compensation, which eliminates the external disturbance torque by compensating the friction torque. Firstly, this article proposes the mechanical structure of humanoid robot hands driven by the tendon sheath with 19 degrees of freedom (DOF). This humanoid robot hands can grasp most irregular objects. Next, the dynamic model of humanoid robot hands’ drive systems is established based on the tendon sheath transmission theory. The driving system’s dynamic model reveals the influence of friction torque on the motion accuracy of the humanoid robot hand. Then, the disturbance observer (DOB) is designed based on the robust stability theorem. The DOB is used to improve the control accuracy of the driving system, thereby improving the operational accuracy of humanoid robot hands. Finally, this article conducts simulation rotation tracking control and prototype grasping control experiments on humanoid robot hands. The experimental results show that the proposed control strategy based on disturbance compensation can effectively improve the operational accuracy of humanoid robot hands. Note to Practitioners—This paper proposes a humanoid hand with 19 degrees of freedom based on the tendon-driven theory and applies the cable theory for its dynamic modeling. To address the issue of decreased precision caused by friction in practical operations, a friction compensation control strategy based on disturbance observer is proposed in this study. This control strategy improves the motion accuracy and stability of the mechanical hand. Finally, the effectiveness of the proposed control strategy is demonstrated through numerical simulation and experimental validation.

This paper presents the design and fabrication of Fil3D-Hand, a humanoid robotic hand with five fingers integrating nanocomposite filament strain sensors to monitor finger movements. The objective is to replicate the functionality of the human hand through an affordable design, with potential applications in humanoid robotics and prosthetic rehabilitation. Based on the investigation of the human hand’s anatomical structure and existing robotic hands, the mechanical design of Fil3D-Hand is elaborated. The kinematics analysis and finger workspace are formulated and illustrated using the Denavit-Hartenberg (D-H) convention and numerical simulations. The proposed design features 17 degrees of freedom, weighs 2.4 kg (including the forearm and 12 servo motors), and achieves a fabrication cost of less than $350. The Fil3D hand’s size is approximately 1.2 the size of the real human hand, allowing for enhanced versatility while maintaining a naturalistic appearance. This design strategy ensures a cost-effective, tendon-driven underactuated mechanism, utilizing servo motors for precise control of finger joints. It also integrates 14 filament strain sensors for accurate monitoring of finger positions. Experimental results demonstrate the performance of these sensors, revealing a highly linear correlation (R2=0.9964) between the finger’s proximal angles and the strain sensor resistance, guaranteeing precise real-time monitoring of finger movements. Evaluation of the Fil3D-Hand’s performance highlights its excellent grasping capabilities, showcasing its potential in practical applications.

Humans possess the remarkable ability to perceive the intricate world by integrating multiple senses. However, the challenge of enabling humanoid robots to achieve multimodal sensing and fine recognition of metallic materials persists. In this study, we propose a flexible tactile sensor that mimics the sensory patterns of human skin, which is assembled by a flexible electromagnetic coil that is engraved on the surface of a polyimide substrate and porous MXene/CNT aerogel. This sensor is capable of detecting pressure, temperature, and inductive signals with minimal interference via three disparate response mechanisms of the piezoresistive sensing, the thermoelectric principle, and the electromagnetic induction effect, allowing the device with the abilities of sensing grasp forces and selectively identifying ferromagnetic and nonferromagnetic metals, which has a high accuracy rate of 99.2% in distinguishing mixed metals with varying ratios based on the fusion algorithm of multimodal sensory data. Further, the sensor was integrated on a humanoid robotic hand to demonstrate its recognition capacity of objects used in a kitchen setting and a simulated scenario of mineral exploration, achieving a remarkable ultrafine accuracy of 100% in distinguishing 16 common metal products. These findings will pave the way for humanoid robots to attain heightened levels of perception and recognition.

Grasp planning is crucial for robots to perform precision grasping tasks, where determining the grasp points significantly impacts the performance of the robotic hand. Currently, the majority of grasp planning methods based on analytic approaches solve the problem by transforming it into a nonlinear constrained planning problem. This method often requires performing convex hull computations, which tend to have high computational complexity. This paper proposes a new algorithm for calculating multi-finger force-closure grasps of three-dimensional objects based on humanoid multi-fingered hands. Firstly, sufficient conditions for the multi-finger force-closure grasps of three-dimensional objects are derived from a point contact model with friction. These three-dimensional force-closure conditions are then transformed into two-dimensional plane conditions, leading to a simple algorithm for multi-finger force-closure determination. This method is purely based on geometric analysis, resulting in low computational demands and enabling the rapid assessment of force-closure grasps, which are beneficial for real-time applications. Finally, the algorithm is validated through two case studies, demonstrating its feasibility and effectiveness.

Complex robotic systems, such as humanoid robot hands, soft robots, and walking robots, pose a challenging control problem due to their high dimensionality and heavy non-linearities. Conventional model-based feedback controllers demonstrate robustness and stability but struggle to cope with the escalating system design and tuning complexity accompanying larger dimensions. In contrast, data-driven methods such as artificial neural networks excel at representing high-dimensional data but lack robustness, generalization, and real-time adaptiveness. In response to these challenges, researchers are directing their focus to biological paradigms, drawing inspiration from the remarkable control capabilities inherent in the human body. This has motivated the exploration of new control methods aimed at closely emulating the motor functions of the brain given the current insights in neuroscience. Recent investigation into these Brain-Inspired control techniques have yielded promising results, notably in tasks involving trajectory tracking and robot locomotion. This paper presents a comprehensive review of the foremost trends in biomimetic brain-inspired control methods to tackle the intricacies associated with controlling complex robotic systems.

Losing the ability to deform the palm is unthinkable for a human hand, and the same is true for a humanoid robotic hand. Here, we aim to evaluate the performance of human palms and develop a palm-movable humanoid robotic hand with a few actuators. We quantified the palm morphological characteristics by analyzing the palm morphological parameters collected from 42 subjects. Subsequently, we constructed a kinematic model of the human hand with these characteristics to quantitatively analyze how a movable palm influences the finger workspace. We found that a movable palm can significantly improve fingertip-reachable space (at least 50%) and increase the capability of fingers to oppose each other (at least 14%). Furthermore, we designed a palm-movable humanoid robotic hand with only four actuators and compared the grasping performance of this prototype with that of a palm-fixed robotic hand and a human hand. The results suggest that a movable palm contributes to enhancing anthropomorphism, adaptability, and grip stability in humanoid robotic hands. Hence, our findings contribute to the development of dexterous robotic hands.

Soft robots, especially soft robotic hands, possess prominent potential for applications in close proximity and direct contact interaction with humans due to their softness and compliant nature. The safety perception of users during interactions with soft robots plays a crucial role in influencing trust, adaptability, and overall interaction outcomes in human-robot interaction (HRI). Although soft robots have been claimed to be safe for over a decade, research addressing the perceived safety of soft robots still needs to be undertaken. The current safety guidelines for rigid robots in HRI are unsuitable for soft robots. In this paper, we highlight the distinctive safety issues associated with soft robots and propose a framework for evaluating the perceived safety in human-soft robot interaction (HSRI). User experiments were conducted, employing a combination of quantitative and qualitative methods, to assess the perceived safety of 15 interactive motions executed by a soft humanoid robotic hand. We analyzed the characteristics of safe interactive motions, the primary factors influencing user safety assessments, and the impact of motion semantic clarity, user technical acceptance, and risk tolerance level on safety perception. Based on the analyzed characteristics, we summarize vital insights to provide valuable guidelines for designing safe, interactive motions in HSRI. The current results may pave the way for developing future soft machines that can safely interact with humans and their surroundings.

The humanoid robots as well as the 5-finger robot hand have gradually appeared in life with more and more functions.This paper presents a design of system with a sensory glove and software to simulate fully movements of humanoid robot hand with the spread and flexion movements of the fingers.The kinematic problem and 3D model of the robot hand with 22 rotational degrees of freedom is built in this study according to the structure of the human hand.The glove uses 17 sensors of GY-521 6DOF IMU MPU6050 with the main board of Arduino mega 2560 to collect the movement data of phalanges and carpal in the real time.The measured data filtered by Kalmann filter is transmitted to the simulation software on the computer through the serial port to update the full movement of the 3D robot hand model.The first version of the sensorized glove and the software was successfully created and tested with the result showing that the 3D robot hand model followed well the human hand movement although there is still a large delay.

This article presents an innovative approach for developing the mechanical and control systems of humanoid 3Dprinted hand with fingers, based on a modular principle. The novelty is in creating the 3D printed fingers as a single assembled component and embedding the actuators and control elements, thus making it a complete independent module. The new approach allows the implementation of the same software and actuating components to be used in finger modules with different individual sizes and joint constraints. The mechanical and control system of the hand is developed and a working prototype is created. It is described how to adjust and control the position of fingers with different sizes and joint constraints. The communication of the modules with the developed software is described. The repeatability of finger movement is studied and the force that each finger is capable of exerting during folding is measured. Functional experiments are performed and discussed. terms-Humanoid hand, control system, communication, 3D printing, modular design.

摘要： 仿人手在抓取物体时需具备运动自适应性，在与非结构环境交互时需体现出良好的柔顺性和抓取刚度。为此，借鉴人类手指天然的刚软柔性结构及抓取特征，提出基于抓取刚度增强的刚软耦合仿人手设计制造方法。首先，提出仿人手指的刚软耦合设计原理及手指参数选择方法，并将其扩展应用于多关节手指情况。其次，研究手指柔性体在抓取刚度增强前后外力作用下的变形情况，以实现两指夹爪的稳定抓取。在此基础上，提出刚软耦合仿人手指的多材料分层制造方法，并进行仿人手的总体结构设计。最后，开展刚软耦合仿人手抓取实验研究，验证所提方法的有效性，并进一步讨论刚软耦合手指的其他潜在应用方式。

The robots with humanoid hands can take the place of humans in carrying objects and using tools efficiently in different situations. This paper presents a novel hydraulic-driven robotic hand named the WLRH-II (the second-generation wheel-legged robot humanoid hand) which bears heavy load and has strong robustness. This paper focuses on the design of humanoid hand driving and transmission mechanisms, kinematic and static analyses. First, the linkage mechanism is proposed according to the particle swarm optimization algorithm. Then, the kinematics and the statics of WLRH-II are analyzed in grasp state. In addition, the hydraulic servo control method is used to control the position of a hand through angle feedback. Finally, position tracking and load experiments are completed to verify the reasonableness and effectiveness of the control. The robotic hand can grasp objects of 30[Formula: see text]kg with 2.85-kg self-weight. The WLRH-II has a high load/self-weight ratio of 10.55 in this paper, which is rare in similar humanoid robotic hands.

In this paper, a new socially assistive robot (SARs) called HBS-1.2 is presented, which uses 6-ply twisted and coiled polymer (TCP) artificial muscles in its hand to perform physical tasks. The utilization of 6-ply TCP artificial muscles in a humanoid robot hand is a pioneering advancement, offering cost effective, lightweight, and compact solution for SARs. The robot is designed to provide safer human–robot interaction (HRI) while performing physical tasks. The paper explains the procedures for fabrication and testing of the 6-ply TCP artificial muscles, along with improving the actuation response by using a Proportional-Integral-Derivative (PID) control method. Notably, the robot successfully performed a vision-based pick and place experiment, showing its potential for use in homecare and other settings to assist patients who suffer from neurological diseases like Alzheimer’s disease. The study also found an optimal light intensity range between 34 to 108 lumens/m2, which ensures minimal variation in calculated distance with 95% confidence intervals for robust performance from the vison system. The findings of this study have important implications for the development of affordable and accessible robotic systems to support elderly patients with dementia, and future research should focus on further improving the use of TCP actuators in robotics.

The core capabilities of soft grippers/soft robotic hands are grasping and manipulation. At present, most related research often improves the grasping and manipulation performance by structural design. When soft grippers rely on compressive force and friction to achieve grasping, the influence of the surface microstructure is also significant. Three types of fingerprint-inspired textures with relatively regular patterns were prepared on a silicone rubber surface via mold casting by imitating the three basic shapes of fingerprint patterns (i.e., whorls, loops, and arches). Tribological experiments and tip pinch tests were performed using fingerprint-like silicone rubber films rubbing against glass in dry and lubricated conditions to examine their performance. In addition to the textured surface, a smooth silicone rubber surface was used as a control. The results indicated that the coefficient of friction (COF) of the smooth surface was much higher than that of films with fingerprint-like textures in dry and water-lubricated conditions. The surface with fingerprint-inspired textures achieved a higher COF in oil-lubricated conditions. Adding the fingerprint-like films to the soft robotic fingers improved the tip pinch gripping performance of the soft robotic hand in lubricated conditions. This study demonstrated that the surface texture design provided an effective method for regulating the grasping capability of humanoid robotic hands.

Nowadays, there is an urgent need for humanoid robots containing human finger-like electronic skins with mechanical endurance and tactile perception. This study reports the development of an ionotronic skin-based humanoid robot hand that can recognize objects precisely through finger tapping or touching. The ionotronic skin is composed of a cytoskeleton-like filament network structure and possesses mechanical properties highly akin to human skins, including softness (Young’s modulus of 51 ± 15 MPa), toughness (1.6 ± 0.7 MJ m–3), and antifatigue-fracture ability. In addition, the i-skin functions as a triboelectric nanogenerator with the ability to perceive the triboelectric signals of an object when in contact with it. By combining triboelectric sensing information, machine learning, and Internet of Things techniques, the humanoid robot hand can accurately recognize different materials among a diverse set of spherical objects and further deliver them to the designated location. The high sorting success rate of 97.2% in 600 tests of recognizing five types of spherical objects, together with the outstanding mechanical and environmental tolerance, allow such humanoid robot hands to be used for intelligent sorting, automatic operation, and assembly in unmanned factories, as well as for the classification of garbage and hazardous materials.

PurposeTo identify the dexterity of spacesuit gloves, they need to undergo bending tests in the development process. The ideal way is to place a humanoid robotic hand into the spacesuit glove, mimicking the motions of a human hand and measuring the bending angle/force of the spacesuit glove. However, traditional robotic hands are too large to enter the narrow inner space of the spacesuit glove and perform measurements. This paper aims to design a humanoid robot hand that can wear spacesuit gloves and perform measurements.Design/methodology/approachThe proposed humanoid robotic hand is composed of five modular fingers and a parallel wrist driven by electrical linear motors. The fingers and wrist can be delivered into the spacesuit glove separately and then assembled inside. A mathematical model of the robotic hand is formulated by using the geometric constraints and principle of virtual work to analyze the kinematics and statics of the robotic hand. This model allows for estimating the bending angle and output force/torque of the robotic hand through the displacement and force of the linear motors.FindingsA prototype of the robotic hand, as well as its testing benches, was constructed to validate the presented methods. The experimental results show that the whole robotic hand can be transported to and assembled in a spacesuit glove to measure the motion characteristics of the glove.Originality/valueThe proposed humanoid robotic hand provides a new method for wearing and measuring the spacesuit glove. It can also be used to other gloves for special protective suits that have highly restricted internal space.

A tactile sensor array is a crucial component for applying physical sensors to a humanoid robot. This work focused on developing a palm-size tactile sensor array (56.0 mm × 56.0 mm) to apply object recognition for the humanoid robot hand. This sensor was based on a PCB technology operating with the piezoresistive principle. A conductive polymer composites sheet was used as a sensing element and the matrix array of this sensor was 16 × 16 pixels. The sensitivity of this sensor was evaluated and the sensor was installed on the robot hand. The tactile images, with resolution enhancement using bicubic interpolation obtained from 20 classes, were used to train and test 19 different DCNNs. InceptionResNetV2 provided superior performance with 91.82% accuracy. However, using the multimodal learning method that included InceptionResNetV2 and XceptionNet, the highest recognition rate of 92.73% was achieved. Moreover, this recognition rate improved when the object exploration was applied to demonstrate.

Abstract A novel grasp optimization algorithm for minimizing the net energy utilized by a five-fingered humanoid robotic hand with twenty degrees of freedom for securing a precise grasp is presented in this study. The algorithm utilizes a compliant contact model with a nonlinear spring and damper system to compute the performance measure, called ‘Grasp Energy’. The measure, subject to constraints, has been minimized to obtain locally optimal cartesian trajectories for securing a grasp. A case study is taken to compare the analytical (applying the optimization algorithm) and the simulated data in MSC.Adams$^{^{\circledR}}$, to prove the efficacy of the proposed formulation.