Dexterous Operation Research Articles

Robotic Dexterous Hands’ Structure Design and Drive-Transmission System

The robotic dexterous hand is one of the most important components of robots. As a new type of end-effector, it can help the robot to interact with the environment, enabling the function of grasping and dexterous operation to objects in various situations such as daily life, medical treatment and industry. This paper investigates the development history and current situation of the robotic dexterous hand in the world, conducts the research and comparative analysis focusing on the characteristics of the existing robotic dexterous hands’ mechanical transmission and drive system, comprehensively obtains its development trend, and finally draws the conclusion and makes the prospect of the future development of the robotic dexterous hand.

Does static friction information predict the onset of sliding for soft material?

Friction behavior between soft and hard materials has long been a question of great interest in the fields of artificial joints, human skin contact, robotic grippers, and others. In this study, we presented a combination of theoretical and experimental analyses to investigate the friction behavior of soft materials. Using some geometric and stick–slip features in the early stage of static friction, the friction property of the interface between a soft material and a hard material is determined. Moreover, the onset of slip, the threshold force at which a friction interface begins to slide, is also predicted by a theoretical model. The predictive ability of this model may provide insights for improving interaction property recognition, as well as for developing fine tactile feedback and dexterous operation of robotic grasping.

Differential-algebraic equation-based model predictive control for trajectory tracking of spacecraft-mounted continuum manipulators

Spacecraft-mounted continuum manipulators (SMCMs) exhibit great potential for performing dexterous operations in unstructured environments due to their inherent compliance and dexterity. However, the dynamic model of the rigid-flexible coupling SMCM is highly nonlinear and typically formulated as a set of implicit differential-algebraic equations (DAEs), posing significant challenges for precise trajectory tracking control. This paper proposes a novel model predictive control (MPC) framework specifically designed for generic DAEs to achieve precise trajectory tracking of the SMCM under uncertain disturbances and input limitations. The DAE model of the SMCM is discretized into a set of nonlinear algebraic equations. By performing implicit differentiation of these equations with respect to the system state, the state transition matrix (STM) for the DAE model is derived. The optimal control action for the SMCM can be further determined based on the derived STM. Additionally, nonlinear complementary functions are introduced to address the issue of input limitations, allowing the problem of determining the optimal control sequence to be equivalently transformed into a set of nonlinear algebraic equations for solving. Numerical simulations demonstrate that the proposed approach can achieve precise trajectory tracking of the SMCM while strictly adhering to input limitations.

Behavioral learning of dish rinsing and scrubbing based on interruptive direct teaching considering assistance rate

Robots are expected to manipulate objects in a safe and dexterous way. For example, washing dishes is a dexterous operation that involves scrubbing the dishes with a sponge and rinsing them with water. It is necessary to learn it safely without splashing water and without dropping the dishes. In this study, we propose a safe and dexterous manipulation system. The robot learns a dynamics model of the object by estimating the state of the object and the robot itself, the control input, and the amount of human assistance required (assistance rate) after the human corrects the initial trajectory of the robot's hands by interruptive direct teaching. By backpropagating the error between the estimated and the reference value using the acquired dynamics model, the robot can generate a control input that approaches the reference value, for example, so that human assistance is not required and the dish does not move excessively. This allows for adaptive rinsing and scrubbing of dishes with unknown shapes and properties. As a result, it is possible to generate safe actions that require less human assistance.

A Compact Aerial Manipulator: Design and Control for Dexterous Operations

The lack of aerial physical interaction capability is one of the choke points limiting the extension of aerial robot applications, such as rescue missions and aerial maintenance. We present a new aerial robotic manipulator (AEROM) for aerial dexterous operations in this work. It contains a robotic manipulator with 6-degree-of-freedom and a compact flight platform. Firstly, we propose a quantitative capability index to evaluate and guide the mechanical design of the AEROM. Based on the proposed quantitative index, we construct a lightweight bird-inspired manipulator to imitate a raptor hindlimb. An additional telescopic joint and an end-effector consisting of three soft fingers allow the AEROM to execute aerial interaction tasks. In addition, the wrist joints enable independent control of the end-effector attitude regardless of the flight platform. After explicitly analyzing the multi-source disturbances during the aerial operation tasks, we develop a refined anti-disturbance controller to compensate for the disturbances with different characteristics. The proposed controller further improves the position accuracy of end-effector to enable dexterous operations during aerial interaction tasks. Finally, the physical experiments verify the effectiveness of the proposed AEROM system.

Piston-like particle jamming for enhanced stiffness adjustment of soft robotic arm

PurposeStiffness adjusting ability is essential for soft robotic arms to perform complex tasks. A soft state enables dexterous operation and safe interaction, while a rigid state enables large force output or heavy weight carrying. However, making a compact integration of soft actuators with powerful stiffness adjusting mechanisms is challenging. This study aims to develop a piston-like particle jamming mechanism for enhanced stiffness adjustment of a soft robotic arm.Design/methodology/approachThe arm has two pairs of differential tendons for spatial bending, and a jamming core consists of four jamming units with particles sealed inside braided tubes for stiffness adjustment. The jamming core is pushed and pulled smoothly along the tendons by a piston, which is then driven by a motor and a ball screw mechanism.FindingsThe tip displacement of the arm under 150 N jamming force and no more than 0.3 kg load is minimal. The maximum stiffening ratio measured in the experiment under 150 N jamming force is up to 6–25 depends on the bending direction and added load of the arm, which is superior to most of the vacuum powered jamming method.Originality/valueThe proposed robotic arm makes an innovative compact integration of tendon-driven robotic arm and motor-driven piston-like particle jamming mechanism. The jamming force is much larger compared to conventional vacuum-powered systems and results in a superior stiffening ability.

Multimodal Sensors Enabled Autonomous Soft Robotic System with Self-Adaptive Manipulation.

Human hands are amazingly skilled at recognizing and handling objects of different sizes and shapes. To date, soft robots rarely demonstrate autonomy equivalent to that of humans for fine perception and dexterous operation. Here, an intelligent soft robotic system with autonomous operation and multimodal perception ability is developed by integrating capacitive sensors with triboelectric sensor. With distributed multiple sensors, our robot system can not only sense and memorize multimodal information but also enable an adaptive grasping method for robotic positioning and grasp control, during which the multimodal sensory information can be captured sensitively and fused at feature level for crossmodally recognizing objects, leading to a highly enhanced recognition capability. The proposed system, combining the performance and physical intelligence of biological systems (i.e., self-adaptive behavior and multimodal perception), will greatly advance the integration of soft actuators and robotics in many fields.

Continuous adaptive gaits manipulation for three-fingered robotic hands via bioinspired fingertip contact events

The remarkable skill of changing its grasp status and relocating its fingers to perform continuous in-hand manipulation is essential for a multifingered anthropomorphic hand. A commonly utilized method of manipulation involves a series of basic movements executed by a high-level controller. However, it remains unclear how these primitives evolve into sophisticated finger gaits during manipulation. Here, we propose an adaptive finger gait-based manipulation method that offers real-time regulation by dynamically changing the primitive interval to ensure the force/moment balance of the object. Successful manipulation relies on contact events that act as triggers for real-time online replanning of multifinger manipulation. We identify four basic motion primitives of finger gaits and create a heuristic finger gait that enables the continuous object rotation of a round cup. Our experimental results verify the effectiveness of the proposed method. Despite the constant breaking and reengaging of contact between the fingers and the object during manipulation, the robotic hand can reliably manipulate the object without failure. Even when the object is subjected to interfering forces, the proposed method demonstrates robustness in managing interference. This work has great potential for application to the dexterous operation of anthropomorphic multifingered hands.

Kinematic Analysis of Multi-Section Opposite Magnetic Catheter Robots With Solution Multiplicity

Magnetic catheter robots have great potential for application in minimally invasive surgical interventions that require submillimeter soft catheters to be inserted into the body. Compared to other medical continuum robots, the kinematic characteristics of the magnetic continuum robot are required to be improved, including operating dexterity. The opposite magnetic continuum robot (OMCR), which has oppositely-magnetized magnets on the robot, is capable of deforming in large and high-order curvatures and shows improved dexterity. However, unstable kinematic behaviors emerge owing to solution multiplicity and affect the actual operation. For example, snapping, which indicates the sudden jump from one state to another, induces danger for medical operations. In addition, the motion sequence of the external mobile magnet affects the final state of OMCR. Therefore, in this study, we first analyze the reason for these emerging behaviors by understanding the solution multiplicity of an OMCR. We then modified the OMCR kinematic model with instability analysis and additional constraints, to predict the critical point for snapping and avoid other unexpected behavior for path planning. Herein, our study aims to analyze the stability of the OMCR with relatively unstable configurations in their actuation space to ensure stable and dexterous motions for medical applications. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The motivation of this work is to analyze, predict, and avoid the unexpected behaviors of the medical magnetic catheter robots, and ensure stable and dexterous operation. Submillimeter magnetic catheter robots have received considerable attention in the past ten years owing to their small size and dexterous motion. They have great potential for minimally invasive surgical interventions that require the navigation of “tighter corners” inside the human body. However, there may exist multiple local solutions under an external field, especially when the robot deforms in high-order curvature for dexterous operation. This situation induces unexpected behaviors and danger for medical use, such as a sudden jump from one state to another. Herein, we analyze the solution evolution as the external field varies and predict the unexpected behaviors using a modified kinematic model with instability analysis. Preliminary experiments demonstrated the effectiveness of the modified model, which can predict unexpected behaviors, and evaluate and enhance the stability during path planning. This method can be implemented for the dexterous and stable operation of magnetic catheter robots, which is significant for medical applications.

Toward Human-Like Grasp: Functional Grasp by Dexterous Robotic Hand Via Object-Hand Semantic Representation.

Intelligent robotic manipulation is a challenging study of machine intelligence. Although many dexterous robotic hands have been designed to assist or replace human hands in executing various tasks, how to teach them to perform dexterous operations like human hands is still a challenge. This motivates us to conduct an in-depth analysis of human behavior in manipulating objects and propose an object-hand manipulation representation. This representation provides an intuitive and clear semantic indication of how the dexterous hand should touch and manipulate an object based on the object's own functional areas. At the same time, we propose a functional grasp synthesis framework, which does not require real grasp label supervision, but relies on the guidance of our object-hand manipulation representation. In addition, in order to obtain better functional grasp synthesis results, we propose a network pre-training method that can make full use of easily obtained stable grasp data, and a network training strategy to coordinate the loss functions. We conduct object manipulation experiments on a real robot platform, and evaluate the performance and generalization of our object-hand manipulation representation and grasp synthesis framework. The project website is: https://github.com/zhutq-github/Toward-Human-Like-Grasp-V2-.

RFCT: Multimodal Sensing Enhances Grasping State Detection for Weak-Stiffness Targets

Accurate grasping state detection is critical to the dexterous operation of robots. Robots must use multiple modalities to perceive external information, similar to humans. The direct fusion method of visual and tactile sensing may not provide effective visual–tactile features for the grasping state detection network of the target. To address this issue, we present a novel visual–tactile fusion model (i.e., RFCT) and provide an incremental dimensional tensor product method for detecting grasping states of weak-stiffness targets. We investigate whether convolutional block attention mechanisms (CBAM) can enhance feature representations by selectively attending to salient visual and tactile cues while suppressing less important information and eliminating redundant information for the initial fusion. We conducted 2250 grasping experiments using 15 weak-stiffness targets. We used 12 targets for training and three for testing. When evaluated on untrained targets, our RFCT model achieved a precision of 82.89%, a recall rate of 82.07%, and an F1 score of 81.65%. We compared RFCT model performance with various combinations of Resnet50 + LSTM and C3D models commonly used in grasping state detection. The experimental results show that our RFCT model significantly outperforms these models. Our proposed method provides accurate grasping state detection and has the potential to provide robust support for robot grasping operations in real-world applications.

The IFMIF-DONES remote handling control system: Experimental setup for OPC UA integration

The devices used to carry out Remote Handling (RH) manipulation tasks in radiation environments address requirements that are significantly different from common robotic and industrial systems due to the lack of repetitive operations and incompletely specified control actions. This imposes the need of control with human-in-the-loop operations. These RH systems are used on facilities such PRIDE, CERN, ESS, ITER or IFMIF-DONES, the reference used for this work.For the RH system is crucial to provide high availability, robustness against radiation, haptic devices for teleoperation and dexterous operation, and smooth coordination and integration with the centralized control room. To achieve this purpose is necessary to find the best approach towards a standard control framework capable of providing a standard set of functionalities, tools, interfaces, communications, and data formats to the different types of mechatronic devices that are usually considered for Remote Handling tasks. This previous phase of homogenization is not considered in most facilities, which leads towards a costly integration process during the commissioning phase of the facility.In this paper, an approach to the IFMIF-DONES RH Control framework with strong standard support based on protocols such as OPC UA has been described and validated through an experimental setup. This test bench includes a set of physical devices (PLC, conveyor belt and computers) and a set of OPC UA compatible software tools, configured and operable from any node of the University of Granada network. This proof-of-concept mockup provides flexibility to modify the dimension and complexity of the setup by using new virtual or physical devices connected to a unique backbone. Besides, it will be used to test different aspects such as control schemes, failure injection, network modeling, predictive maintenance studies, operator training on simulated/real scenarios, usability or ergonomics of the user interfaces before the deployment. In this contribution, the results are described and illustrated using a conveyor belt set-up, a small but representative reference used to validate the RH control concepts here proposed.

FC-EODR: Immersive Humanoid Dual-Arm Dexterous Explosive Ordnance Disposal Robot

In this study, we proposes a humanoid dual-arm explosive ordnance disposal (EOD) robot design. First, a seven-degree-of-freedom high-performance collaborative and flexible manipulator is developed, aiming at the transfer and dexterous operation of dangerous objects in EOD tasks. Furthermore, an immersive operated humanoid dual-arm dexterous explosive disposal robot (FC-EODR) is designed, which has a high passability to complex terrains such as low walls, slope roads, and stairs. It can remotely detect, manipulate, and remove explosives in dangerous environments through immersive velocity teleoperation. In addition, an autonomous tool-changing system is constructed, which enables the robot to flexibly switch between different tasks. The effectiveness of the FC-EODR is finally verified through a series of experiments, including the platform performance test, manipulator load test, teleoperated wire trimming, and screw-screwing experiments. This letter provides the technical foundation for robots to replace humans in EOD tasks and emergency situations.

A Handheld Steerable Surgical Drill With a Novel Miniaturized Articulated Joint Module for Dexterous Confined-Space Bone Work.

Steerable surgical drills have the potential to minimize intraoperative tissue damage to patients. However, due to their large shaft diameters, large bend radii, and small bend angles, existing steerable drills are unsuitable for dexterous operations in confined spaces. This article presents a handheld steerable drill with a ϕ4.5-mm miniaturized tip, capable of abruptly bending up to ±65°. To achieve a small tip diameter and a large bend angle, we propose a novel articulated joint module composed of a tendon-driven geared rolling joint and a double universal joint for steering the drill shaft and transmitting drilling torques, respectively. We integrate this joint module with a customized compact actuation unit into a handheld device. The integrated handheld steerable drill is slim and lightweight, supporting burdenless, single-handed grips and easy integration into existing surgical procedures. Experiments and analysis showed the proposed steerable drill has high distal dexterity and is capable to remove target tissues dexterously through a small passage/incision with minimized collateral damage. The results suggest the potential of the proposed miniaturized articulated drill for dexterous bone work in confined spaces. By enhancing distal dexterity and reach for surgeons when dealing with hard bony tissues, the proposed device can potentially minimize surgical invasiveness and thus collateral tissue damage to patients for a better clinical outcome.

2.5-mm articulated endoluminal forceps using a compliant mechanism.

Gastrointestinal cancer can be treated using a flexible endoscope through a natural orifice. However, treatment instruments with limited degrees of freedom (DOFs) require a highly skilled operator. Articulated devices useful for endoluminal procedures, such as endoscopic submucosal dissection and biopsy, have been developed. These devices enable dexterous operation in a narrow lumen; however, they suffer from limitations such as large size and high cost. To overcome these limitations, we developed a 2.5-mm articulated forceps that can be inserted into a standard endoscope channel based on a compliant mechanism. The compliant mechanism allows the device to be compact and affordable, which is possible due to its monolithic structure. The proposed mechanism consists of two segments, 1-DOF grasping and 2-DOF bending, that are actuated by tendon-sheath mechanisms. A prototype was designed based on finite element analysis results. To confirm the effectiveness of the proposed mechanism, we fabricated the prototype using a 3D printer. A series of mechanical performance tests on the prototype revealed that it achieved the following specifications: (1) DOF: 1-DOF grasping + 2-DOF bending, (2) outer diameter: 2.5 mm, (3) length of the bending segment: 30 mm, and (4) range of motion: [Formula: see text] to [Formula: see text] (grasping) and [Formula: see text] to [Formula: see text] (bending). Finally, we performed a tissue manipulation test on an excised porcine colon and found that a piece of mucous membrane tissue was successfully resected using an electric knife while being lifted with the developed forceps. The results of the evaluation experiment demonstrated a positive feasibility of the proposed mechanism, which has a simpler structure compared to those of other conventional mechanisms; furthermore, it is potentially more cost-effective and is disposable. The mechanical design, prototype implementation, and evaluations are reported in this paper.

Energy-aware redundant actuation for safe spring-assisted modular and reconfigurable robot

AbstractA spring-assisted modular and reconfigurable robot (SA-MRR) has been recently developed at our laboratory to reinforce its performance and enable safe and dexterous operation in human–robot environments. Multiple working mode (MWM) control enables each SA-MRR joint module to switch independently between working in a primary actuation mode and a secondary, spring-assisted mode that may improve task-specific energy performance measures and safety in a variety of manipulation tasks. The spring-assisted mode is characterized by synergy of spring and motor energy and may be summoned to offset motor energy demands or to safeguard a reconfigurable set of secondary joint limits. In this research work, two spring-assisted working mode strategies are proposed, and their characteristics have been investigated for SA-MRR actuation energy advantages while safe robot segregation in collaboration tasks is maintained. One MWM strategy has been designed to safeguard task-specific joint limits and is able to decrease motor energy consumption in some tasks. Another MWM strategy has been designed for energy efficiency and was able to reduce motor energy per cycle by $ \text{72}$ % in a simulated manipulation task while maintaining spatial safety constraints. Numerical simulations have demonstrated the effectiveness of the proposed spring-assisted working mode strategies for energy-aware safe manipulation applications.

BIT-DMR: A Humanoid Dual-Arm Mobile Robot for Complex Rescue Operations

Using robots to assist or even replace rescuers for searching and rescuing has always been a research hotspot. Robots that can carry out dexterous operations at the scene are of great significance to reduce the life threat of rescue workers. Aiming at the characteristics of narrow space and frequent physical interaction in rescue scene, a dual-arm mobile robot BIT-DMR composed of humanoid upper body and tracked mobile platform is developed in this letter. The main work is as follows: a) A high-performance lightweight and heavy payload human-machine cooperative manipulator is developed, which has a slender shape and a large flexible workspace, improving the operation ability of the robot in complex rescue environment. b) A highly integrated tracked mobile platform is developed, which can change a variety of forms leading to a high trafficability in complex terrain. The flexibility of BIT-DMR to carry out rescue operation is verified by a series of experiments like payload testing, mobile platform performance, obstacle removing, door opening and casualty rescuing. This letter provides technical reserves for robots to assist or replace human beings to carry out rescue operations at the scene.

Joint Modeling and Closed-Loop Control of a Robotic Hand Driven by the Tendon-Sheath

Robotic dexterous operation is a long-term challenging problem. Although recent advances in robotic hand have successfully realized some complex operations, its high dexterity and high precision control are still challenges for robotic hands to replace the human hand. Here, this letter developed a tendon-sheath transmission-based robotic hand with decoupling joint design and high-precision closed-loop control strategy. Firstly, the designed hand is introduced, and the nonlinear joint transmission model is established. Then, the transfer function of the joint is established based on the dynamic model, and an adaptive controller with neural network compensation is designed to realize the high-precision joint control. The simulation and experiment results show that it is feasible to apply the tendon-sheath transmission method to the robotic hand, and the proposed closed-loop control strategy can significantly improve the joint control accuracy. This research opens new prospects for the novel tendon-sheath driving method applied in the design of robotic hand, which is expected to have a long-term impact for the design and control of dexterous hand.

Trigger-Based Dexterous Operation with Multimodal Sensors for Soft Robotic Hand

This paper focuses on how to improve the operation ability of a soft robotic hand (SRH). A trigger-based dexterous operation (TDO) strategy with multimodal sensors is proposed to perform autonomous choice operations. The multimodal sensors include optical-based fiber curvature sensor (OFCS), gas pressure sensor (GPS), capacitive pressure contact sensor (CPCS), and resistance pressure contact sensor (RPCS). The OFCS embedded in the soft finger and the GPS series connected in the gas channel are used to detect the curvature of the finger. The CPCS attached on the fingertip and the RPCS attached on the palm are employed to detect the touch force. The framework of TDO is divided into sensor detection and action operation. Hardware layer, information acquisition layer, and decision layer form the sensor detection module; action selection layer, actuator drive layer, and hardware layer constitute the action operation module. An autonomous choice decision unit is used to connect the sensor detecting module and action operation module. The experiment results reveal that the TDO algorithm is effective and feasible, and the actions of grasping plastic framework, pinching roller ball pen and screwdriver, and handshake are executed exactly.

Centrosymmetric‐ and Axisymmetric‐Patterned Flexible Tactile Sensor for Roughness and Slip Intelligent Recognition

Next‐generation robots are being designed to function autonomously in complex and unstructured environments. In particular, based on the real‐time measurement and differentiation of normal pressure and shear force, robots can be equipped with the capabilities of damage‐free grasp within minimum force limits, as well as dexterous operation through surface roughness and slip information. Herein, a flexible tactile sensor with a small cylinder protrusion and four arc‐shaped protrusions is developed. Due to its center symmetry and axisymmetry characteristics, the normal pressure and shear force can be decoupled from the complex stress without any interference from torsion. The flexible tactile sensor exhibits good linearity and superior cycling stability and is capable of determining the magnitude and direction of the applied force accurately. The flexible tactile sensor is comfortable to wear, and it is integrated onto the manipulator to realize various delicate and dexterous tasks, such as pressure detection, interaction with fragile objects, and roughness identification. Moreover, intelligent recognition of the sliding and stationary states can be achieved by decoding signals of sliding friction and static friction from the feedback information, leading to real time and precise adjustment of the grasping state of the manipulator.