Dexterous Manipulation Research Articles

Evaluation of (Shared) Autonomy in Robot-Assisted Vitreoretinal Surgery Using a Surgical Model.

Robot-assisted vitreoretinal surgery makes it easier for the surgeons to perform precise and dexterous manipulations required in vitreoretinal procedures. We systematically evaluated manual surgery, conventional two-hand teleoperation, a novel one-hand teleoperation, and automation in a needle positioning task using a realistic surgical eye model, measuring the expert surgeon's performances and the novice's learning curves. The proposed one-hand teleoperation improved the positioning accuracy of expert surgeons , enabled novices to achieve a consistent accuracy more quickly , decreased the novice's workload more quickly , and made it easier for novices to learn to conduct the task quickly . Moreover, our autonomous positioning achieved an equivalent accuracy to the surgeons. The benefits and potential of task autonomy were shown. Further work is needed to evaluate the proposed methods in a more complex task.

A dexterous and compliant aerial continuum manipulator for cluttered and constrained environments

Aerial manipulators can manipulate objects while flying, allowing them to perform tasks in dangerous or inaccessible areas. Advanced aerial manipulation systems are often based on rigid-link mechanisms, but the balance between dexterity and payload capacity limits their broader application. Combining unmanned aerial vehicles with continuum manipulators emerges as a solution to this trade-off, but these systems face challenges with large actuation systems and unstable control. To address these challenges, we propose Aerial Elephant Trunk, an aerial continuum manipulator inspired by the elephant trunk, featuring a small-scale quadrotor and a dexterous, compliant tendon-driven continuum arm for versatile operation in both indoor and outdoor settings. We develop state estimation for the quadrotor using an Extended Kalman Filter, shape estimation for the continuum arm based on piecewise constant curvature, and whole-body motion planning using minimum jerk principles. Through comprehensive fundamental verifications, we demonstrate that our system can adapt to various constrained environments, such as navigating through narrow holes, tubes, or crevices, and can handle a range of objects, including slender, deformable, irregular, or heavy items. Our system can potentially be deployed in challenging conditions, such as pipeline maintenance or electricity line inspection at high altitudes.

Communicating women’s rights: the relevance of the Ofiori-ndọ oral artist in traditional marriages of the Efik of Cross River, in Nigeria

ABSTRACT Efforts from female voices in propagating women’s rights and raising the society’s consciousness towards female capabilities is fast attaining volume and recognition in many traditional African societies. In this paper, we will argue that the spoken art rendition of the Ofiori-ndọ (translated loosely as ‘marriage-crier’) in Efik marriages is a heightened voicing in female rights activism, leadership, rhetoric, sublimity and stage energy as it interrogates the roles the female oral artist plays in Efik marriage contexts. With many dimensions to its delivery, this art which is an exclusive preserve of women echoes elements of entertainment, and profitable instruction via the dexterous manipulation of indigenous language resources. The study pursues its task via a qualitative research design which constitutes library and field work. Data was entirely sourced from semi-structured interviews, close observation, conversations and audio-visual recordings. The study submits that the Ofiori-ndọ is crucial in Efik marriages as her art fulfills the ethics and customs of Efik marriages.

Versatile adhesive skin enhances robotic interactions with the environment.

Electronic skins endow robots with sensory functions but often lack the multifunctionality of natural skin, such as switchable adhesion. Current smart adhesives based on elastomers have limited adhesion tunability, which hinders their effective use for both carrying heavy loads and performing dexterous manipulations. Here, we report a versatile, one-size-fits-all robotic adhesive skin using shape memory polymers with tunable rubber-to-glass phase transitions. The adhesion strength of our adhesive skin can be changed from minimal (~1 kilopascal) for sensing and handling ultralightweight objects to ultrastrong (>1 megapascal) for picking up and lifting heavy objects. Our versatile adhesive skin is expected to greatly enhance the ability of intelligent robots to interact with their environment.

Highly Sensitive Linear Triaxial Force Sensor Based on Multimodal Sensing for 3D Pose Reconstruction

AbstractFlexible sensing offers real‐time force monitoring, presenting a versatile and effective solution for dexterous manipulation, healthcare, environmental exploration, and perception of physical properties. Nonetheless, a limitation of many existing flexible force sensors stems from their isotropic structure or material properties, preventing them from simultaneously detecting both the direction and magnitude of the applied force. Herein, a high‐performance 3D force sensor based on orthogonal multimodal sensing, the cancellation principle, and the strain effect is proposed. Finite element analysis further reveals the decoupling and anti‐interference mechanisms of the innovative capacitor‐resistance dual‐mode sensing based on a solid mechanics and electrostatic multiphysics model. The sensor demonstrates the ability to measure both the magnitude and direction of normal and shear forces in any combination using the proposed decoupling and reconstruction algorithms, showing the potential for accurately reconstructing the posture of objects.

The Study of Dexterous Hand Manipulation: A Synergy-Based Complexity Index

The Study of Dexterous Hand Manipulation: A Synergy-Based Complexity Index

TARS: Tactile Affordance in Robot Synesthesia for Dexterous Manipulation

TARS: Tactile Affordance in Robot Synesthesia for Dexterous Manipulation

Human-like dexterous manipulation for anthropomorphic five-fingered hands: A review

Human-like dexterous manipulation for anthropomorphic five-fingered hands: A review

Human Collaborative Control of Lower-Limb Prosthesis Based on Game Theory and Fuzzy Approximation.

For leg prosthesis user, the soft tissue and skin under the stump of are not accustomed to weight bearing, excessive continuous contact pressure can lead to the risk of degenerative tissue ulceration. This article presents a novel human-robot collaborative control scheme that achieves control weight self-adjustment for robotic prostheses to minimize interaction torque. To establish the human-robot interaction relationship, we regard the contact pressure between human residual limb and the prosthetic receiving cavity as the interaction force. We aim at reducing the interaction force under the premise of minimally changing the original motion trajectory of the robotic prosthesis. The control scheme mainly includes trajectory optimization based on a dual-agent game control scheme under a cooperative relationship, and a fuzzy logic system for improving the control accuracy of trajectory tracking of robotic prostheses with unknown dynamic parameters. Experiments were carried out on two amputee participants to verify the proposed human-robot interactive control scheme in a robotic prosthesis. The results show that the interaction torque could be reduced while maintaining minimal trajectory tracking error. The proposed control scheme could potentially facilitate the dexterous manipulation of leg prostheses, thus benefiting amputees.

Recent Progress in Tactile Sensing and Machine Learning for Texture Perception in Humanoid Robotics

ABSTRACTHumanoid robots have garnered substantial attention recently in both academia and industry. These robots are becoming increasingly sophisticated and intelligent, as seen in health care, education, customer service, logistics, security, space exploration, and so forth. Central to these technological advancements is tactile perception, a crucial modality through which humanoid robots exchange information with their external environment, thereby facilitating human‐like behaviors such as object recognition and dexterous manipulation. Texture perception is particularly vital for these tasks, as the surface morphology of objects significantly influences recognition and manipulation abilities. This review addresses the recent progress in tactile sensing and machine learning for texture perception in humanoid robots. We first examine the design and working principles of tactile sensors employed in texture perception, differentiating between touch‐based and sliding‐based approaches. Subsequently, we delve into the machine learning algorithms implemented for texture perception using these tactile sensors. Finally, we discuss the challenges and future opportunities in this evolving field. This review aims to provide insights into the state‐of‐the‐art developments and foster advancements in tactile sensing and machine learning for texture perception in humanoid robotics.

A High-Repeatability Three-Dimensional Force Tactile Sensing System for Robotic Dexterous Grasping and Object Recognition.

Robotic devices with integrated tactile sensors can accurately perceive the contact force, pressure, sliding, and other tactile information, and they have been widely used in various fields, including human-robot interaction, dexterous manipulation, and object recognition. To address the challenges associated with the initial value drift, and to improve the durability and accuracy of the tactile detection for a robotic dexterous hand, in this study, a flexible tactile sensor is designed with high repeatability by introducing a supporting layer for pre-separation. The proposed tactile sensor has a detection range of 0-5 N with a resolution of 0.2 N, and the repeatability error is as relatively small as 1.5%. In addition, the response time of the proposed tactile sensor under loading and unloading conditions are 80 ms and 160 ms, respectively. Moreover, a three-dimensional force decoupling detection method is developed by distributing tactile sensor units on a non-coplanar robotic fingertip. Finally, using a backpropagation neural network, the classification and recognition processes of nine types of objects with different shapes and categories are realized, achieving an accuracy higher than 95%. The results show that the proposed three-dimensional force tactile sensing system could be beneficial for the delicate manipulation and recognition for robotic dexterous hands.

TacFR-Gripper: A Reconfigurable Fin-Ray-Based Gripper with Tactile Skin for In-Hand Manipulation

This paper introduces the TacFR-Gripper, a novel reconfigurable soft robotic gripper inspired by the Fin-Ray effect and equipped with tactile skin. The gripper incorporates a four-bar mechanism for accurate finger bending and a reconfigurable design to change the relative positions between the fingers and palm, enabling precise and adaptable object grasping. This 5-Degree-of-Freedom (DOF) soft gripper can facilitate dexterous manipulation of objects with diverse shapes and stiffness and is beneficial to the safe and efficient grasping of delicate objects. An array of Force Sensitive Resistor (FSR) sensors is embedded within each robotic fingertip to serve as the tactile skin, enabling the robot to perceive contact information during manipulation. Moreover, we implemented a threshold-based tactile perception approach to enable reliable grasping without accidental slip or excessive force. To verify the effectiveness of the TacFR-Gripper, we provide detailed workspace analysis to evaluate its grasping performance and conducted three experiments, including (i) assessing the grasp success rate across various everyday objects through different finger configurations, (ii) verifying the effectiveness of tactile skin with different control strategies in grasping, and (iii) evaluating the in-hand manipulation capabilities through object pose control. The experimental results indicate that the TacFR-Gripper can grasp a wide range of complex-shaped objects with a high success rate and deliver dexterous in-hand manipulation. Additionally, the integration of tactile skin is demonstrated to enhance grasp stability by incorporating tactile feedback during manipulations.

Multi-tactile sensor calibration via motion constraints with tactile measurements

This paper addresses a fundamental problem in multi-finger robot dexterous manipulation, Multi-Tactile Sensor Calibration (MTSC). It involves estimating the relative poses between multiple tactile sensors using their intrinsic measurements, crucial for coordinating multiple fingers in a dexterous robotic hand, especially when accurate encoders are unavailable. Unlike conventional multi-sensor calibration methods (like cameras), which rely on overlapping sensing regions and feature point matching, calibrating multiple tactile sensors presents unique challenges because these sensors cannot have overlapping sensing regions, precluding the use of shared key-points visible from different sensors. In this paper, we establish the theoretical basis for the MTSC problem, constructing constraint equations based on motions measured at multiple locations on the same grasped object from different tactile sensors. The key is that all these measurements correspond to a shared rigid body motion. A calibration scheme is proposed accordingly, with theoretical analysis conducted to enhance precision. Results in simulation, with objects of different shapes, confirm the validity of the proposed calibration approach. Furthermore, calibration of the relative pose between two GelSlim vision-based tactile sensors in real experiments demonstrates good agreement with the ground truth. The proposed theory and method not only shed light on explaining the accuracy gap between proprioception and tactile sensing in multi-finger manipulation but also pave the way for tactile-encoder-mixed or encoder-free solutions in dexterous multi-finger coordination.

In good hands: A case for improving robotic dexterity.

Twenty-first-century roboticists envision robots capable of sorting objects and packaging them, of chopping vegetables and folding clothes. But although many today believe that the only factors necessary for robots to achieve dexterous manipulation are data and artificial intelligence (AI), managing all the mundane manipulations that humans perform daily requires constant adaptation to changing conditions. These operations include tasks demanding millisecond-level responses to prevent irreparable damage, such as quickly stabilizing a slipping package or halting a knife just before it cuts into a table. Although recent advances in intelligent robot control have considerably expanded-and will continue to expand-the range and complexity of tasks that robots can perform, AI, data, and control theory alone are not enough. A brain cannot manipulate items effectively without a body-and specifically hands-suited to the task.

Progressive Transfer Learning for Dexterous In-Hand Manipulation With Multifingered Anthropomorphic Hand

Progressive Transfer Learning for Dexterous In-Hand Manipulation With Multifingered Anthropomorphic Hand

Contributions of the Primary Sensorimotor Cortex and Posterior Parietal Cortex to Motor Learning and Transfer.

Transferring learned manipulations to new manipulation tasks has enabled humans to realize thousands of dexterous object manipulations in daily life. Two-digit grasp and three-digit grasp manipulations require different fingertip forces, and our brain can switch grasp types to ensure good performance according to motor memory. We hypothesized that several brain areas contribute to the execution of the new type of motor according to the motor memory. However, the motor memory mechanisms during this transfer period are still unclear. In the present functional magnetic resonance imaging (fMRI) study, we aimed to investigate the cortical mechanisms involved in motor memory during the transfer phase of learned manipulation tasks. Using a custom-built T-shaped object with an adjustable weight distribution, the participants performed grasp and lift manipulation tasks under different conditions to simulate the learning and transfer phases. The learning phase consisted of four grasp-and-lift repetitions with one motor type, followed by a transfer phase with four repetitions involving different motors (adding or removing a digit). By comparing brain activity in the learning and transfer phases, we identified three regions (the superior frontal gyrus, supramarginal gyrus, and postcentral gyrus) associated with motor memory during the transfer of learned manipulations. Our findings improve the understanding of the role of the posterior parietal cortex in motor memory, highlighting how sensory information from memory and real-time input is integrated to generate novel motor control signals that guide the precise reapplication of control strategies. Furthermore, we believe that these areas contribute to motor learning from motor memory and may serve as key regions of interest for investigating neurodegenerative diseases.

Magnetic manipulation of unknown and complex conductive nonmagnetic objects with application in the remediation of space debris

This article extends recent work in magnetic manipulation of conductive, nonmagnetic objects using rotating magnetic dipole fields. Eddy-current-based manipulation provides a contact-free way to manipulate metallic objects. We are particularly motivated by the large amount of aluminum in space debris. We previously demonstrated dexterous manipulation of solid spheres with all object parameters known a priori. This work expands the previous model, which contained three discrete modes, to a continuous model that covers all possible relative positions of the manipulated spherical object with respect to the magnetic field source. We further leverage this new model to examine manipulation of spherical objects with unknown physical parameters by applying techniques from the online-optimization and adaptive-control literature. Our experimental results validate our new dynamics model, showing that we get improved performance compared to the previously proposed model, while also solving a simpler optimization problem for control. We further demonstrate the first physical magnetic manipulation of aluminum spheres, as previous controllers were only physically validated on copper spheres. We show that our adaptive control framework can quickly acquire useful object parameters when weakly initialized. Finally, we demonstrate that the spherical-object model can be used as an approximate model for adaptive control of nonspherical objects by performing magnetic manipulation of a variety of objects for which a spherical model is not an obvious approximation.

Exploring the longitudinal course of motor dexterity in first-episode psychosis: a 10-year follow-up.

This study aimed to examine longitudinal changes in motor dexterity (MD) performance in first episode of psychosis (FEP), focusing on diagnosis-specific trajectories. Participants were recruited from the project PAFIP in Spain (134 FEP, 84 HC). MD was assessed using the Grooved Pegboard Test at baseline and at 10-year follow-up. Clinical and premorbid data were collected for the patients. FEP participants were classified as having schizophrenia (SCZ) or other psychosis (OP) and compared with a group of healthy controls (HC). MD correlated significantly with age and intelligence in all participants. MD in SCZ patients was additionally correlated with premorbid adjustment and negative symptoms, whereas in the OP group the association was with antipsychotic dose. SCZ patients showed a slight decline in MD at follow-up, whereas the OP and HC groups remained stable. There may be different long-term trajectories of MD across diagnoses in FEP patients. Early developmental factors such as premorbid adaptation and cognitive function, together with age-related changes, may influence a mild decline in MD specifically in schizophrenia.

Dexterous Manipulation Based on Object Recognition and Accurate Pose Estimation Using RGB-D Data.

This study presents an integrated system for object recognition, six-degrees-of-freedom pose estimation, and dexterous manipulation using a JACO robotic arm with an Intel RealSense D435 camera. This system is designed to automate the manipulation of industrial valves by capturing point clouds (PCs) from multiple perspectives to improve the accuracy of pose estimation. The object recognition module includes scene segmentation, geometric primitives recognition, model recognition, and a color-based clustering and integration approach enhanced by a dynamic cluster merging algorithm. Pose estimation is achieved using the random sample consensus algorithm, which predicts position and orientation. The system was tested within a 60° field of view, which extended in all directions in front of the object. The experimental results show that the system performs reliably within acceptable error thresholds for both position and orientation when the objects are within a ±15° range of the camera's direct view. However, errors increased with more extreme object orientations and distances, particularly when estimating the orientation of ball valves. A zone-based dexterous manipulation strategy was developed to overcome these challenges, where the system adjusts the camera position for optimal conditions. This approach mitigates larger errors in difficult scenarios, enhancing overall system reliability. The key contributions of this research include a novel method for improving object recognition and pose estimation, a technique for increasing the accuracy of pose estimation, and the development of a robot motion model for dexterous manipulation in industrial settings.

Robust tube-based MPC with smooth computation for dexterous robot manipulation

Robust tube-based MPC with smooth computation for dexterous robot manipulation