Hand-arm Robot Research Articles

Cybernic robot hand-arm that realizes cooperative work as a new hand-arm for people with a single upper-limb dysfunction.

A robot hand-arm that can perform various tasks with the unaffected arm could ease the daily lives of patients with a single upper-limb dysfunction. A smooth interaction between robot and patient is desirable since their other arm functions normally. If the robot can move in response to the user's intentions and cooperate with the unaffected arm, even without detailed operation, it can effectively assist with daily tasks. This study aims to propose and develop a cybernic robot hand-arm with the following features: 1) input of user intention via bioelectrical signals from the paralyzed arm, the unaffected arm's motion, and voice; 2) autonomous control of support movements; 3) a control system that integrates voluntary and autonomous control by combining 1) and 2) to thus allow smooth work support in cooperation with the unaffected arm, reflecting intention as a part of the body; and 4) a learning function to provide work support across various tasks in daily use. We confirmed the feasibility and usefulness of the proposed system through a pilot study involving three patients. The system learned to support new tasks by working with the user through an operating function that does not require the involvement of the unaffected arm. The system divides the support actions into movement phases and learns the phase-shift conditions from the sensor information about the user's intention. After learning, the system autonomously performs learned support actions through voluntary phase shifts based on input about the user's intention via bioelectrical signals, the unaffected arm's motion, and by voice, enabling smooth collaborative movement with the unaffected arm. Experiments with patients demonstrated that the system could learn and provide smooth work support in cooperation with the unaffected arm to successfully complete tasks they find difficult. Additionally, the questionnaire subjectively confirmed that cooperative work according to the user's intention was achieved and that work time was within a feasible range for daily life. Furthermore, it was observed that participants who used bioelectrical signals from their paralyzed arm perceived the system as part of their body. We thus confirmed the feasibility and usefulness of various cooperative task supports using the proposed method.

Mixed-Reality-Enhanced Human–Robot Interaction with an Imitation-Based Mapping Approach for Intuitive Teleoperation of a Robotic Arm-Hand System

This paper presents an integrated mapping of motion and visualization scheme based on a Mixed Reality (MR) subspace approach for the intuitive and immersive telemanipulation of robotic arm-hand systems. The effectiveness of different control-feedback methods for the teleoperation system is validated and compared. The robotic arm-hand system consists of a 6 Degrees-of-Freedom (DOF) industrial manipulator and a low-cost 2-finger gripper, which can be manipulated in a natural manner by novice users physically distant from the working site. By incorporating MR technology, the user is fully immersed in a virtual operating space augmented by real-time 3D visual feedback from the robot working site. Imitation-based velocity-centric motion mapping is implemented via the MR subspace to accurately track operator hand movements for robot motion control and enables spatial velocity-based control of the robot Tool Center Point (TCP). The user control space and robot working space are overlaid through the MR subspace, and the local user and a digital twin of the remote robot share the same environment in the MR subspace. The MR-based motion and visualization mapping scheme for telerobotics is compared to conventional 2D Baseline and MR tele-control paradigms over two tabletop object manipulation experiments. A user survey of 24 participants was conducted to demonstrate the effectiveness and performance enhancements enabled by the proposed system. The MR-subspace-integrated 3D mapping of motion and visualization scheme reduced the aggregate task completion time by 48% compared to the 2D Baseline module and 29%, compared to the MR SpaceMouse module. The perceived workload decreased by 32% and 22%, compared to the 2D Baseline and MR SpaceMouse approaches.

Exodex Adam-A Reconfigurable Dexterous Haptic User Interface for the Whole Hand.

Applications for dexterous robot teleoperation and immersive virtual reality are growing. Haptic user input devices need to allow the user to intuitively command and seamlessly “feel” the environment they work in, whether virtual or a remote site through an avatar. We introduce the DLR Exodex Adam, a reconfigurable, dexterous, whole-hand haptic input device. The device comprises multiple modular, three degrees of freedom (3-DOF) robotic fingers, whose placement on the device can be adjusted to optimize manipulability for different user hand sizes. Additionally, the device is mounted on a 7-DOF robot arm to increase the user’s workspace. Exodex Adam uses a front-facing interface, with robotic fingers coupled to two of the user’s fingertips, the thumb, and two points on the palm. Including the palm, as opposed to only the fingertips as is common in existing devices, enables accurate tracking of the whole hand without additional sensors such as a data glove or motion capture. By providing “whole-hand” interaction with omnidirectional force-feedback at the attachment points, we enable the user to experience the environment with the complete hand instead of only the fingertips, thus realizing deeper immersion. Interaction using Exodex Adam can range from palpation of objects and surfaces to manipulation using both power and precision grasps, all while receiving haptic feedback. This article details the concept and design of the Exodex Adam, as well as use cases where it is deployed with different command modalities. These include mixed-media interaction in a virtual environment, gesture-based telemanipulation, and robotic hand–arm teleoperation using adaptive model-mediated teleoperation. Finally, we share the insights gained during our development process and use case deployments.

Learning to Prevent Grasp Failure with Soft Hands: From Online Prediction to Dual‐Arm Grasp Recovery

Soft hands allow to simplify the grasp planning to achieve a successful grasp, thanks to their intrinsic adaptability. At the same time, their usage poses new challenges, related to the adoption of classical sensing techniques originally developed for rigid end defectors, which provide fundamental information, such as to detect object slippage. Under this regard, model‐based approaches for the processing of the gathered information are hard to use, due to the difficulties in modeling hand–object interaction when softness is involved. To overcome these limitations, in this article, we proposed to combine distributed tactile sensing and machine learning (recurrent neural network) to detect sliding conditions for a soft robotic hand mounted on a robotic manipulator, targeting the prediction of the grasp failure event and the direction of sliding. The outcomes of these predictions allow for an online triggering of a compensatory action performed with a second robotic arm–hand system, to prevent the failure. Despite the fact that the network is trained only with spherical and cylindrical objects, we demonstrate high generalization capabilities of our framework, achieving a correct prediction of the failure direction in of cases, and a of successful regrasps, for a selection of 12 objects of common use.

Precision Grasp Using an Arm-Hand System as a Hybrid Parallel-Serial System: A Novel Inverse Kinematics Solution

In this letter, we present a novel inverse kinematics (IK) solution for a robotic arm-hand system to achieve precision grasp. This problem is kinematically over-constrained and to address the issue and to solve the problem, we propose a new approach with three key insights. First, we propose a human-inspired thumb-first strategy and consider one finger of the robotic hand as the “thumb” to narrow down the search space and increase the success rate of our algorithm. Second, we formulate the arm-thumb serial chain as a closed chain such that the entire arm-hand system is controlled as a hybrid parallel-serial system. The closed-chain formulation simplifies the task hierarchy of the entire arm-hand system. Third, we attach a virtual revolute joint to the thumb's tip with its rotation axis aligning with the thumb's contact normal The virtual joint will embody the thumb's functional redundancy. By selecting the thumb's joints including the added virtual revolute joint as active joints of the arm-thumb closed chain, the arm-thumb system's self-motion (i.e., the palm pose) and the thumb's functional redundancy can be directly controlled without using the null space projection. This provides a new possibility to control the self-motion of arm-hand systems. Simulation results will demonstrate the advantages and superior performance of the proposed approach for achieving precision grasp compared to other classical approaches.

Inverse Kinematics of High Dimensional Robotic Arm-Hand Systems for Precision Grasping

Conventionally, the arm and the hand are considered separately for robotic manipulation and grasp. The hand configuration and the wrist pose are obtained during grasp planning and afterwards, the arm configuration is found to accommodate the wrist pose. However, without considering the arm during grasp planning, this approach can become very inefficient as significant time and computation power are spent on evaluating and planning grasps that are simply unreachable. This paper presents an efficient method for obtaining a desired configuration of the arm and the hand simultaneously for precision grasping. Our method assumes that contact points and contact normals are given for the desired grasp. Inspired by the human grasp strategy, our approach implements a thumb-first strategy to narrow down the search space and increase the success rate. To precisely fulfill all fingers’ requirements, the inverse kinematics (IK) solution for the position and orientation of each finger are regarded as independent tasks. These tasks are organized in a well-designed hierarchy and the resulting joint movements from each level are combined through nullspace projection with a nullspace enlargement method to ensure the correctness of the results. Comprehensive simulation results will demonstrate that the proposed method can significantly improve the performance of classic IK algorithms.

Robotic hand-arm system for on-orbit servicing missions in Tiangong-2 Space Laboratory

PurposeThe purpose of this study is to develop a robotic hand–arm system for on-orbit servicing missions at the Tiangong-2 (TG-2) Space Laboratory.Design/methodology/approachThe hand–arm system is mainly composed of a lightweight arm, a dexterous hand, an electrical cabinet, a global camera, a hand–eye camera and some human–machine interfaces. The 6-DOF lightweight arm and the 15-DOF dexterous hand adopt the modular design philosophy that greatly reduces the design cycle and cost. To reduce the computational burden on the central controller and simplify system maintenance, an electrical system which has a hierarchical structure is introduced.FindingsThe prototypical operating experiments completed in TG-2 space laboratory demonstrate the performance of the hand–arm system and lay foundations for the future applications of space manipulators.Originality/valueThe main contributions of this paper are as follows a robotic hand–arm system which can perform on-orbit servicing missions such as grasping the electric drill, screwing the bolt, unscrewing J599 electrical connector has been developed; a variable time step motion plan method is proposed to adjust the trajectories of the lightweight arm to reduce or eliminate the collision force; and a dexterous hand uses the coordinated grasp control based on the object Cartesian stiffness to realize stable grasp. To solve the kinematic mapping from the cyber glove commands to the dexterous hand, a fingertip-position-based method is proposed to acquire precise solutions.

Simultaneous gravity and gripping force compensation mechanism for lightweight hand-arm robot with low-reduction reducer

SummaryIn this paper, we propose a novel mechanism to compensate for gravity and the gripping force in a hand-arm robot. This mechanism compensates for the gravitational torque produced by an object gripped by the hand-arm robot. The gripping force required for the robot hand to prevent the object from dropping is also simultaneously compensated for. This mechanism requires only one actuator placed on the shoulder part of the robot. Therefore, this mechanism can reduce the torque requirement of joint actuators and lower the weight of the robot. The gear ratio of the reduction gears in each robot joint can then also be reduced. These advantages are critical for future robots that perform tasks in unstructured environments and collaborate with humans. We carried out experiments with a 6-DoF robot arm having a 1-DoF gripper to demonstrate the effectiveness of the proposed mechanism.

Robotic teleoperation systems using a wearable multimodal fusion device

Teleoperation is of great importance in the area of robotics especially when people’s presence at the robot working space is unavailable. It provides an alternative to employ human intelligence in the control of the robot remotely. We establish robotic teleoperation systems with a wearable multimodal fusion device. The device is integrated with 18 low-cost inertial and magnetic measurement units, which cover all segments of the arm and hand. The multimodal fusion algorithm based on extended Kalman filter is deduced to determine the orientations and positions of each segment. Then, the robotic teleoperation systems using the proposed device are designed. The novel teleoperation schemes can be applied for 11DOF robotic arm–hand system and 10DOF robotic arm–hand system, in which the operator’s fingers are used for robotic hand teleoperation, and the arms with palm are used for robotic arm teleoperation. Meanwhile, the proposed robotic teleoperation systems are fully realized with a user-friendly human–machine interaction interface. Finally, a series of experiments are conducted with our robotic teleoperation system successfully.

A Primal-Dual Interior-Point Method for Optimal Grasping Manipulation of Multi-fingered Hand-Arm Robots

In this paper, we consider an optimization problem of the grasping manipulation of multi-fingered hand-arm robots. We first formulate an optimization model for the problem, based on the dynamic equations of the object and the friction constraints. Then, we reformulate the model as a convex quadratic programming over circular cones. Moreover, we propose a primal-dual interior-point algorithm based on the kernel function to solve this convex quadratic programming over circular cones. We derive both the convergence of the algorithm and the iteration bounds for large- and small-update methods, respectively. Finally, we carry out the numerical tests of $$180^{\circ }$$ and $$90^{\circ }$$ manipulations of the hand-arm robot to demonstrate the effectiveness of the proposed algorithm.

A novel data glove using inertial and magnetic sensors for motion capture and robotic arm-hand teleoperation

PurposeThe purpose of this paper is to present a novel data glove which can capture the motion of the arm and hand by inertial and magnetic sensors. The proposed data glove is used to provide the information of the gestures and teleoperate the robotic arm-hand.Design/methodology/approachThe data glove comprises 18 low-cost inertial and magnetic measurement units (IMMUs) which not only make up the drawbacks of traditional data glove that only captures the incomplete gesture information but also provide a novel scheme of the robotic arm-hand teleoperation. The IMMUs are compact and small enough to wear on the upper arm, forearm, palm and fingers. The calibration method is proposed to improve the accuracy of measurements of units, and the orientations of each IMMU are estimated by a two-step optimal filter. The kinematic models of the arm, hand and fingers are integrated into the entire system to capture the motion gesture. A positon algorithm is also deduced to compute the positions of fingertips. With the proposed data glove, the robotic arm-hand can be teleoperated by the human arm, palm and fingers, thus establishing a novel robotic arm-hand teleoperation scheme.FindingsExperimental results show that the proposed data glove can accurately and fully capture the fine gesture. Using the proposed data glove as the multiple input device has also proved to be a suitable teleoperating robotic arm-hand system.Originality/valueIntegrated with 18 low-cost and miniature IMMUs, the proposed data glove can give more information of the gesture than existing devices. Meanwhile, the proposed algorithms for motion capture determine the superior results. Furthermore, the accurately captured gestures can efficiently facilitate a novel teleoperation scheme to teleoperate the robotic arm-hand.

Deriving Humanlike Arm Hand System Poses

Robots are rapidly becoming part of our lives, coexisting, interacting, and collaborating with humans in dynamic and unstructured environments. Mapping of human to robot motion has become increasingly important, as human demonstrations are employed in order to “teach” robots how to execute tasks both efficiently and anthropomorphically. Previous mapping approaches utilized complex analytical or numerical methods for the computation of the robot inverse kinematics (IK), without considering the humanlikeness of robot motion. The scope of this work is to synthesize humanlike robot trajectories for robot arm-hand systems with arbitrary kinematics, formulating a constrained optimization scheme with minimal design complexity and specifications (only the robot forward kinematics (FK) are used). In so doing, we capture the actual human arm-hand kinematics, and we employ specific metrics of anthropomorphism, deriving humanlike poses and trajectories for various arm-hand systems (e.g., even for redundant or hyper-redundant robot arms and multifingered robot hands). The proposed mapping scheme exhibits the following characteristics: (1) it achieves an efficient execution of specific human-imposed goals in task-space, and (2) it optimizes anthropomorphism of robot poses, minimizing the structural dissimilarity/distance between the human and the robot arm-hand systems.

RGB-D画像からの把持パタン想起に基づくハンドアームによる物体把持

RGB-D画像からの把持パタン想起に基づくハンドアームによる物体把持

ハンドアームロボットによる動力学を考慮したボールジャグリング動作

In this paper, we study ball juggling as a dexterous and accurate high-speed manipulation like a human. We have already achieved two-ball juggling with our high-speed robotic handarm system which has three general-purpose fingers and high-speed vision system. However, the number of times of ball-catching is not so many because the trajectory of the motion of the robot is not optimized. In this paper, we propose a trajectory generation method using smoothing splines. The motion of throwing a ball is optimized so as to minimize the torques of the joint angles. The motion of catching is controlled by visual feedback. As a result, stable throwing motion is achieved so that the success rate of ball-catching is about 90%.

An assistive decision-and-control architecture for force-sensitive hand–arm systems driven by human–machine interfaces

Fully autonomous applications of modern robotic systems are still constrained by limitations in sensory data processing, scene interpretation, and automated reasoning. However, their use as assistive devices for people with upper-limb disabilities has become possible with recent advances in “soft robotics”, that is, interaction control, physical human–robot interaction, and reflex planning. In this context, impedance and reflex-based control has generally been understood to be a promising approach to safe interaction robotics. To create semi-autonomous assistive devices, we propose a decision-and-control architecture for hand–arm systems with “soft robotics” capabilities, which can then be used via human–machine interfaces (HMIs). We validated the functionality of our approach within the BrainGate2 clinical trial, in which an individual with tetraplegia used our architecture to control a robotic hand–arm system under neural control via a multi-electrode array implanted in the motor cortex. The neuroscience results of this research have previously been published by Hochberg et al. In this paper we present our assistive decision-and-control architecture and demonstrate how the semi-autonomous assistive behavior can help the user. In our framework the robot is controlled through a multi-priority Cartesian impedance controller and its behavior is extended with collision detection and reflex reaction. Furthermore, virtual workspaces are added to ensure safety. On top of this we employ a decision-and-control architecture that uses sensory information available from the robotic system to evaluate the current state of task execution. Based on a set of available assistive skills, our architecture provides support in object interaction and manipulation and thereby enhances the usability of the robotic system for use with HMIs. The goal of our development is to provide an easy-to-use robotic system for people with physical disabilities and thereby enable them to perform simple tasks of daily living. In an exemplary real-world task, the participant was able to serve herself a beverage autonomously for the first time since her brainstem stroke, which she suffered approximately 14 years prior to this research.

A Survey of Robotic Hand- Arm Systems

From the last three decades creating human robotic hand replica with enhanced capabilities is of concern and lot of efforts have been put into it. This paper focuses on understanding the different techniques that are used for human robot interaction in robotic hand arm systems. Diversification is stated in areas of human and robotic hand interaction, the degrees of freedom, the grasping ability, number of fingers and materials used for the hand. The flexibility of grasp is compared in terms of Degrees Of Freedom (DOF) and the number of finger end effectors. The controlling method is either through sensor based or gesture controlled or simulation based or pre-defined positions.

1P2-I06 ハンドアームロボットによる内外旋軌道に基づくボールジャグリング

Juggling is one of tasks involving skillful dynamic manipulation, and visual information is essential to perform juggling. We achieved a two-ball juggling task with our high-speed robotic hand-arm system which had three general-purpose fingers and high-speed vision system. But it could not afford enough long operation time for juggling because the trajectory was not optimized. Therefore this algorithm often caused miss of catching the ball. In this study, we work on the improvement of the robot's trajectory at the juggling. We get the 3-dimensional hand position in human ball juggling by a 3D sensor. As a result the characteristics of human juggling is obtained. Based on the motion analysis, we make the throwing motion whose orbit is a circle including internal and external motion. We are successful in the throwing and catching motion by a new program.

A space robot manipulator system: designed for capture

The paper describes a space robot hand arm system designed for non-cooperative object capture. According to analysis on the characteristics of non-cooperative target capture on-orbit, a three fingers end-effector was designed. The detailed design of a modular space manipulator joint was described. To meet the nice performance of hand arm system in space applications, a communication and computing embedded systems was designed. We established an experimental platform in library for single joint test, with that proved the joint had enough precision, speed, and torque performance. At the same time, we constructed an experimental system for manipulator on air-bearing table, with that proved the manipulator had nice location precision. Experimental results showed that the hand arm system was able to meet the needs of the on-orbit application.

Method of Grasping Control by Computing Internal and External Impedances for Two Robot Fingers, and Its Application to Admittance Control of a Robot Hand-Arm System

Impedance control is an important technology used in the grasping control of a robot hand. Numerous studies related to grasping algorithms have been reported in recent years, with the contact force between robot fingers and the object to be grasped being primarily discussed in most cases. Generally, a coupling effect occurs between the internal loop of the grasping operation and the external loop of the interaction with the environment when a multi-fingered robot hand is used to complete a contact task. Therefore, a robot hand cannot hold an object using a large external force to complete a wide range of tasks by applying the conventional method. In this paper, the coupling of the internal/external forces occurring in grasping operations using multiple fingers is analysed. Then, improved impedance control based on the previous method is proposed as an effective tool to solve the problem of grasping failure caused by single-finger contact. Furthermore, a method for applying the improved grasping algorithm to the admittance control of a robot hand-arm system is also proposed. The proposed method divides the impedance effect into the grasping control of the hand and the cooperative control of the arm, so that expanding the task space and increasing the flexibility of impedance adjustment can be achieved. Experiments were conducted to demonstrate the effectiveness of the proposed method.

Parental scaffolding as a bootstrapping mechanism for learning grasp affordances and imitation skills

SUMMARYParental scaffolding is an important mechanism that speeds up infant sensorimotor development. Infants pay stronger attention to the features of the objects highlighted by parents, and their manipulation skills develop earlier than they would in isolation due to caregivers' support. Parents are known to make modifications in infant-directed actions, which are often called “motionese”7. The features that might be associated with motionese are amplification, repetition and simplification in caregivers' movements, which are often accompanied by increased social signalling. In this paper, we extend our previously developed affordances learning framework to enable our hand-arm robot equipped with a range camera to benefit from parental scaffolding and motionese. We first present our results on how parental scaffolding can be used to guide the robot learning and to modify its crude action execution to speed up the learning of complex skills. For this purpose, an interactive human caregiver-infant scenario was realized with our robotic setup. This setup allowed the caregiver's modification of the ongoing reach and grasp movement of the robot via physical interaction. This enabled the caregiver to make the robot grasp the target object, which in turn could be used by the robot to learn the grasping skill. In addition to this, we also show how parental scaffolding can be used in speeding up imitation learning. We present the details of our work that takes the robot beyond simple goal-level imitation, making it a better imitator with the help of motionese.