Bilateral tele-control using mullti-fingered humanoid root an with communication delay

This paper presents a tele-control system constructed from a multi-fingered robot hand and operator. The angle of the robot hand is controlled by the angle of the operator’s finger, and the operator feels the environmental force, as detected by the robot hand, constituting so-called bilateral master/slave control.

This paper presents a massage motion control system comprised of position control and force control in a multi-fingered robot hand. By making use of an algorithm which converted the desired fingertip trajectory into the desired angle of links in each finger by means of inverse kinematics, the finger position control from the initial position of the multi-fingered robot hand to a target position of the objects for massage was achieved. Its controller was used until the robot hand contacted the objects for massage. After contact was made, the fingertip position control was switched to a force control position needed to apply pressure for the massage. The fingertip forces exerted by an expert human therapist was measured using sheet distribution pressure sensors, and the data obtained was recorded in a computer. After the measurements were taken, the human expert's fingertip force was reproduced by the robot hand. The fingertip force of the robot hand was controlled using feedback obtained with a 6-axis force sensor. To make the force of each fingertip of the four-fingered robot hand track to the fingertip force exerted by the expert human massage therapist, PI servo compensation and a Jacobian matrix were really applied for the human's shoulder. Through simulation and experiments, the usefulness of the proposed control systems was demonstrated.

A multi-finger robotic hand with an iris mechanism that we previously developed was driven by a single actuator and could grasp an object by wrapping fingers completely around its circumference at multiple points. However, the blades used to grasp objects were within the robotic hand mechanism, so it could only grasp objects small enough to fit within the hollow disk comprising the outer surface of the device body. Furthermore, the hand could not grasp objects smaller than the thickness of the hollow disk. The multi-fingered robotic hand proposed in this study has a new mechanism in which the blades of the iris are placed outside of the hand mechanism, and fingers shaped as equilateral triangular prisms extend perpendicular to the disk of the robotic hand body and are attached to the blade tip. Placing the blade outside the mechanism and adjusting the gear ratios within allows adjustments to the gripping torque and speed. The vertically extended fingers can thus grasp small objects and objects longer than the blade diameter. In this study, we performed geometric and theoretical analyses of the proposed multi-fingered robotic hand. We then fabricated an actual robotic hand, verified the validity of the analyses.

Multi-finger Robotic hands (MFRH) are desired similar to human hands in order to perform stable grasping and fine manipulation of different objects. Their industrial applications including material handling fulfills the requirement of unique end-effector tool empowering specific reach, payloads, and flexibility. The design and control of dexterous and prosthetic robotic hands is of important concern these days. The performance of these hands depends on their mechanical design, prosthetics etc. The mechanical range of movement must be properly controlled and monitored to get the best performance of the robotic hand. In order to obtain the desired outcome from these robotic hands, various design parameters are discussed. The control issues of the multi-finger hand-arm system in order to interact with the human environment are also discussed. The objective of this paper is to evaluate multi-finger robotic hands capable of grasping a large variety of products. An overview of the relations between the designing features for the robotic hand, its anthropomorphism and dexterity is reported. Also, the best known robotic hands developed so far are reviewed emphasizing on their ergonomics and mechanical features. Based on these parameters, a newly designed four fingered tendon actuated robotic hand is discussed along with its mechanical structure.

Thermal Modeling of Robotic Arm and Hand Moving in a Nonhomogeneous Temperature Field

Multifingered hand dexterous manipulation is quite challenging in the domain of robotics. One remaining issue is how to achieve compliant behaviors. In this work, we propose a human-in-the-loop learning-control approach for acquiring compliant grasping and manipulation skills of a multifinger robot hand. This approach takes the depth image of the human hand as input and generates the desired force commands for the robot. The markerless vision-based teleoperation system is used for the task demonstration, and an end-to-end neural network model (i.e., TeachNet) is trained to map the pose of the human hand to the joint angles of the robot hand in real-time. To endow the robot hand with compliant human-like behaviors, an adaptive force control strategy is designed to predict the desired force control commands based on the pose difference between the robot hand and the human hand during the demonstration. The force controller is derived from a computational model of the biomimetic control strategy in human motor learning, which allows adapting the control variables (impedance and feedforward force) online during the execution of the reference joint angles. The simultaneous adaptation of the impedance and feedforward profiles enables the robot to interact with the environment compliantly. Our approach has been verified in both simulation and real-world task scenarios based on a multifingered robot hand, that is, the Shadow Hand, and has shown more reliable performances than the current widely used position control mode for obtaining compliant grasping and manipulation behaviors.

One of the greatest challenges of humanoid robotics is to provide a robotic systems with autonomous and dextrous skills. Dextrous manipulation skills, for personal and service robots in unstructured environments, are of fundamental importance, in order to accomplish manipulation tasks in human-like ways and to realize a proper and safe cooperation between humans and robots. The contributions presented in this thesis are aimed at modeling and controlling multifingered robotic hands with soft covers for manipulation tasks. The control issue of a hand-arm robotic system involved in grasping tasks, which can interact with the environment or a human, is also addressed. A port-Hamiltonian model of a multifingered robotic hand, with soft-pads on the finger tips, grasping an object has been developed. The port-Hamiltonian framework is based on the description of systems in terms of energy variables, and their interconnection in terms of power ports. Any physical systems can be described by a set of elements storing kinetic or potential energy, a set of energy dissipating elements, and a set of power ports interconnected by power preserving interconnections. The viscoelastic behavior of the contact is described in terms of energy storage and dissipation. Using the concept of power ports, the dynamics of the hand, the contact, and the object are described. The algebraic constraints of the in- terconnected systems are represented by a geometric object, called Dirac structure. This provides a powerful way to describe the non-contact to contact transition and contact viscoelasticity, by using the concept of energy flows and power preserving interconnections. Using the port based model, an Intrinsically Passive Controller (IPC) is used to control the internal forces and the motion of the object. In grasping tasks, in the case that also interaction with the environment or a human is involved, the control issue of a hand-arm robotic system, is addressed. Thecontrol law adopted for the arm is a compliance object-level control, which aims to reduce the interaction forces. The control action is based on the reconstruction of the external load applied to the object, using the force sensors measurement at the fingertips. Force sensing is also used to compute in real time the desired contact forces, able to guarantee the stability of the grasp. The regulation of the grasping forces is in charge of the hand control. In detail, the contents of the thesis are organized as follows. Chapter 1 provides an introduction on grasping and manipulation applications in the context of advanced robotics where the robot has to operate in unstructured environment. Here the relevance of dexterous manipulation skills in performing many di®erent tasks is emphasized. The framework of the research work in this section is introduced, i.e., the activities in the European project DEXMART. A brief description of the research objectives and the key innovations carried out within the DEXMART project are given. Chapter 2 contains an overview on the relations between the designing features of a robotic hand and its anthropomorphism and dexterity. Then the robotic hand built within the DEXMART project is introduced. A detailed description of the mechanical structure and of the actuation system by means of tendons is provided. Moreover, the kinematics, the statics and the dynamics of the hand are derived. The control structure and the control of the interaction in presence of soft contact is analyzed. Chapter 3 presents a port-Hamiltonian model of a multifingered robotic hand, with soft-pads, while grasping and manipulating an object. An introduction to the port-based formulation is provided. For the validation of the model, a simple example modeled in 20-sim simulation software is considered. Simulation results are presented to validate the model and to show the behavior of the system when an IPC based controller is applied. In Chapter 4 the control issue of a hand-arm robotic system involved in grasping tasks, which can interact with the environment or a human, is addressed. An introduction on the combined control of hand-arm systems is given. The proposed control action is based on the reconstruction of the forces appliedto the object, using the measurement at the fingertips, in order to obtain a compliant behavior of the arm and to reduce the interaction forces. A detailed simulation model of the robotic hand has been developed with the aim of testing the control strategies, using the SimMechanics toolbox of MATLAB. Simulation tests in MATLAB/SimMechanics environment demonstrate the effectiveness of the proposed approach. Chapter 5 contains concluding remarks and proposals for further investigations.

The multi-finger robot hand (IRA-Hand I) is specially designed for massage applications. Surface electromyographic (SEMG) evaluation of therapeutic massage effects using multi-finger robot hand are presented in this paper. After an isometric 50% MVC (maximum voluntary contraction) of human back muscles is performed for 90 seconds, SEMG signals of the subject show a muscular fatigue. A grasp-kneading massage by the robot hand is applied on the subject's shoulder for recovering from muscular fatigue. To evaluate the therapeutic effects of massage, the SEMG signals measured from the trapezius muscles before and after the massage therapy are analyzed. Electrical activity (EA) and median frequency (MF) of the SEMG signals are calculated as indexes of the muscle physiological states. The experimental results show that EA increases from the occurrence of fatigue, while MF shifts towards lower frequency in the spectral distribution. After massage, a decrease in EA and an increase in MF are observed which demonstrate the effectiveness of recovery through the grasp-kneading massage by the robot hand. In addition, the joint analysis of EMG spectrum and amplitude (JASA), which considers the changes in time domain and in frequency domain simultaneously, verifies that the therapeutic massage recovers the trapezius muscle from fatigue effectively. For comparison, the experiments with a massage specialist performing the massage therapy are conducted with the same procedures. It is evidenced that the robot hand massage has even better effectiveness than the human hand in most cases.

Based on inspiration of human grasping activities, a new idea is developed in this paper that grasping forces in a multifingered robotic hand can be regulated and controlled through its compliance by actively coordinating small joint motions in its fingers. According to this idea, a grasping force control model is formulated by means of a compliance model developed by the authors before, and a novel theory is then developed for grasping force control in a multifingered robot hand. The developed theory is expected to lead to a new force control method which could serve as a promising alternative for the active stiffness method. As an application of the developed theory, a two-fingered planar robotic hand is also analyzed, and the simulation results verify the developed theory.

Recently, aerial manipulations are becoming more and more important for the practical applications of unmanned aerial vehicles (UAV) to choose, transport, and place objects in global space. In this paper, an aerial manipulation system consisting of a UAV, two onboard cameras, and a multi-fingered robotic hand with proximity sensors is developed. To achieve self-contained autonomous navigation to a targeted object, onboard tracking and depth cameras are used to detect the targeted object and to control the UAV to reach the target object, even in a Global Positioning System-denied environment. The robotic hand can perform proximity sensor-based grasping stably for an object that is within a position error tolerance (a circle with a radius of 50 mm) from the center of the hand. Therefore, to successfully grasp the object, a requirement for the position error of the hand (=UAV) during hovering after reaching the targeted object should be less than the tolerance. To meet this requirement, an object detection algorithm to support accurate target localization by combining information from both cameras was developed. In addition, camera mount orientation and UAV attitude sampling rate were determined by experiments, and it is confirmed that these implementations improved the UAV position error to within the grasping tolerance of the robot hand. Finally, the experiments on aerial manipulations using the developed system demonstrated the successful grasping of the targeted object.

This paper presents an electromyographic (EMG) signal integrated multi-finger robot hand control for massage applications. This research explores the feasible application of multi-finger robot hands except for the use as prostheses and grasping applications. The forearm EMG of a person who is massaged by the human hands is recorded and analyzed statistically. First, the root mean square (RMS) of the raw data is computed as the discrimination between normal and contracted states of the muscle. Then the EMG signal at contracted state is further divided into painful and comfortable groups based on the impulse factor which is defined to estimate the sharpness of waveform variations. As a consequence, two discriminative values of the EMG signal are generated to distinguish painful and comfortable feelings. Based on the relationship between the human feeling and the massage force, we get an appropriate range of input commands of the robot hand for massage applications. A grasp-kneading massage is performed on the human shoulder to verify the proposed process. As a result, an effective and comfortable massage using the multi-finger robot hand is realized.

Multifingered robot hands have the ability to manipulate and grasp various objects, like a human hand. In previous studies, we have looked at paper folding operations (Origami) as an example of dexterous manipulation to be performed by dual robot hands. It is difficult for robot hands to manipulate a sheet of paper because of its deformation, and it is thus necessary to recognize its shape during Origami operations. In this paper, we propose a method of estimating a 3D model of paper based on depth information obtained from a 3D sensor. The 3D model of the paper is expressed by using a physics simulator composed of nodes with springs and dampers. As a result, highly accurate 3D shape estimation was achieved during Origami operations.

Multifingered robot hands: Control for grasping and manipulation

This paper presents a tele-control system comprised of a robot hand/arm and an operator. In our system, the angle of the finger of the robot hand is controlled according to the angle of the operator's finger, and the position of the robot arm is controlled according to the position of the operator's arm. Simultaneously, the operator feels the environmental force as detected by the touch sensor attached to the robot hand, resulting in so-called bilateral master/slave control. To date, there have been a few studies of bilateral master/slave systems that use a multi-fingered humanoid robot hand in communication networks with delay caused by physical distance. The purpose of our study was to achieve tele-operation with bilateral master/slave control between the operator's hand/arm and the multi-fingered humanoid robot hand/arm using the Internet with delay caused by distance. For our experiment, the operator was in the USA, and the multi-fingered humanoid robot hand/arm and object were in Japan. Using our system, the operator could successfully grasp and move an object while he felt the reaction force from the object. This technology can applicable to tele-operation such as feeling the hardness and weight of the stone which exist in the moon, on the ground etc.

We have presented the newly developed anthropomorphic robot hand named the KH Hand type S and its master slave system using the bilateral controller. The use of an elastic body has improved the robot hand in terms of weight, the backlash of the transmission, and friction between the gears. We have demonstrated the expression of the Japanese finger alphabet. We have also shown an experiment of a peg-in-hole task controlled by the bilateral controller. These results indicate that the KH Hand type S has a higher potential than previous robot hands in performing not only hand shape display tasks but also in grasping and manipulating objects in a manner like that of the human hand. In our future work, we are planning to study dexterous grasping and manipulation by the robot.