Origami operations by multifingered robot hand with realtime 3D shape estimation of paper

This paper presents a tele-control system constructed from a multi-fingered robot hand and operator. The angle of the robot hand is controlled by the angle of the operator’s finger, and the operator feels the environmental force, as detected by the robot hand, constituting so-called bilateral master/slave control.

We have presented the newly developed anthropomorphic robot hand named the KH Hand type S and its master slave system using the bilateral controller. The use of an elastic body has improved the robot hand in terms of weight, the backlash of the transmission, and friction between the gears. We have demonstrated the expression of the Japanese finger alphabet. We have also shown an experiment of a peg-in-hole task controlled by the bilateral controller. These results indicate that the KH Hand type S has a higher potential than previous robot hands in performing not only hand shape display tasks but also in grasping and manipulating objects in a manner like that of the human hand. In our future work, we are planning to study dexterous grasping and manipulation by the robot.

This paper presents a tele-control system constructed from a multi-fingered robot hand and operator. The angle of the robot hand is controlled by the angle of the operator's finger, and the operator feels the environmental force, as detected by the robot hand, constituting so-called bilateral master/slave control. In the experiments, the operator grasped the object in spite of Round Trip Time (RTT) as Osec, 0.56sec, using a multi-fingered humanoid robot hand by master/slave control feeling fingertip force. However, with increases in the RTT, the operation became more difficult. We also analyzed the stability of the master site and the slave site by frequency characteristics. The results showed that this system was unstable. However, grasping by tele-control with a communication delay was demonstrated.

One of the greatest challenges of humanoid robotics is to provide a robotic systems with autonomous and dextrous skills. Dextrous manipulation skills, for personal and service robots in unstructured environments, are of fundamental importance, in order to accomplish manipulation tasks in human-like ways and to realize a proper and safe cooperation between humans and robots. The contributions presented in this thesis are aimed at modeling and controlling multifingered robotic hands with soft covers for manipulation tasks. The control issue of a hand-arm robotic system involved in grasping tasks, which can interact with the environment or a human, is also addressed. A port-Hamiltonian model of a multifingered robotic hand, with soft-pads on the finger tips, grasping an object has been developed. The port-Hamiltonian framework is based on the description of systems in terms of energy variables, and their interconnection in terms of power ports. Any physical systems can be described by a set of elements storing kinetic or potential energy, a set of energy dissipating elements, and a set of power ports interconnected by power preserving interconnections. The viscoelastic behavior of the contact is described in terms of energy storage and dissipation. Using the concept of power ports, the dynamics of the hand, the contact, and the object are described. The algebraic constraints of the in- terconnected systems are represented by a geometric object, called Dirac structure. This provides a powerful way to describe the non-contact to contact transition and contact viscoelasticity, by using the concept of energy flows and power preserving interconnections. Using the port based model, an Intrinsically Passive Controller (IPC) is used to control the internal forces and the motion of the object. In grasping tasks, in the case that also interaction with the environment or a human is involved, the control issue of a hand-arm robotic system, is addressed. Thecontrol law adopted for the arm is a compliance object-level control, which aims to reduce the interaction forces. The control action is based on the reconstruction of the external load applied to the object, using the force sensors measurement at the fingertips. Force sensing is also used to compute in real time the desired contact forces, able to guarantee the stability of the grasp. The regulation of the grasping forces is in charge of the hand control. In detail, the contents of the thesis are organized as follows. Chapter 1 provides an introduction on grasping and manipulation applications in the context of advanced robotics where the robot has to operate in unstructured environment. Here the relevance of dexterous manipulation skills in performing many di®erent tasks is emphasized. The framework of the research work in this section is introduced, i.e., the activities in the European project DEXMART. A brief description of the research objectives and the key innovations carried out within the DEXMART project are given. Chapter 2 contains an overview on the relations between the designing features of a robotic hand and its anthropomorphism and dexterity. Then the robotic hand built within the DEXMART project is introduced. A detailed description of the mechanical structure and of the actuation system by means of tendons is provided. Moreover, the kinematics, the statics and the dynamics of the hand are derived. The control structure and the control of the interaction in presence of soft contact is analyzed. Chapter 3 presents a port-Hamiltonian model of a multifingered robotic hand, with soft-pads, while grasping and manipulating an object. An introduction to the port-based formulation is provided. For the validation of the model, a simple example modeled in 20-sim simulation software is considered. Simulation results are presented to validate the model and to show the behavior of the system when an IPC based controller is applied. In Chapter 4 the control issue of a hand-arm robotic system involved in grasping tasks, which can interact with the environment or a human, is addressed. An introduction on the combined control of hand-arm systems is given. The proposed control action is based on the reconstruction of the forces appliedto the object, using the measurement at the fingertips, in order to obtain a compliant behavior of the arm and to reduce the interaction forces. A detailed simulation model of the robotic hand has been developed with the aim of testing the control strategies, using the SimMechanics toolbox of MATLAB. Simulation tests in MATLAB/SimMechanics environment demonstrate the effectiveness of the proposed approach. Chapter 5 contains concluding remarks and proposals for further investigations.

Multifingered hand dexterous manipulation is quite challenging in the domain of robotics. One remaining issue is how to achieve compliant behaviors. In this work, we propose a human-in-the-loop learning-control approach for acquiring compliant grasping and manipulation skills of a multifinger robot hand. This approach takes the depth image of the human hand as input and generates the desired force commands for the robot. The markerless vision-based teleoperation system is used for the task demonstration, and an end-to-end neural network model (i.e., TeachNet) is trained to map the pose of the human hand to the joint angles of the robot hand in real-time. To endow the robot hand with compliant human-like behaviors, an adaptive force control strategy is designed to predict the desired force control commands based on the pose difference between the robot hand and the human hand during the demonstration. The force controller is derived from a computational model of the biomimetic control strategy in human motor learning, which allows adapting the control variables (impedance and feedforward force) online during the execution of the reference joint angles. The simultaneous adaptation of the impedance and feedforward profiles enables the robot to interact with the environment compliantly. Our approach has been verified in both simulation and real-world task scenarios based on a multifingered robot hand, that is, the Shadow Hand, and has shown more reliable performances than the current widely used position control mode for obtaining compliant grasping and manipulation behaviors.

Multi-finger Robotic hands (MFRH) are desired similar to human hands in order to perform stable grasping and fine manipulation of different objects. Their industrial applications including material handling fulfills the requirement of unique end-effector tool empowering specific reach, payloads, and flexibility. The design and control of dexterous and prosthetic robotic hands is of important concern these days. The performance of these hands depends on their mechanical design, prosthetics etc. The mechanical range of movement must be properly controlled and monitored to get the best performance of the robotic hand. In order to obtain the desired outcome from these robotic hands, various design parameters are discussed. The control issues of the multi-finger hand-arm system in order to interact with the human environment are also discussed. The objective of this paper is to evaluate multi-finger robotic hands capable of grasping a large variety of products. An overview of the relations between the designing features for the robotic hand, its anthropomorphism and dexterity is reported. Also, the best known robotic hands developed so far are reviewed emphasizing on their ergonomics and mechanical features. Based on these parameters, a newly designed four fingered tendon actuated robotic hand is discussed along with its mechanical structure.

The multi-finger robot hand (IRA-Hand I) is specially designed for massage applications. Surface electromyographic (SEMG) evaluation of therapeutic massage effects using multi-finger robot hand are presented in this paper. After an isometric 50% MVC (maximum voluntary contraction) of human back muscles is performed for 90 seconds, SEMG signals of the subject show a muscular fatigue. A grasp-kneading massage by the robot hand is applied on the subject's shoulder for recovering from muscular fatigue. To evaluate the therapeutic effects of massage, the SEMG signals measured from the trapezius muscles before and after the massage therapy are analyzed. Electrical activity (EA) and median frequency (MF) of the SEMG signals are calculated as indexes of the muscle physiological states. The experimental results show that EA increases from the occurrence of fatigue, while MF shifts towards lower frequency in the spectral distribution. After massage, a decrease in EA and an increase in MF are observed which demonstrate the effectiveness of recovery through the grasp-kneading massage by the robot hand. In addition, the joint analysis of EMG spectrum and amplitude (JASA), which considers the changes in time domain and in frequency domain simultaneously, verifies that the therapeutic massage recovers the trapezius muscle from fatigue effectively. For comparison, the experiments with a massage specialist performing the massage therapy are conducted with the same procedures. It is evidenced that the robot hand massage has even better effectiveness than the human hand in most cases.

A multi-finger robotic hand with an iris mechanism that we previously developed was driven by a single actuator and could grasp an object by wrapping fingers completely around its circumference at multiple points. However, the blades used to grasp objects were within the robotic hand mechanism, so it could only grasp objects small enough to fit within the hollow disk comprising the outer surface of the device body. Furthermore, the hand could not grasp objects smaller than the thickness of the hollow disk. The multi-fingered robotic hand proposed in this study has a new mechanism in which the blades of the iris are placed outside of the hand mechanism, and fingers shaped as equilateral triangular prisms extend perpendicular to the disk of the robotic hand body and are attached to the blade tip. Placing the blade outside the mechanism and adjusting the gear ratios within allows adjustments to the gripping torque and speed. The vertically extended fingers can thus grasp small objects and objects longer than the blade diameter. In this study, we performed geometric and theoretical analyses of the proposed multi-fingered robotic hand. We then fabricated an actual robotic hand, verified the validity of the analyses.

Thermal modeling of a robotic arm equipped with a multifingered robotic hand (end effector) is considered in this paper. The robotic arm is assumed to move its multifingered hand into and out of a high-temperature medium while gripping an object for a pick and place assembly and heat treatment process. If the rate of heat transfer from robotic hand to robotic arm is small, a lumped-capacitance model can be used to find the transient temperature response of the robotic hand. Different models to approximate the effect of thermal coupling between the robotic hand and robotic arm are discussed. To analyze the effect of heat transfer from robotic hand to robotic arm, transient temperature distribution in one dimension of a rod periodically moving into a hot medium is found numerically. The rod is divided into two parts; the first part simulates the robotic hand and second part simulates the robotic arm. Using different thermal parameters and environmental conditions, transient and quasi-steady state t...

Robots can deal with different kinds of challenges using tactile sensing arrays as a primary resource. This paper demonstrates the ability of robotic hands to recognize objects' shapes using only a flexible tactile sensor arrays attached to the robotic hand's surface without building the 3D models of objects. A telemanipulation module was developed to achieve a co-moving mechanism between the robotic hand and human hands so that the robotic hand can directly learn the best way to grasp objects from human hands without additional path planning process. Tactile array data were collected while the robotic hand was performing a reiterative grasping process. By extracting the proper features from the tactile sensor array data, the support vector machines (SVMs) were employed to perform object classification. From experiments, the proposed method can achieve 96.67% classification accuracy based on sensory data and SVMs.

This paper presents an electromyographic (EMG) signal integrated multi-finger robot hand control for massage applications. This research explores the feasible application of multi-finger robot hands except for the use as prostheses and grasping applications. The forearm EMG of a person who is massaged by the human hands is recorded and analyzed statistically. First, the root mean square (RMS) of the raw data is computed as the discrimination between normal and contracted states of the muscle. Then the EMG signal at contracted state is further divided into painful and comfortable groups based on the impulse factor which is defined to estimate the sharpness of waveform variations. As a consequence, two discriminative values of the EMG signal are generated to distinguish painful and comfortable feelings. Based on the relationship between the human feeling and the massage force, we get an appropriate range of input commands of the robot hand for massage applications. A grasp-kneading massage is performed on the human shoulder to verify the proposed process. As a result, an effective and comfortable massage using the multi-finger robot hand is realized.

The multi-fingered robot hand has much attention in various fields. Many robot hands have been proposed so far. However, the robot hand cannot execute any tasks autonomously because the robot hand does not have enough motion and sensing ability to task complexities. Therefore, we have developed a robot hand teleoperation system with the motion capture data glove CyberGlove. Here, each joint of the robot hand is controlled according to the corresponding joint of the human hand. In this paper, we develop the teleoperation method that the robot hand joint is controlled to the multiple human hand joints. The positional error of the fingertips can be decrease by using the multiple joints reference. We show the effectiveness of the developed method through some experiments.

This paper presents a massage motion control system comprised of position control and force control in a multi-fingered robot hand. By making use of an algorithm which converted the desired fingertip trajectory into the desired angle of links in each finger by means of inverse kinematics, the finger position control from the initial position of the multi-fingered robot hand to a target position of the objects for massage was achieved. Its controller was used until the robot hand contacted the objects for massage. After contact was made, the fingertip position control was switched to a force control position needed to apply pressure for the massage. The fingertip forces exerted by an expert human therapist was measured using sheet distribution pressure sensors, and the data obtained was recorded in a computer. After the measurements were taken, the human expert's fingertip force was reproduced by the robot hand. The fingertip force of the robot hand was controlled using feedback obtained with a 6-axis force sensor. To make the force of each fingertip of the four-fingered robot hand track to the fingertip force exerted by the expert human massage therapist, PI servo compensation and a Jacobian matrix were really applied for the human's shoulder. Through simulation and experiments, the usefulness of the proposed control systems was demonstrated.

The human hand is an amazing tool, demonstrated by its incredible motor capability and remarkable sense of touch. To enable robots to work in a human-centric environment, it is desirable to endow robotic hands with human-like capabilities for grasping and object manipulation. However, due to its inherent complexity and inevitable model uncertainty, robotic grasping and manipulation remains a challenge. This thesis focuses on grasp adaptation in the face of model and sensing uncertainties: Given an object whose properties are not known with certainty (e.g., shape, weight and external perturbation), and a multifingered robotic hand, we aim at determining where to put the fingers and how the fingers should adaptively interact with the object using tactile sensing, in order to achieve either a stable grasp or a desired dynamic behaviour. A central idea in this thesis is the object-centric dynamics: namely, that we express all control constraints into an object-centric representation. This simplifies computa- tion and makes the control versatile to the type of hands. This is an essential feature that distinguishes our work from other robust grasping work in the literature, where generating a static stable grasp for a given hand is usually the primary goal. In this thesis, grasp adaptation is a dynamic process that flexibly adapts the grasp to fit some purpose from the objectâ s perspective, in the presence of a variety of uncertainties and/or perturbations. When building a grasp adaptation for a given situation, there are two key problems that must be addressed: 1) the problem of choosing an initial grasp that is suitable for future adaptation, and more importantly 2) the problem of design- ing an adaptation strategy that can react adequately to achieve desired behaviour of the grasped object. To address challenge 1 (planning a grasp under shape uncertainty), we propose an approach to parameterizing the uncertainty in object shape using Gaussian Processes (GPs) and incorporate it as a constraint into contact-level grasp planning. To realize the planned contacts using different hands interchangeably, we further develop a prob- abilistic model to predict the feasible hand configurations, including hand pose and finger joints, given the desired contact points only. The model is built using the con- cept of Virtual Frame(VF), and it is independent from the choice of hand frame and object frame. The performance of the proposed approach is validated on two differ- ent robotic hands, an industrial gripper (4 DOF Barrett hand) and a humanoid hand (16 DOF Allegro hand) to manipulate objects of daily use with complex geometry and various texture (a spray bottle, a tea caddy, a jug and a bunny toy). In the second part of this thesis, we propose an approach to the design of adapta- tion strategy to ensure grasp stability in the presence of physical uncertainties of objects(object weight, friction at contacts and external perturbation). Based on an object-level impedance controller, we first design a grasp stability estimator in the object frame using the grasp experience and tactile sensing. Once a grasp is predicted to be unstable during online execution, the grasp adaptation strategy is triggered to improve the grasp stability, by either changing the stiffness at finger level or relocating the position of one fingertip to a better area.

The purpose of this paper is to present the expert massage robot using a multi-fingered robot hand. First, the fingertip forces applied by an expert human therapist was measured using sheet distrib...