Dual-arm Robot Research Articles

The Algorithm of Multiple Obstacle Avoidance Tasks for Dual-Arm Robots

The Algorithm of Multiple Obstacle Avoidance Tasks for Dual-Arm Robots

Learning Dynamic Robot-to-Robot Object Handover

Object handover is an essential skill for collaborative robots in both services robotics and manufacturing scenarios. Most previous works were conducted from the perspective of human-robot interaction. The object handover between robots for collaborative task execution, aiming at optimizing time efficiency with human-like smooth behaviour, has not been extensively addressed. In this work, we propose a skill framework based on variable impedance control and dynamic motion primitives to optimize not only the motion trajectories and variable impedance, but also the timing of hand actions during dynamic motion. The effectiveness of the proposed framework is evaluated on a real dual-arm robot system under two handover scenarios with different constraints on the timing of hand actions. The experiment results demonstrate significant time efficiency improvement with reduction of the execution time by 9.4% and 23.7%, compared with accelerating the motion speed of the demonstrated handover. Furthermore, it can be observed that the robot successfully learned dynamic object handover without requiring transfer action to be triggered after both hands stop and remain still. In addition, in the second experiment, it is shown that the object can be transferred even without ensuring firm contact, which indicates that object handover is possible to be realized by throwing-like motion.

Towards Haptic-Based Dual-Arm Manipulation

Vision is the main component of current robotics systems that is used for manipulating objects. However, solely relying on vision for hand-object pose tracking faces challenges such as occlusions and objects moving out of view during robotic manipulation. In this work, we show that object kinematics can be inferred from local haptic feedback at the robot-object contact points, combined with robot kinematics information given an initial vision estimate of the object pose. A planar, dual-arm, teleoperated robotic setup was built to manipulate an object with hands shaped like circular discs. The robot hands were built with rubber cladding to allow for rolling contact without slipping. During stable grasping by the dual arm robot, under quasi-static conditions, the surface of the robot hand and object at the contact interface is defined by local geometric constraints. This allows one to define a relation between object orientation and robot hand orientation. With rolling contact, the displacement of the contact point on the object surface and the hand surface must be equal and opposite. This information, coupled with robot kinematics, allows one to compute the displacement of the object from its initial location. The mathematical formulation of the geometric constraints between robot hand and object is detailed. This is followed by the methodology in acquiring data from experiments to compute object kinematics. The sensors used in the experiments, along with calibration procedures, are presented before computing the object kinematics from recorded haptic feedback. Results comparing object kinematics obtained purely from vision and from haptics are presented to validate our method, along with the future ideas for perception via haptic manipulation.

Cooperative Manipulation of Deformable Objects by Single-Leader–Dual-Follower Teleoperation

This article proposes a method for single-leader–dual-follower teleoperation, where one robot (direct-follower robot, DFR) is directly teleoperated and the other robot (assisting-follower robot, AFR) can autonomously cooperate with the DFR to hold and move a deformable object, with the contact force regulated to a desired value. Since the AFR does not know its partner’s movement, first, it achieves position alignment with the DFR by using the contact force. Second, we develop an adaptive movement estimation algorithm according to the Lyapunov theory, such that DFR’s position is estimated in the presence of dynamic uncertainties. Finally, we adopt the impedance control to generate a reference trajectory for the AFR to track, in order to maintain the desired contact force. Several simulations are conducted on a platform with two three-degrees-of-freedom robots. Experiments are performed with a dual-arm robot (Baxter), results of which demonstrate the feasibility and effectiveness of the proposed method.

Mixtures of Gaussian Processes for Robot Motion Planning Using Stochastic Trajectory Optimization

Robot motion planning methods based on trajectory optimization can efficiently generate feasible and optimal trajectories by minimizing a suitable cost function, even in high-dimensional spaces. However, the main drawback of these methods lies in their proneness to infeasible local minima, especially in complex environments. To mitigate this issue, we propose a novel motion planning method that represents trajectories as samples from a mixture of continuous-time Gaussian processes (MGP) and employs stochastic optimization in order to update the MGP parameters in a cost-minimizing manner. The contributions of the proposed trajectory optimization method arise from the introduced mixture representation and stochastic gradient estimation, dominantly enabling better exploration of the trajectory space and including nondifferentiable optimizing costs. We evaluated the proposed method in multiple simulation benchmarks featuring 7 degree-of-freedom (DOF) robot arms and a 10DOF mobile manipulator. We also conducted a real-world experiment with a 14DOF dual-arm robot. The experimental results demonstrated that the proposed method achieves higher success rate than several state-of-the-art methods, while the advantages stemming from MGPs and stochastic optimization, like trajectory smoothness, support of nondifferentiable cost functions, multiple trajectory solutions, and the ability to tackle high-dimensional planning problems, are inherently kept.

Graph-Based Framework on Bimanual Manipulation Planning from Cooking Recipe

It is difficult to effectively operate a dual-arm robot using only the information written in a cooking recipe. To cope with this problem, this paper proposes a graph-based approach on bimanual cooking motion planning from a cooking recipe. In our approach, we first decompose the cooking recipe into graph elements. Then, we try to connect the graph elements taking into account the attributes of the input/output nodes. If two graph elements cannot be connected to each other, we search for a graph element that can be inserted between them from a motion database. Since the constructed graph includes the whole sequence of the robot’s motions to perform the cooking task, we can generate a task sequence of a dual-arm manipulator simultaneously performing two different tasks by using two arms. Through experimental study, we show that it is possible to generate robot motions from a cooking recipe and perform the cooking motions while simultaneously moving the left and right arms.

Simulating dual-arm robot motions to avoid collision by rigid body dynamics for laboratory bench work

Simulating dual-arm robot motions to avoid collision by rigid body dynamics for laboratory bench work

A Metaheuristic Optimization Approach to Solve Inverse Kinematics of Mobile Dual-Arm Robots

This work presents an approach to solving the inverse kinematics of mobile dual-arm robots based on metaheuristic optimization algorithms. First, a kinematic analysis of a mobile dual-arm robot is presented. Second, an objective function is formulated based on the forward kinematics equations. The kinematic analysis does not require using any Jacobian matrix nor its estimation; for this reason, the proposed approach does not suffer from singularities, which is a common problem with conventional inverse kinematics algorithms. Moreover, the proposed method solves cooperative manipulation tasks, especially in the case of coordinated manipulation. Simulation and real-world experiments were performed to verify the proposal’s effectiveness under coordinated inverse kinematics and trajectory tracking tasks. The experimental setup considered a mobile dual-arm system based on the KUKA® Youbot® robot. The solution of the inverse kinematics showed precise and accurate results. Although the proposed approach focuses on coordinated manipulation, it can be implemented to solve non-coordinated tasks.

Design and experiment of on-orbit assembly ground simulation robot

This paper proposes a space telescope assembly scheme and a ROS-based on-orbit assembly robot simulation system. First, the system fuses the hand-eye camera and the global camera ArUco marked image information to complete the identification and position and pose estimation of the structural parts to be assembled. Then, the pose information is transmitted to the ROS motion control node through Modbus-TCP communication, and the dual-arm robot system is controlled to complete motion planning and obstacle avoidance, arriving at the designated position for precise pick and place. After many simulation experiments of the designed space telescope model assembly scheme, it has been proved that the target object pose estimation algorithm is accurate, and the dual-arm collaborative robot has high position accuracy, which can realize the ground simulation autonomous assembly of large-scale space facilities. The system has certain practical application value with high integration, strong expansibility and good portability.

Development of display and disposal work system for convenience stores using dual-arm robot

ABSTRACT Labor shortages have been reported in numerous developed economies, and they are becoming increasingly severe, particularly in convenience stores. An important task in a convenience store is the display and disposal of a wide variety of products. However, there are technological gaps between the present technology and the expected setup. To solve the demand for the display and disposal tasks in convenience stores, ‘World Robot Summit Future Convenience Store Challenge(WRS FCSC)’ was established in 2017. The tasks of WRS FCSC are important; however, it is difficult to completely achieve the required tasks because the tasks in convenience stores are optimized for human workers. In this study, we first analyze the functions required to solve the problems of tasks in WRS FCSC. Through the analysis, we have developed several system components including two kinds of grippers for dual-arm and its manipulation strategy, intelligent shelf with manipulation support function and human detection function surrounding the shelf, object pose detection algorithm, precise autonomous navigation strategy, and management system to integrate the all the components. The effectiveness of the developed system is evaluated through the same scenario of WRS FCSC. Through the experiment, we confirmed the effectiveness our developed system.

Dual-Arm Visuo-Haptic Optical Tweezers for Bimanual Cooperative Micromanipulation of Nonspherical Objects.

Cooperative manipulation through dual-arm robots is widely implemented to perform precise and dexterous tasks to ensure automation; however, the implementation of cooperative micromanipulation through dual-arm optical tweezers is relatively rare in biomedical laboratories. To enable the bimanual and dexterous cooperative handling of a nonspherical object in microscopic workspaces, we present a dual-arm visuo-haptic optical tweezer system with two trapped microspheres, which are commercially available end-effectors, to realize indirect micromanipulation. By combining the precise correction technique of distortions in scanning optical tweezers and computer vision techniques, our dual-arm system allows a user to perceive the real contact forces during the cooperative manipulation of an object. The system enhances the dexterity of bimanual micromanipulation by employing the real-time representation of the forces and their directions. As a proof of concept, we demonstrate the cooperative indirect micromanipulation of single nonspherical objects, specifically, a glass fragment and a large diatom. Moreover, the precise correction method of the scanning optical tweezers is described. The unique capabilities offered by the proposed dual-arm visuo-haptic system can facilitate research on biomedical materials and single-cells under an optical microscope.

Coordinated control of dual-arm robot on space structure for capturing space targets

This paper presents a coordinated control scheme for two capture tasks via a dual-arm space robot with its base on a flexible space structure. The robot system consists of a base, a mission arm for capture, and a balance arm for removing disturbance forces to the base. To counteract the impact of structural vibration, a two-stage strategy containing a vibration control stage and a target capture stage is proposed. In the first stage, the robot and structure are treated as a mass-spring system in which the coupling effect between the robot and the flexible structure is exploited to suppress structural vibration. Notably, the collision avoidance between two arms is achieved by a velocity inequality constraint, wherein the reactionless motion is utilized. In the second stage, a reactionless trajectory is planned for the mission arm to reach the target location and attitude, as the balance arm remove the coupling disturbance to the base. Afterwards, the balance arm becomes the new mission arm for capture and the other arm is defined as the new balance arm. Finally, numerical simulations are given to validate the efficiency of the proposed coordinated control strategy.

Solving a Simple Geduldspiele Cube with a Robotic Gripper via Sim-to-Real Transfer

Geduldspiele cubes (also known as patience cubes in English) are interesting problems to solve with robotic systems on the basis of machine learning approaches. Generally, highly dexterous hand and finger movement is required to solve them. In this paper, we propose a reinforcement-learning-based approach to solve simple geduldspiele cubes of a flat plane, a convex plane, and a concave plane. The key idea of the proposed approach is that we adopt a sim-to-real framework in which a robotic agent is virtually trained in simulation environment based on reinforcement learning, then the virtually trained robotic agent is deployed into a physical robotic system and evaluated for tasks in the real world. We developed a test bed which consists of a dual-arm robot with a patience cube in a gripper and the virtual avatar system to be trained in the simulation world. The experimental results showed that the virtually trained robotic agent was able to solve simple patience cubes in the real world as well. Based on the results, we could expect to solve the more complex patience cubes by augmenting the proposed approach with versatile reinforcement learning algorithms.

Real-time obstacle avoidance algorithm and coordination motion strategy for a redundant dual-arm robot

Real-time obstacle avoidance algorithm and coordination motion strategy for a redundant dual-arm robot

A Dual-Arm Robot That Autonomously Lifts Up and Tumbles Heavy Plates Using Crane Pulley Blocks

This paper proposes a combined planning and optimization method that enables a dual-arm robot to lift up and flip heavy plates using crane pulley blocks. The problem is motivated by the low payload of modern collaborative robots. Instead of directly manipulating heavy plates that collaborative robots cannot afford, the paper develops a planner for collaborative robots to operate crane pulley blocks. The planner assumes a target plate is pre-attached to the crane hook. It optimizes dual-arm action sequences and plans the robot’s dual-arm motion that pulls the rope of the crane pulley blocks to lift up the plate. The crane pulley blocks reduce the payload that each robotic arm needs to bear. When the plate is lifted up to a satisfying pose, the planner plans a sliding-pushing motion for one of the robot arms to tumble over the plate while considering force and moment constraints. The article presents the technical details of the planner and several experiments and analysis carried out using a dual-arm robot made by two Universal Robots UR3 arms. The influence of various parameters and optimization goals are investigated and compared in depth. The results show that the proposed planner is flexible and efficient. This paper is motivated by a cleaning process in a factory that produces sewage press machines. The pressboard of sewage press machines could be as heavy as 1000 kg. Human workers need to flip and clean both sides of the board before installing them to the main axis of a sewage machine. Their solution is using a gantry crane. They attach the board to the crane hook using bearing belts, activate the crane to lift up the board. When the board is raised to a satisfying pose, the workers turn the board over by pushing it. Motivated by human workers’ actions, we developed the planner presented in this paper. We assumed crane pulley blocks in the experiments and analysis, but in practice, they may be replaced with electronic ones to improve effort and efficiency. Using the electronic ones will be a sub-problem since there is no need for pulling ropes. The proposed method is expected to help a company’s technicians better judge if they need a heavy payload manipulator or keep their current crane equipment while employing several intelligent collaborative robots to operate them. As a result, it may help to accelerate the upgrade of manufacturing sites while reducing reforming budgets. Note to Practitioners—This paper is motivated by a cleaning process in a factory that produces sewage press machines. The pressboard of sewage press machines could be as heavy as 1000 kg. Human workers need to flip and clean both sides of the board before installing them to the main axis of a sewage machine. Their solution is using a gantry crane. They attach the board to the crane hook using bearing belts, activate the crane to lift up the board. When the board is raised to a satisfying pose, the workers turn the board over by pushing it. Motivated by human workers’ actions, we developed the planner presented in this paper. We assumed crane pulley blocks in the experiments and analysis, but in practice, they may be replaced with electronic ones to improve effort and efficiency. Using the electronic ones will be a sub-problem since there is no need for pulling ropes. The proposed method is expected to help a company’s technicians better judge if they need a heavy payload manipulator or keep their current crane equipment while employing several intelligent collaborative robots to operate them. As a result, it may help to accelerate the upgrade of manufacturing sites while reducing reforming budgets.

Four-Arm Collaboration: Two Dual-Arm Robots Work Together to Manipulate Tethered Tools

This article presents a planner for a master dual-arm robot to manipulate tethered tools with the help of an assistant dual-arm robot. The assistant robot assists the master robot by manipulating the tool cable and avoiding collisions. The provided assistance allows the master robot to perform tool placements on the robot workspace table and regrasp it, which would typically fail since cable tension may change the tool positions. It also allows the master robot to perform tool handovers, which would normally cause entanglements or collisions with the cable and the environment without assistance. Simulations and real-world experiments are performed to validate the proposed planner. Experimental results show that with the help of the assistant dual-arm robot and the proposed method, the master robot obtained improved flexibility in working with tethered tools.

Planning to Build Block Structures With Unstable Intermediate States Using Two Manipulators

The work is inspired by the assembly of Soma block puzzles. Soma block puzzles usually include unstable intermediate states that require additional support to maintain stability temporarily. In the puzzles’ solution manual, we can observe that designers consider the characteristics that humans have two hands and can avoid an unstable intermediate state by using one hand to support the finished component and using the other hand to assemble an upcoming workpiece. Motivated by human behavior, this paper develops a planner that automatically finds an optimal assembly sequence for a dual-arm robot to build a woodblock structure while considering various constraints and supporting grasps from a second hand. It uses the mesh model of wood blocks and the final assembly state to generate possible assembly sequences and evaluate the optimal assembly sequence by considering the stability, graspability, assemblability, and the need for a second hand. Especially, the need for a second hand is resolved when supports from worktables and other workpieces are not enough to produce a stable assembly. A second hand can hold and support the unstable components so that the robot can further assemble new workpieces until the structure state becomes stable again. The output of the planner includes the optimal assembly orders, candidate grasps, assembly directions, and the supporting grasps (if needed). The output can help guide a dual-arm robot to perform motion planning and thus generate assembly motion. Experiments using various blocks and structures show the effectiveness of the proposed planner. Note to Practitioners—The presented planner can generate an optimal assembly order for a large variety of structures like decoration accessories, furniture, home interiors, frames, etc., in the practices. They can also be used for scenarios that need stacking or piling up multiple objects. The generated optimal assembly order is more friendly to dual-arm robot systems than previous assembly planners that ignored the merits of robotic collaboration. Also, the proposed assembly planner generates the necessary information for the motion planner, such as grasp poses and optimal assembly directions. A motion planner can directly use the generated results to plan robotic assembly motion.. The proposed assembly planner is expected to significantly reduce human effort and increase the efficiency of robotic assembly lines.

Peg-in-Hole Assembly With Dual-Arm Robot and Dexterous Robot Hands

This study focuses on implementing robotic peg-in-hole in a more human-like approach, namely using dual-arms and dexterous robotic hands. The peg-in-hole strategy in this study mainly consists of two parts: grasping strategy (hand part) and assembly strategy (arm part). The grasping strategy explains the fundamental grasping method called “advanced blind grasping” and the in-hand manipulation method that is required for the reorientation of the workpiece. A feed-forward task space force control scheme is proposed for the actual implementation. The assembly strategy presents fundamental unit motions called “perturbation pattern” and proposes four assembly stages that are constructed by a combination of the unit motions. A force-position hybrid control for the implementation of the assembly strategy was also addressed. For evaluation, a peg-in-hole assembly demonstration with a keyhole-like shape was conducted using a human-sized 50-DOF upper-body robot.

Practical Tracking Control for Dual-arm Robot with Output Constraints

Practical Tracking Control for Dual-arm Robot with Output Constraints

Constrained Probabilistic Movement Primitives for Robot Trajectory Adaptation

Placing robots outside controlled conditions requires versatile movement representations that allow robots to learn new tasks and adapt them to environmental changes. The introduction of obstacles or the placement of additional robots in the workspace, the modification of the joint range due to faults or range-of-motion constraints are typical cases where the adaptation capabilities play a key role for safely performing the robot's task. Probabilistic movement primitives (ProMPs) have been proposed for representing adaptable movement skills, which are modelled as Gaussian distributions over trajectories. These are analytically tractable and can be learned from a small number of demonstrations. However, both the original ProMP formulation and the subsequent approaches only provide solutions to specific movement adaptation problems, e.g., obstacle avoidance, and a generic, unifying, probabilistic approach to adaptation is missing. In this paper we develop a generic probabilistic framework for adapting ProMPs. We unify previous adaptation techniques, for example, various types of obstacle avoidance, via-points, mutual avoidance, in one single framework and combine them to solve complex robotic problems. Additionally, we derive novel adaptation techniques such as temporally unbound via-points and mutual avoidance. We formulate adaptation as a constrained optimisation problem where we minimise the Kullback-Leibler divergence between the adapted distribution and the distribution of the original primitive while we constrain the probability mass associated with undesired trajectories to be low. We demonstrate our approach on several adaptation problems on simulated planar robot arms and 7-DOF Franka-Emika robots in a dual robot arm setting.