Dual-arm Robot Research Articles

Constrained Probabilistic Movement Primitives for Robot Trajectory Adaptation

Placing robots outside controlled conditions requires versatile movement representations that allow robots to learn new tasks and adapt them to environmental changes. The introduction of obstacles or the placement of additional robots in the workspace, the modification of the joint range due to faults or range-of-motion constraints are typical cases where the adaptation capabilities play a key role for safely performing the robot's task. Probabilistic movement primitives (ProMPs) have been proposed for representing adaptable movement skills, which are modelled as Gaussian distributions over trajectories. These are analytically tractable and can be learned from a small number of demonstrations. However, both the original ProMP formulation and the subsequent approaches only provide solutions to specific movement adaptation problems, e.g., obstacle avoidance, and a generic, unifying, probabilistic approach to adaptation is missing. In this paper we develop a generic probabilistic framework for adapting ProMPs. We unify previous adaptation techniques, for example, various types of obstacle avoidance, via-points, mutual avoidance, in one single framework and combine them to solve complex robotic problems. Additionally, we derive novel adaptation techniques such as temporally unbound via-points and mutual avoidance. We formulate adaptation as a constrained optimisation problem where we minimise the Kullback-Leibler divergence between the adapted distribution and the distribution of the original primitive while we constrain the probability mass associated with undesired trajectories to be low. We demonstrate our approach on several adaptation problems on simulated planar robot arms and 7-DOF Franka-Emika robots in a dual robot arm setting.

Learning to Cooperate: A Hierarchical Cooperative Dual Robot Arm Approach for Underactuated Pick-and-Placing

Cooperative multiagent manipulation systems allow to extend on the manipulative limitations of individual agents, increasing the complexity of the manipulation tasks the ensemble can handle. Controlling such a system requires meticulous planning of subsequent subtasks, queried to the individual agents, in order to execute the master task successfully. Real-time (re)planning is essential to ensure the task can still be achieved when subtasks execution suffers from uncertainty or when the master task changes intermittently requiring real-time reconfiguration of the plan. In this article, we develop a supervisory control architecture tailored to the cooperation of two robotic manipulators equipped with standard pick-and-place facilities in the plane. We control the planar position and orientation of an object using two underactuated manipulators so that only the position of the object can be controlled directly. The desired orientation follows from the accumulation of alternating relative angles. A time-invariant policy function is trained using deep reinforcement learning, which can determine a finite sequence of pick-and-place maneuvers to manipulate the object to a desired configuration. Two policy architectures are compared. The first uses the kinematic model to determine the final step, whilst the second policy makes this decision itself. The more information is given to the policy, the easier it trains. In return, it becomes less adaptable and loses some of its generalizability.

Dynamic Modeling and Control of Deformable Linear Objects for Single-Arm and Dual-Arm Robot Manipulations

Robotic manipulation of deformable linear objects (DLOs) is important in many applications such as the assembly of deformable wire harnesses and cables in manufacturing. Despite some relevant work in modeling, few studies have studied the comprehensive dynamic modeling and precise automatic control of DLOs using robots due to their high degrees of freedom and high flexibility. To fill this gap, the precise single-arm and dual-arm robot manipulation control of DLOs is studied in this article. First, a more comprehensive dynamic model of DLOs is established based on a discrete elastic rod model, which takes into account the twisting deformation of DLOs. The collisions, contacts, and frictions between the DLOs and the plane, as well as the kinematic constraints of both ends of the DLOs, are also considered in the model. Second, practical dynamic control schemes of robots are proposed to realize the precise control of DLOs for both the single-arm control and dual-arm control. For validations, we built an experimental platform using an ABB Yumi robot to implement and validate the proposed approaches in addition to simulations. Finally, various DLO manipulation tasks are conducted and the results for both single-arm and dual-arm manipulations validate the proposed modeling and control approaches.

Prescribed performance adaptive control of dual-arm robots with guaranteed motion precision

Prescribed performance adaptive control of dual-arm robots with guaranteed motion precision

Contouring Control of an Innovative Manufacturing System Based on Dual-Arm Robot

In this article, an innovative manufacturing system based on a dual-arm robot, each arm with five degree-of-freedom (DOF), is proposed. Although the proposed system can work in the automation mode or machining mode, only the function of machining is studied here. In this case, the system is equivalent to a five-axis machine tool. The method of equivalent errors is employed to design a robust contouring controller. It is shown that reducing the equivalent contour error in the joint space is equivalent to reducing both position contour error and tool orientation error in the task space. As a comparison, a conventional distributed controller is also designed. Several cases of experiments have been conducted, including a circular path with four different feedrates and an elliptic path. The experimental results show that the proposed method can achieve much better results in all cases, and demonstrate the feasibility of adopting the dual-arm robotic system for machining task. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This article was motivated by the traditional manufacturing system that is a cooperation between computer numerical controlled (CNC) machine tool(s) and robot arm(s). This article proposes a novel usage of a dual-arm robot with a fixed spindle and cutting tool in order to integrate the main functions of the CNC machine and robot arm together. This system alone can perform machining and automation tasks, which can significantly reduce the overall cost of a manufacturing system. The existing dual-arm robots have seldom been designed to perform machining tasks. For machining, a contouring controller is necessary and is proposed in this article. This model-based contouring controller relies on the accurate system parameters to achieve high precision machining. For example, the mass and inertia changes of a workpiece during machining should be considered. This issue will be addressed in our future research.

Assembly-Oriented Task Sequence Planning for a Dual-Arm Robot

One of the goals for the deployment of dual-arm robots is to imitate and replace human workers to complete manufacturing tasks. Moreover, especially in scenarios where workpieces are randomly scattered on the workbench, endowing a robotic system with a capacity of scheduling task sequences will undoubtedly expand its application prospects and improve the autonomy. For that purpose, this paper proposes a dual-arm robot task sequence planning approach based on environmental constraints and causal reasoning among tasks. In this work, the Monte Carlo method, the Gaussian Mixture Model and the binary functions are adopted to evaluate the constraints from the robots. Meanwhile, constraints from workpieces are addressed with a geometric method combined with its semantic relations. In addition to the environmental constraints, which consist of constraints from the robot and workpieces, the causal relations among tasks are considered. Finally, all the above information is exploited to construct a graphical structure for task sequence planning, where workpieces are regarded as vertices and their semantic relations are edges with attributes. The effectiveness of the approach is demonstrated using various simulation experiments within different scene layouts rather than using several specified tasks.

Fast pivoting gait generation by model predictive control designed with basis functions

Pivoting gait is an efficient way for robots to manipulate a heavy object. Although we can cope with the contact constraints of the pivoting gait by using the model predictive control (MPC), systems with complex dynamics, including the pivoting gait, usually require long horizons in the MPC and it leads to a heavy computational load. To overcome this problem, we introduce basis functions to parameterize the free variables in the MPC and formulate an optimization problem with new decision variables, which are coefficients of the basis functions. We especially introduce multiple basis functions and compare their performances in generating the robotic pivoting gait. As a result, the most effective reduction in the dimension of the free variables is achieved by using the Laguerre basis function and the computational efficiency of the MPC is greatly improved. The simulation and experiments show that the time cost of the generation of pivoting gaits by the proposed method is remarkably reduced and the generated pivoting gaits are feasible and robust where a dual-arm robot successfully manipulates a toy piano by the pivoting gait.

An admittance based hierarchical control framework for dual-arm cobots

Dual-arm robotic platforms have solid arguments to match the growing need for versatility in the industry. Coupling the control of two manipulators for cooperative purposes enlarges the scope of feasible operations, while adding perception capabilities allows to navigate in dynamic environments. In this respect, we propose a complete online kinematic control framework for dual-arm robots operating in unstructured industrial settings. We base our approach on admittance control in the cooperative task space. Regulating internal and external efforts offer safe bimanual task execution and enables physical interaction. We implement a hierarchical quadratic programming architecture that applies a prioritization of tasks: most efforts are concentrated on the proper tracking of relative motions of the arms, which is the most critical for safety reasons. We demonstrate the performance of our framework through a “teaching-by-demonstration” experiment on the dual-arm mobile cobot BAZAR.

Trajectory optimisation in collaborative robotics based on simulations and genetic algorithms

The separation distance between humans and robots in manufacturing shop-floors has been progressively reduced, thanks to the modern safety functionalities available in robot controllers. However, the activation of these safety criteria usually stops the production or reduces the productivity of machines and robots. With the aim of improving this situation, this paper presents a real-time trajectory optimisation method for collaborative robots. The robot trajectory is parameterised at instruction level, i.e. through the parameters characterizing the robot motion instruction. A genetic algorithm randomly modifies the parameters of the nominal trajectory of the robot, thus obtaining new sets of candidate trajectories. Each trajectory is simulated on a digital twin of the collaborative workspace, which allows to reproduce and simulate the robot motion, and to represent the volume of the work-cell occupied by the human operator. A lexicographic optimization is used to evaluate online the optimal robot trajectory that simultaneously minimizes the risk of collision with the human operator and the trajectory traversal time. The method is validated in an industrial scenario involving the ABB YuMi dual-arm robot for a small parts assembly task.

Self-Collision Avoidance Control of Dual-Arm Multi-Link Robot Using Neural Network Approach

The problem of mutual collisions of manipulators of a dual-arm multi-link robot (so-called self-collisions) arises during the performance of a cooperative technological operation. Self-collisions can lead to non-fulfillment of the technological operation or even to the failure of the manipulators. In this regard, it is necessary to develop a method for online detection and avoidance of self-collisions of manipulators. The article presents a method for detecting and avoiding self-collisions of multi-link manipulators using an artificial neural network by the example of the dual-arm robot SAR-401. A comparative analysis is carried out and the architecture of an artificial neural network for self-collisions avoidance control of dual-arm robot manipulators is proposed. The novelty of the proposed approach lies in the fact that it is an alternative to the generally accepted methods of detecting self-collisions based on the numerical solution of inverse kinematics problems for manipulators in the form of nonlinear optimization problems. Experimental results performed based on MATLAB model, the simulator of the robot SAR-401 and on the real robot itself confirmed the applicability and effectiveness of the proposed approach. It is shown that the detection of possible self-collisions using the proposed method based on an artificial neural network is performed approximately 10 times faster than approaches based on the numerical solution of the inverse kinematics problem while maintaining the specified accuracy.

Impedance control method with reinforcement learning for dual-arm robot installing slabstone

Impedance control method with reinforcement learning for dual-arm robot installing slabstone

Learning garment manipulation policies toward robot-assisted dressing.

Assistive robots have the potential to support people with disabilities in a variety of activities of daily living, such as dressing. People who have completely lost their upper limb movement functionality may benefit from robot-assisted dressing, which involves complex deformable garment manipulation. Here, we report a dressing pipeline intended for these people and experimentally validate it on a medical training manikin. The pipeline is composed of the robot grasping a hospital gown hung on a rail, fully unfolding the gown, navigating around a bed, and lifting up the user's arms in sequence to finally dress the user. To automate this pipeline, we address two fundamental challenges: first, learning manipulation policies to bring the garment from an uncertain state into a configuration that facilitates robust dressing; second, transferring the deformable object manipulation policies learned in simulation to real world to leverage cost-effective data generation. We tackle the first challenge by proposing an active pre-grasp manipulation approach that learns to isolate the garment grasping area before grasping. The approach combines prehensile and nonprehensile actions and thus alleviates grasping-only behavioral uncertainties. For the second challenge, we bridge the sim-to-real gap of deformable object policy transfer by approximating the simulator to real-world garment physics. A contrastive neural network is introduced to compare pairs of real and simulated garment observations, measure their physical similarity, and account for simulator parameters inaccuracies. The proposed method enables a dual-arm robot to put back-opening hospital gowns onto a medical manikin with a success rate of more than 90%.

An algorithm for trajectory optimization of dual-arm coordination based on arm angle constraints

In this paper, the motion planning of a dual-arm robot with kinematic constraints is studied based on arm-angle constraints. When a dual-arm robot moves a common object, a closed kinematic chain is formed between the dual-arm and the object. The standard sampling-based trajectory planning algorithm solves the problem with closed-chain constraint, but this causes other problems; the running time increases, the success rate decreases, and the motion trajectory of the end effector is not smooth resulting in large output error. Therefore, this paper proposes a dual-arm coordinated trajectory optimization algorithm based on arm-angle constraints. Firstly, the kinematics of the dual-arm robot is modeled and analyzed, and the definition of the arm-angle in a seven-axis robot is proposed, the workspace of the dual-arm coordinated operation is considered to constrain it, the kinematics equation combined with the single/multi-objective optimization algorithm is used to optimize the end output error, and the joint trajectory is parameterized. This paper solves the problems that the slave arm lags behind the main arm, the motion trajectory of the dual-arm is not smooth, and the dual-arm are squeezed due to internal force during the coordinated movement of the dual-arm. The trajectory optimization improves the synchronization of the coordinated operation of the dual-arm, reduces the output error of the velocity and acceleration at the end of the dual-arm. After limiting the arm-angle, dual-arm manipulation is anthropomorphic the robot does not produce distorted arm configurations.

Obstacle Avoidance Path Planning for the Dual-Arm Robot Based on an Improved RRT Algorithm

In the future of automated production processes, the manipulator must be more efficient to complete certain tasks. Compared to single-arm robots, dual-arm robots have a larger workspace and stronger load capacity. Coordinated motion planning of multi-arm robots is a problem that must be solved in the process of robot development. This paper proposes an obstacle avoidance path planning method for the dual-arm robot based on the goal probability bias and cost function in a rapidly-exploring random tree algorithm (GA_RRT). The random tree grows to the goal point with a certain probability. At the same time, the cost function is calculated when the random state is generated. The point with the lowest cost is selected as the child node. This reduces the randomness and blindness of the RRT algorithm in the expansion process. The detection algorithm of the bounding sphere is used in the process of collision detection of two arms. The main arm conducts obstacle avoidance path planning for static obstacles. The slave arm not only considers static obstacles, but also takes on the role of the main arm at each moment as a dynamic obstacle for path planning. Finally, MATLAB is used for algorithm simulation, which proves the effectiveness of the algorithm for obstacle avoidance path planning problems for the dual-arm robot.

An Online Task-Planning Framework Using Mixed Integer Programming for Multiple Cooking Tasks Using a Dual-Arm Robot

This work proposes an online task-scheduling method using mixed-integer programming for a multi-tasking problem regarding a dual-arm cooking robot in a controlled environment. Given each task’s processing time, their location in the working space, dependency, the required number of arms, and the kinematic constraints of the dual-arm robot, the proposed optimization algorithm can produce a feasible solution to scheduling the cooking order for each task and for each associated arms so that the total cooking time and the total moving distance for each arm are minimized. We use a subproblem optimization strategy in which the number of tasks to be planned is divided into several groups instead of planning all tasks at the same time. By doing so, the planning time can be significantly decreased, making the algorithm practical for online implementation. The feasibility of our optimization method and the effectiveness of the subproblem optimization strategy were verified through simulated experiments consisting of 30 to 120 tasks. The results showed that our strategy is advantageous in terms of computation time and makespan for large problems.

New Jerk-Level Configuration Adjustment Schemes Applied to Constrained Redundant Robots

In this article, the new jerk-level configuration adjustment (JLCA) schemes are proposed to achieve the configuration adjustment of constrained redundant robots. Specifically, by applying the zeroing neurodynamics design rule three times, the new JLCA performance index is first derived; then, together with the joint physical constraints incorporated, the new JLCA schemes are obtained for dual-arm and single-arm redundant robots, respectively. For comparison purposes, three other configuration adjustment schemes are also presented. Moreover, the comparative simulative experiments based on a planar dual-arm redundant robot (i.e., five-link dual-arm robot) and a spatial single-arm redundant robot (i.e., Kinova JACO <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^2$</tex-math></inline-formula> robot) are performed to verify the efficacy and superiority of the proposed JLCA schemes, as compared with the three other configuration adjustment schemes. At last, the comparative physical experiments are conducted on the real Kinova JACO <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^2$</tex-math></inline-formula> robot to substantiate the practicability and excellent performance of the proposed JLCA scheme for single-arm redundant robots.

Coupled Multiple Dynamic Movement Primitives Generalization for Deformable Object Manipulation

Dynamic Movement Primitives (DMP) are widely applied in movement representation due to their ability to encode tasks using generalization properties. However, the coupled multiple DMP generalization cannot be directly solved based on the original DMP formula. Prior works provide satisfactory performance for the coupled DMP generalization in rigid object manipulation, but their extension to deformable objects may degrade due to the intrinsic uncertainty of the deformable model structure and parameters. This letter introduces an adaptive term to replace the fixed term to couple multiple DMP generalizations and model the deformable object using the classic mass-spring-damper model. Based on the modeling, the manipulation of a deformable object can be treated as a second-order system, which provides additional implementation flexibility and robustness in deformable object transportation. To validate the proposed method, we perform extensive simulations for cooperatively transporting a rope and a deformable thin film, imitating the manipulation with a semi-ellipse trajectory and M-shape trajectory. We further implement our method on a dual-arm robot platform for rope manipulation with depth visual feedback. Both simulation and experiment results demonstrate satisfactory DMP generalization, collision avoidance, and configuration preservation.

Dual-Arm Robot Trajectory Planning Based on Deep Reinforcement Learning under Complex Environment.

In this article, the trajectory planning of the two manipulators of the dual-arm robot is studied to approach the patient in a complex environment with deep reinforcement learning algorithms. The shape of the human body and bed is complex which may lead to the collision between the human and the robot. Because the sparse reward the robot obtains from the environment may not support the robot to accomplish the task, a neural network is trained to control the manipulators of the robot to prepare to hold the patient up by using a proximal policy optimization algorithm with a continuous reward function. Firstly, considering the realistic scene, the 3D simulation environment is built to conduct the research. Secondly, inspired by the idea of the artificial potential field, a new reward and punishment function was proposed to help the robot obtain enough rewards to explore the environment. The function is consisting of four parts which include the reward guidance function, collision detection, obstacle avoidance function, and time function. Where the reward guidance function is used to guide the robot to approach the targets to hold the patient, the collision detection and obstacle avoidance function are complementary to each other and are used to avoid obstacles, and the time function is used to reduce the number of training episode. Finally, after the robot is trained to reach the targets, the training results are analyzed. Compared with the DDPG algorithm, the PPO algorithm reduces about 4 million steps for training to converge. Moreover, compared with the other reward and punishment functions, the function used in this paper will obtain many more rewards at the same training time. Apart from that, it will take much less time to converge, and the episode length will be shorter; so, the advantage of the algorithm used in this paper is verified.

Adaptive hybrid impedance control for dual-arm cooperative manipulation with object uncertainties

In this paper, a novel adaptive hybrid impedance control for a dual-arm robot is proposed to cooperatively manipulate unknown objects. The proposed adaptive hybrid control scheme could be employed to improve the control performance of motion tracking and internal force regulation when dual-arm cooperative robot grasp an unknown object. Along with the analysis of human motion in the process of transferring, the controller has a master–slave structure in three levels: the first level is an absolute-relative motion controller satisfying the closed-chain constraints; the second level introduces an adaptive impedance control with variable stiffness to regulate the internal force; the motion and force control signals are fused in the third level with a select matrix. The controller uses the forces sensed at the end-effectors to compute the internal force with only kinematics, and the dynamics of the object is not required. To verify the effectiveness of the algorithm, a dual-arm robotic system is set up with two UR5 arms and the experimental results illustrate the performance and feasibility of the proposed control strategy.

Patient-Specific Magnetic Catheters for Atraumatic Autonomous Endoscopy.

Despite increasing interest in minimally invasive surgical techniques and related developments in flexible endoscopes and catheters, follow-the-leader motion remains elusive. Following the path of least resistance through a tortuous and potentially delicate environment without relying on interaction with the surrounding anatomy requires the control of many degrees of freedom. This typically results in large-diameter instruments. One viable solution to obtain dexterity without increasing size is via multiple-point magnetic actuation over the length of the catheter. The main challenge of this approach is planning magnetic interaction to allow the catheter to adapt to the surrounding anatomy during navigation. We design and manufacture a fully shape-forming, soft magnetic catheter of 80 mm length and 2 mm diameter, capable of navigating a human anatomy in a follow-the-leader fashion. Although this system could be exploited for a range of endoscopic or intravascular applications, here we demonstrate its efficacy for navigational bronchoscopy. From a patient-specific preoperative scan, we optimize the catheters' magnetization profiles and the shape-forming actuating field. To generate the required transient magnetic fields, a dual-robot arm system is employed. We fabricate three separate prototypes to demonstrate minimal contact navigation through a three-dimensional bronchial tree phantom under precomputed robotic control. We also compare a further four separate optimally designed catheters against mechanically equivalent designs with axial magnetization profiles along their length and only at the tip. Using our follow-the-leader approach, we demonstrate up to 50% more accurate tracking, 50% reduction in obstacle contact time during navigation over the state of the art, and an improvement in targeting error of 90%.