Dual-arm Manipulator Research Articles

Centralized vs. Decoupled Dual-arm Planning Taking into Account Path Quality

The aim of coordinated planning is to avoid robot-to-robot collisions in a multi-robot system, and there are two standard solution approaches: centralized planning and decoupled planning. Our first contribution is a decoupled planning approach that ensures C2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{\\mathcal {C}^2}$$\\end{document}-continuous control commands with zero velocities and zero accelerations at the start and goal. We benchmark our decoupled approach with a centralized approach. Contrary to literature, we show that for a standard motion planning pipeline, such as the one used by MoveIt!, centralized planning is superior to decoupled planning in dual-arm manipulation: It has a lower computation time and a higher robustness. Our second contribution is an optimization that minimizes the rotational motion of an end-effector while considering obstacle avoidance. We derive the analytic gradients of this optimization problem, making the algorithm suitable for online motion planning. Our optimization extends an existing path quality improvement method. Integrating it into our decoupled approach overcomes its shortcomings and provides a motion planning pipeline that is robust at up to 99.9 % with a planning time of less than 1 s and that computes high-quality paths.

Sensor placement determination for a wearable device in dual-arm manipulation tasks

To enhance the quality of life for individuals in various sectors, such as industry, healthcare, and everyday activities, the development of a highly efficient and cost-effective wearable device is essential. This paper presents a feasibility study for a general estimation methodology for human activity assessment using a single sensor placed on the back. The methodology considers the minimal variability in human body movements when carrying different weights. Our approach involves optimizing the sensor's position through a simple pre-processing technique. This technique utilizes virtual sensor position and orientation data derived from a virtual mannequin based on real inertial measurement unit (IMU) measurements obtained during two-arm manipulation tasks in a warehouse setting. The primary contribution is the optimization of a single, non-invasive sensor placement on the back to accurately assess activities involving two-hand manipulation with low variability in movement. We identified 54 different activity classes with an accuracy of 91.77% using a hybrid two-dimensional Convolutional Neural Network (CNN) combined with a Bidirectional Long Short-Term Memory (BiLSTM) network (2D CNN-BiLSTM) model. This model does not require additional sensors that measure other physical quantities, such as electromyography activity. Identifying repetitive tasks involving significant bending, stretching, and twisting, which pose risks of musculoskeletal disorders and back pain, offers a solution for designing wearable devices. The portability and autonomy of these devices are crucial, and current sensor technology meets these needs with low size, cost, and consumption. Wearable medical devices can thus be effectively used for self-monitoring health, preventive medicine, and rehabilitation.

The design and implementation of automated maintenance system for the first-wall based on dual-arm manipulator

This paper proposed a framework of a dual-arm-based robotic maintenance system, including visual recognition, trajectory planning, force feedback control and master–slave control. To meet the requirements of automated maintenance, we proposed an improved design for the structures of the first wall tiles and support boards, and established a complete communication structure for the maintenance system that can adapt to different hardware versions. Based on the proposed framework, an experimental platform with dual-arm manipulator was established to demonstrate the maintenance scenario of the FW of the blanket in Vacuum Vessel (VV). The experimental result verified the feasibility of automated robotic maintenance system applied to the future fusion reactor.

Force and time-optimal trajectory planning for dual-arm unilateral cooperative grasping

This paper studies the dual-arm manipulation of an object by means of two collaborative robots. The latter hold the object through limited contact areas, thus applying unilateral contact constraints. This manipulation strategy increases versatility, since it does not require specific grippers depending on the object shape and size. However, to ensure grasping stability (i.e. no slipping of the object), a suitable internal force must be prescribed to ensure the fulfillment of the static-friction condition. In this work, the trend of the internal force is included among the inputs of a time-optimal trajectory planning, in order to find the minimal internal prestress that is able to both satisfy the static-friction condition and manipulate the object in minimal time. Admittance control is used to modulate the forces exerted by the robot end-effectors on the object. An extensive experimentation, on different 6-dimensional trajectories reaching linear and angular accelerations up to 4.5 m/s2 and 7.4 rad/s2, is presented and discussed.

Velocity-Free Prescribed Performance Control of Dual-Arm Manipulator Based on Neural Networks

Velocity-Free Prescribed Performance Control of Dual-Arm Manipulator Based on Neural Networks

FabricFolding: learning efficient fabric folding without expert demonstrations

AbstractAutonomous fabric manipulation is a challenging task due to complex dynamics and potential self-occlusion during fabric handling. An intuitive method of fabric-folding manipulation first involves obtaining a smooth and unfolded fabric configuration before the folding process begins. However, the combination of quasi-static actions like pick & place and dynamic action like fling proves inadequate in effectively unfolding long-sleeved T-shirts with sleeves mostly tucked inside the garment. To address this limitation, this paper introduces an enhanced quasi-static action called pick & drag, specifically designed to handle this type of fabric configuration. Additionally, an efficient dual-arm manipulation system is designed in this paper, which combines quasi-static (including pick & place and pick & drag) and dynamic fling actions to flexibly manipulate fabrics into unfolded and smooth configurations. Subsequently, once it is confirmed that the fabric is sufficiently unfolded and all fabric keypoints are detected, the keypoint-based heuristic folding algorithm is employed for the fabric-folding process. To address the scarcity of publicly available keypoint detection datasets for real fabric, we gathered images of various fabric configurations and types in real scenes to create a comprehensive keypoint dataset for fabric folding. This dataset aims to enhance the success rate of keypoint detection. Moreover, we evaluate the effectiveness of our proposed system in real-world settings, where it consistently and reliably unfolds and folds various types of fabrics, including challenging situations such as long-sleeved T-shirts with most parts of sleeves tucked inside the garment. Specifically, our method achieves a coverage rate of 0.822 and a success rate of 0.88 for long-sleeved T-shirts folding. Supplemental materials and dataset are available on our project webpage at https://sites.google.com/view/fabricfolding.

Experimental Evaluation of Manipulator Teleoperation System Based on Trajectory Planning for Obstacle Removal Task in Nuclear Plant Decommissioning

In the teleoperation system of a dual-arm manipulator developed to retrieve fuel debris and reactor interior structures at the Fukushima Daiichi Nuclear Power Plant, we used software in which an obstacle avoidance function (trajectory planning) was implemented to conduct a mock test simulating obstacle removal operations in narrow spaces. The test results confirmed the validity of the obstacle avoidance function, the executability of a series of necessary tasks, and the improved operability. In addition, issues were identified using the test data.

Modelling and identification methods for simulation of cable-suspended dual-arm robotic systems

This paper proposes rigid-body modelling and identification procedures for long-reach dual-arm manipulators in a cable-suspended pendulum configuration. The proposed model relies on a virtually constrained open kinematic chain and lends itself to be simulated through the most commonly used robotic simulators without explicitly account for the cables constraints and flexibility. Moreover, a dynamic parameters identification procedure is devised to improve the simulation model fidelity and reduce the sim-to-real gap for controllers deployment. We show the capability of our model to handle different cable configurations and suspension mechanisms by customising it for two representative cable-suspended dual-arm manipulation systems: the LiCAS arms suspended by a drone and the CRANEbot system, featuring two Pilz arms suspended by a crane. The identified dynamic models are validated by comparing their evolution with data acquired from the real systems showing a high (between 91.3% to 99.4%) correlation of the response signals. In a comparison performed with baseline pendulum models, our model increases the simulation accuracy from 64.4% to 85.9%. The simulation environment and the related controllers are released as open-source code.

Compliance motion control of the hydraulic dual-arm manipulator with adaptive mass estimation of unknown object

Compliance motion control of the hydraulic dual-arm manipulator with adaptive mass estimation of unknown object

Reconfigurable Manufacturing Systems through Intelligent Workpiece Handling and Artificial Intelligence

In recent years, businesses have strived to align their manufacturing goods with the unique needs and preferences of their customers. However, the mass customization paradigm introduces challenges mostly associated to the manipulation of small product batches and knowledge management. This paper proposes a framework for intelligent reconfigurable manufacturing systems, that leverages robotic automation supported by a collaborative dual-arm manipulator, cognitive mechatronic devices, and machine-learning algorithms. In more detail, pioneer end-effectors and machinery provide the capacity of grasping and handling a variety of products without time-consuming hardware modifications. The backbone of this manipulation scheme is a set of machine learning tools for: a) the planning and synchronization of handling agents, b) identification and localization of workpieces, c) the generation of optimal packaging configurations based on customer orders and identified workpieces, and d) the allocation of the respective handling operations to robots, operators, and machinery. The proposed framework is evaluated through its implementation on a use case deriving from the metal industry. The particularities of this use case, as expressed in product variety, workpiece complexity, customer orders and intralogistics, provide a fertile ground for the validation of the discussed technologies.

Physical Human–Robot Interaction of Dual-Arm Manipulators With Human Intention Estimation and Two-Level Admittance Adaptation

Physical Human–Robot Interaction of Dual-Arm Manipulators With Human Intention Estimation and Two-Level Admittance Adaptation

Predictive Multi-Agent-Based Planning and Landing Controller for Reactive Dual-Arm Manipulation

Predictive Multi-Agent-Based Planning and Landing Controller for Reactive Dual-Arm Manipulation

Enabling Versatility and Dexterity of the Dual-Arm Manipulators: A General Framework Toward Universal Cooperative Manipulation

Enabling Versatility and Dexterity of the Dual-Arm Manipulators: A General Framework Toward Universal Cooperative Manipulation

A Lyapunov-based model predictive control strategy with a disturbances compensation mechanism for dual-arm manipulators

This paper proposes a Lyapunov-based nonlinear model predictive control (LMPC) - based on adaptive Lyapunov to solve existing problems in nonlinear dual-arm systems such as system constraints and unknown external disturbances. In practice, the constraints tend to adversely affect the system’s performance and stability. The nonlinear model predictive control NMPC is considered a promising candidate for handling system constraints while enhancing the robustness of the system. However, the rigour of the modeling procedure has a significant influence on the execution of the NMPC, system convergence cannot be assured in the face of modeling uncertainty. To solve this problem, the proposed controller takes into account external disturbances and unidentified parameters by using an adaptive mechanism constructed via the Radial Basis Function Neural Network (RBFNN). Furthermore, the dominant problem of the NMPC algorithm is the system stability which is considered by the Lyapunov theory backbones by a nonlinear Sliding Mode Control (SMC). The numerical simulations are carried out based on a pseudo-physical model to show the efficiency of the proposed control method.

Neural network-based variable stiffness impedance control for internal/external forces tracking of dual-arm manipulators under uncertainties

The desired interaction between manipulators, objects, and environments has resulted in the internal/external force control for dual-arm manipulators being in increasing demand. Consequently, this study focused on the internal/external force tracking for dual-arm manipulator systems under external disturbances, geometries, and stiffness uncertainties which continuously lead to unsatisfactory internal force tracking. The proposed scheme is based on a two-level adaptive impedance control scheme, where the stiffness coefficient is adjusted to adapt to uncalibrated objects. An object-level hybrid impedance controller was used to regulate the external disturbance to produce a compliant response. A manipulator-level neural network-based variable stiffness impedance controller (NNVSIC) was proposed to regulate the internal force under various uncertainties. Additionally, an adaptive wavelet neural network was designed to compensate for the geometric estimation errors of the object. The variable stiffness coefficient could automatically adapt to an unknown object during the cooperation process. One advantage of the proposed method is that no prior knowledge was required. The same controller parameters could be adapted to various objects. The asymptotic stability of the proposed NNVSIC was proven via Lyapunov stability analysis. A series of experiments were conducted using two self-developed nine-degrees-of-freedom redundant manipulators. Furthermore, hard and soft objects of various geometries and stiffnesses were used to verify the effectiveness of the algorithm. The experimental results demonstrated the efficiency and superiority of the proposed controller through performance comparison with various algorithms.

Cooperative Dynamic Motion Planning for Dual Manipulator Arms Based on RRT*Smart-AD Algorithm.

Intelligent manufacturing requires robots to adapt to increasingly complex tasks, and dual-arm cooperative operation can provide a more flexible and effective solution. Motion planning serves as a crucial foundation for dual-arm cooperative operation. The rapidly exploring random tree (RRT) algorithm based on random sampling has been widely used in high-dimensional manipulator path planning due to its probability completeness, handling of high-dimensional problems, scalability, and faster exploration speed compared with other planning methods. As a variant of RRT, the RRT*Smart algorithm introduces asymptotic optimality, improved sampling techniques, and better path optimization. However, existing research does not adequately address the cooperative motion planning requirements for dual manipulator arms in terms of sampling methods, path optimization, and dynamic adaptability. It also cannot handle dual-manipulator collaborative motion planning in dynamic scenarios. Therefore, in this paper, a novel motion planner named RRT*Smart-AD is proposed to ensure that the dual-arm robot satisfies obstacle avoidance constraints and dynamic characteristics in dynamic environments. This planner is capable of generating smooth motion trajectories that comply with differential constraints and physical collision constraints for a dual-arm robot. The proposed method includes several key components. First, a dynamic A* cost function sampling method, combined with an intelligent beacon sampling method, is introduced for sampling. A path-pruning strategy is employed to improve the computational efficiency. Strategies for dynamic region path repair and regrowth are also proposed to enhance adaptability in dynamic scenarios. Additionally, practical constraints such as maximum velocity, maximum acceleration, and collision constraints in robotic arm applications are analyzed. Particle swarm optimization (PSO) is utilized to optimize the motion trajectories by optimizing the parameters of quintic non-uniform rational B-splines (NURBSs). Static and dynamic simulation experiments verified that the RRT*Smart-AD algorithm for cooperative dynamic path planning of dual robotic arms outperformed biased RRT* and RRT*Smart. This method not only holds significant practical engineering significance for obstacle avoidance in dual-arm manipulators in intelligent factories but also provides a theoretical reference value for the path planning of other types of robots.

A Predefined Fixed-Time Convergence ZNN and Its Applications to Time-Varying Quadratic Programming Solving and Dual-Arm Manipulator Cooperative Trajectory Tracking

The zeroing neural network (ZNN) model, a powerful approach for addressing time-varying problems, has been extensively applied in the calculation and optimization fields. In this paper, a new pattern activation function, the power piecewise activation function (PPAF), is proposed to establish a predefined fixed-time convergent zeroing neural network (PFTZNN) for finding solutions to the time-varying quadratic programming problem. In comparison with the traditional activation functions, multi-segmentation is a remarkable feature of the PPAF; consequently, the advantage of PPAF is that its parameters can be flexibly adjusted according to actual needs. Specifically, because of the multi-segment characteristics of the PPAF, the convergence speed of the PPAF-activated PFTZNN model can be flexibly adjusted based on distinct requirements. The fixed-time convergence property of the PPAF-activated PFTZNN model is validated by detailed mathematical theoretical analysis, and its upper bound convergence time is directly calculated. Then, the comparative simulation results of the PPAF-activated PFTZNN model with other existing ZNN models for time-varying quadratic programming are provided for the further verification of its superior convergence speed and robustness. In addition, the proposed PFTZNN model is applied for dual-arm manipulator cooperative trajectory tracking, and its practical application ability is demonstrated by united simulation experiments of Matlab and Robot studio. Finally, the PFTZNN model is also applied to control a real dual-arm manipulator to complete the trajectory tracking task, which further validates its superior performance together and widespread applicability.

A systematic review on cooperative dual-arm manipulators: modeling, planning, control, and vision strategies

A systematic review on cooperative dual-arm manipulators: modeling, planning, control, and vision strategies

A 94.1 g scissors-type dual-arm cooperative manipulator for plant sampling by an ornithopter using a vision detection system

AbstractThe sampling and monitoring of nature have become an important subject due to the rapid loss of green areas. This work proposes a possible solution for a sampling method of the leaves using an ornithopter robot equipped with an onboard 94.1 g dual-arm cooperative manipulator. One hand of the robot is a scissors-type arm and the other one is a gripper to perform the collection, approximately similar to an operation by human fingers. In the move toward autonomy, a stereo camera has been added to the ornithopter to provide visual feedback for the stem, which reports the position of the cutting and grasping. The position of the stem is detected by a stereo vision processing system and the inverse kinematics of the dual-arm commands both gripper and scissors to the right position. Those trajectories are smooth and avoid any damage to the actuators. The real-time execution of the vision algorithm takes place in the lightweight main processor of the ornithopter which sends the estimated stem localization to a microcontroller board that controls the arms. The experimental results both indoors and outdoors confirmed the feasibility of this sampling method. The operation of the dual-arm manipulator is done after the perching of the system on a stem. The topic of perching has been presented in previous works and here we focus on the sampling procedure and vision/manipulator design. The flight experimentation also approves the weight of the dual-arm system for installation on the flapping-wing flying robot.

Adaptive Synchronization Sliding Mode Control for an Uncertain Dual-Arm Robot with Unknown Control Direction

In this paper, an adaptive synchronization sliding mode control is proposed for a dual-arm robot against parameter variations, external disturbance, and unknown control directions. The proposed control is designed by using cross-coupling error and sliding mode control to guarantee the position synchronization of the dual-arm manipulator. The control objective of the proposed control is to synchronize the movement of both arms beside the trajectory tracking issue. In order to manage the lumped uncertainties caused by the parameter variations, external disturbance, and unknown control directions, an extended state observer is used in the proposed control. It enhances the stability of the controlled system against uncertainties. Additionally, a Nussbaum gain function is integrated into the control algorithm to deal with the issue of unknown control direction. Lyapunov stability theory is used to demonstrate the stability of the controlled system. Finally, some simulations are implemented in MATLAB Simulink with a dual 3-DOF manipulator system. The results of the proposed control are compared to other controllers to verify its effectiveness.