3D Manipulation Research Articles

Autonomous Wall-building and Firefighting: Team NimbRo’s UGV Solution for MBZIRC 2020

Autonomous robotic systems for various applications including transport, mobile manipulation, and disaster response are becoming more and more complex. Evaluating and analyzing such systems is challenging. Robotic competitions are designed to benchmark complete robotic systems on complex state-of-the-art tasks. Participants compete in defined scenarios under equal conditions. We present our UGV solution developed for the Mohamed Bin Zayed International Robotics Challenge 2020. Our hardware and software components to address the challenge tasks of wall-building and firefighting are integrated into a fully autonomous system. The robot consists of a wheeled, omnidirectional base, a 6 DoF manipulator arm equipped with a magnetic gripper, a highly efficient storage system to transport box-shaped objects, and a water-spraying system to extinguish fires. The robot perceives its environment using 3D LiDAR, as well as RGB and thermal camera-based perception modules and is capable of picking box-shaped objects and constructing a pre-defined wall structure. Its sensor modules also facilitate detecting and localizing heat sources to extinguish potential fires. A high-level planner coordinates and applies the robot’s skills to complete the Challenge tasks. We analyze and discuss our successful participation during the MBZIRC 2020 finals, present further experiments, and provide insights to our lessons learned.

Magnetic Soft Millirobots 3D Printed by Circulating Vat Photopolymerization to Manipulate Droplets Containing Hazardous Agents for In Vitro Diagnostics.

3D printing via vat photopolymerization (VP) is a highly promising approach for fabricating magnetic soft millirobots (MSMRs) with accurate miniature 3D structures; however, magnetic filler materials added to resin either strongly interfere with the photon energy source or sediment too fast, resulting in the nonuniformity of the filler distribution or failed prints, which limits the application of VP. To this end, a circulating vat photopolymerization (CVP) platform that can print MSMRs with high uniformity, high particle loading, and strong magnetic response is presented. After extensive characterization of materials and 3D printed parts, it is found that SrFe12 O19 is an ideal magnetic filler for CVP and can be printed with 30% particle loading and high uniformity. By using CVP, various tethered and untethered MSMRs are 3D printed monolithically and demonstrate the capability of reversible 3D-to-3D transformation and liquid droplet manipulation in 3D, an important task for in vitro diagnostics that are not shown with conventional MSMRs. A fully automated liquid droplet handling platform that manipulates droplets with MSMR is presented for detecting carbapenem antibiotic resistance in hazardous biosamples as a proof of concept, and the results agree with the benchmark.

Path planning of manipulator based on improved RRT algorithm

Aiming at the problem that the 6-DOF manipulator can quickly avoid obstacles in high-dimensional space, the method of path planning based on the improved rapid expansion random tree (RRT) algorithm is proposed. The improved RRT algorithm combines the advantages of the target bias strategy and proposes a method for setting the intermediate bias point. In random sampling, the sampling point is biased to the intermediate bias point,with a certain probability to reduce the blindness of sampling, and use at the same time the path shortening method reduces path nodes and turning points, and optimizes the motion path of the robotic arm. By comparing the simulation results with the RRT and P-RRT algorithms, the effectiveness of the improved RRT algorithm in the calculation time and the number of path nodes is proved.

Metrics for 3D Object Pointing and Manipulation in Virtual Reality: The Introduction and Validation of a Novel Approach in Measuring Human Performance

Assessing the performance of human movements during teleoperation and virtual reality (VR) is a challenging problem, particularly in 3D space, due to complex settings. Despite the presence of a multitude of measurements, a compelling standardized 3D metric is yet missing, aggravating interstudy comparability. Hence, evaluating human performance in virtual environments (VEs) is a long-standing research goal, and a performance metric that combines two or more measurements under one formulation remains largely unexplored, particularly in higher dimensions. The absence of such a metric is primarily attributed to the discrepancies between pointing and manipulation, the complex spatial variables in 3D, and the combination of translational and rotational movements.

A Novel Trajectory Tracking Control with Modified Supertwisting Sliding Mode for Human-Robot Cooperation Manipulator in Assembly Line

In this paper, the trajectory tracking problem of industrial manipulator with fixed periodic external force on human-robot cooperative assembly line is studied. An advanced fuzzy adaptive supertwisting sliding mode control (FASTSMC) algorithm is proposed to realize the rapid convergence and continuous control of nonlinear control system. In order to enhance the robustness of the system, an arctangent terminal sliding mode surface is designed to deal with the concentrated uncertain disturbance. The supertwisting method is used to overcome the chattering problem in the control law. The fuzzy adaptive technique is used to compensate the centralized disturbance with unknown upper bound. A stiffness identification model is designed to estimate the deviation of the manipulator under external force. Finally, the feasibility of the proposed control scheme is verified by a simulation example of a 4-DOF manipulator.

Development of the Method of Formation the Reference Signals for Electric Actuators of Manipulators with Redundant Degrees of Freedom

The solution of task of maintaining the dynamic accuracy of control of working tools of multilink manipulators (MM) when they move along arbitrary spatial trajectories is presented in this paper. In this case, constructive restrictions in all degrees of freedom (DoF) of manipulators and special cases of location of their links are taken into account. In the first part of the paper the features of the new solution to the inverse kinematics problem are considered and special (singular) positions for 6-DoF manipulators with PUMA kinematic schemes are described. These singular positions are characterized by ambiguity in solving inverse kinematics problem of these MM. The presented solution of the inverse kinematics problem takes into account various combinations of generalized coordinates of the MM, which ensure the movement of their working tools to the specified positions with the required spatial orientation. In the second part of the paper a method is considered, which, due to use of an additional (redundant) DoF of the MM, which ensures its movement near work objects, does not allow the manipulator to enter unpredictable in advance singular positions, excludes approaching of working tools to the boundaries of MM working area, as well as some of its DoFs — to restrictions leading to a sharp decrease in the accuracy of the manipulator in the process of performing any technological operations with previously unknown trajectories of movement. Based on this method, a system has been synthesized that makes it possible to automatically generate reference signals for all DoFs of MM, taking into account the current reference positions and orientations of the working tools specified in the absolute coordinate system. As a result, emergency situations are not created and the high quality of the prescribed operations and work is maintained. The results of the performed mathematical simulation confirmed the high efficiency of using the proposed approach to the contour control of the MM.

Development of an underwater robotic arm using multibody dynamics approach

Underwater robotic arms are important devices that enables workers to carry out tasks remotely from a safe distance reducing or eliminating the risks that are involved with the task. The primary objective of the robotic manipulator is to perform maintenance and cleaning activities of the hull of a ship. However, the control of these devices underwater is quite complicated due to the numerous factors that make these systems unstable and non-linear. The aim of this study is to develop a multibody dynamic robotic manipulator model, integrated with a control strategy to optimize and obtain stable kinematics solutions. The hydrodynamic forces are integrated to the manipulator model considering buoyancy forces and surface drag forces. A basic algorithm is used to generate the joint angles using 7 geometrical parameters. The control of the manipulator was done to simply follow any path that represents the given coordinates. The P, I and D parameters are tuned individually to optimize the kinematic solution of the manipulator. 3-DOF articulated manipulator is the commonly used manipulator configuration. However, a 6-DOF manipulator configuration was selected in this study to allow for change in orientation using wrist motions.

Mathematical Model Design for a 7-DOF robot and 6-DOF comparison simulation

Currently, serial manipulators are the boom in the industry. Most companies are looking for solutions that improve production processes and that are human collaborative. They seek to simulate the skills that humans perform. Nowadays there is a constant question of comparison between a redundant robot of 7-DOF and one that meets the necessary joints to perform complex jobs useful in the industry as is the 6-DOF. In this paper we design a mathematical model that describes the forwards kinematics of the 7-DOF manipulator using the famous D-H convention and determine if the kinematic chain of the manipulators interferes in the performance of their work and thus reach a comparison with its predecessor of 6-DOF in the industry by means of a simulation in CoppeliaSim. As the main results the 6-DOF is more useful for repetitive production assignments and the 7-DOF for human collaboration.

Continuous 3D particles manipulation based on cooling thermal convection

Techniques that can dexterously manipulate particles/cells are significant for biological detection, analytical chemistry, and material synthesis applications. Although there are many methods to realize the 3D manipulation in a static condition, there is a lack of sufficient 3D manipulating strategies in a continuous flow. We develop an integrated microsystem with a cooling system based on thermal convection to realize 3D manipulation in continuous flow. Theoretical modeling and experiments demonstrate that with increasing the temperature of a microheater, the moving direction of particles transfers from the y-axis to the z-axis. Based on the principle, focusing flow and single particles are manipulated in the y-axis and z-axis. Besides, five outlets in different z-axis are fabricated to demonstrate the application of particle sorting. This novel strategy offers a new direction for particle manipulation vertically and benefits to reveal the heat transfer in micro-scale, which can pave a path for generating the next integrated microfluidic system for particles/cells sorting.

Fault tolerant motion planning and control of space manipulators with free-swinging joint failure based on controllable degree

Space manipulators are usually used to assist or replace the astronaut to execute on-orbit tasks. However, due to the burdensome tasks and harsh space environment, free-swinging joint failure is prone to happen, causing the task to be unable to continue. Once the joint failure occurs, the base and free-swinging joint belong to underactuated units, so their trajectories have to be controlled by the active joint. To execute on-orbit tasks successfully after the joint failure occurs, a fault tolerant motion planning and control method (FTMPC) of the space manipulator based on controllable degree is proposed. First, based on the dynamics model of space manipulators, the dynamics coupling relationship between the active joint and the underactuated unit is established. Second, according to the inertia matrix, Coriolis and centrifugal terms of the dynamics model, a controllable degree index of the active joint over the underactuated unit is constructed. Third, according to the specific task requirement and controlled object, the FTMPC based on the controllable degree is designed. The FTMPC is applied according to the number of active joints and underactuated units and whether there is a dynamic singularity during the task execution. The simulation is carried out with a 7-DOF space manipulator, verifying the correctness and effectiveness of the FTMPC. The method has two advantages: 1) Coriolis and centrifugal terms are both considered for the first time in evaluating the control ability of active joints, making the evaluation result more in line with the motion characteristics of space manipulators; 2) A FTMPC based on the controllable degree is proposed, completing the task without dynamic singularity or huge torque on the active joint.

Development of the Method of Formation the Reference Signals for Electric Actuators of Manipulators with Redundant Degrees of Freedom. Part I

The solution of task of maintaining the dynamic accuracy of control of working tools of multilink manipulators (MM) when they move along arbitrary spatial trajectories is presented in this paper. In this case, constructive restrictions in all degrees of freedom (DoF) of manipulators and special cases of location of their links are taken into account. These special (singular) positions are characterized by ambiguity in solving inverse kinematics problem of these MM. Maintaining of control accuracy is proposed to be ensured by excluding the enter of all DoFs of manipulators to the restrictions, as well as excluding the enter of their working tools to the boundaries of the working area. This is accomplished by using a redundant DoF when approaching these undesirable positions. In the first part of the article, the features of the new solution of the inverse kinematics problem are considered and singular positions for 6-DoF manipulators with PUMA kinematic schemes are described. The presented solution of the inverse kinematics problem takes into account various combinations of genera- lized coordinates of the MM, which ensure the movement of their working tools to the specified positions with the required spatial orientation. This solution has low computational complexity compared to other known methods. This allows the use of low-power microprocessor computers to control the MM. The results of the performed mathematical simulation confirmed the high efficiency of using the proposed approach to the contour control of the MM.

A manipulator size optimization method based on dexterous workspace volume

Background: Manipulators for robots are required to have high manipulability for adaptability in different tasks. However, general methods for designing manipulators with high manipulability are deficient. Here, aiming at improving the manipulability of the six degrees-of-freedom (DOF) manipulator, a method for optimizing structure size parameters based on the dexterous workspace volume is proposed. Methods: Firstly, the kinematic analysis of the manipulator is performed. Then, the manipulability of the single working point of the manipulator is judged based on reachability in different postures. The workspace of the manipulator is discretized to obtain the volume of the overall dexterous workspace. By taking the maximum volume of the dexterous workspace as the optimization goal, Genetic Algorithm (GA) is used to optimize the structure size parameters to achieve optimal manipulability. AUBO 6-DOF manipulator is defined and analyzed as an example. Results: The optimization results indicate that the dexterous workspace of the manipulator expands in volume and its manipulability is improved. The validity of the proposed optimization method is verified by comparing the distribution of dexterous workspace of the manipulator. Conclusions: This article proposes an optimization method for the structure size parameters of a 6-DOF manipulator, which can be implemented to improving the manipulability of the manipulator.

Generating a 3D optical needle array with prescribed characteristics

Unlike the general optical needle along the optical axis, we propose a method to generate a three-dimensional (3D) array formed by optical needles with prescribed length and polarization direction. Moreover, the geometric model of the created array can be specified. With the aid of antenna array pattern synthesis theory and time reversal technology, a virtual uniform line source (ULS) antenna array arranged regularly near the confocal region of two objectives of high numerical apertures is employed to obtain the required illumination in the pupil plane for creating the desired focal fields. Numerical results demonstrate that there is a one-to-one correspondence between the focal field and the elements of the virtual ULS antenna array The length and polarization direction of the optical needles depend on the length and spatial direction of the virtual ULS antenna. The peculiarities of the focal field array, such as the polarization, length, number, spatial position and array structure, can be customized according to application requirements. The created optical needle array can be used for such applications as 3D synchronous particle acceleration and manipulation, and 3D parallel fabrication.

Sensor-based skeleton modeling and MVEEs approach of the redundant manipulator for reaction motion

With the wide application of redundant manipulators, sharing a working space with humans and dealing with uncertainty seems an inevitable problem, especially in the dynamic and unstructured domain. How to deal with obstacle avoidance is of particular importance that robots and humans/environments are safe interactions to fulfill the complex cooperating tasks. This paper aimed at solving the problem of multiple points avoidance for the reaction motion based on the skeleton algorithm in unstructured and dynamic environments. A method named “sensor-based skeleton modeling and MVEEs approach of the redundant manipulator for the reaction motion” is proposed. The extraction of skeleton information from image is obtained to calculate the distances of the multiple control points and establish the repulsion in this method. Afterward, the force Jacobian related to the priority weighting factors is calculated and then a reaction force with damping term is established, which is corresponding nominal torque commands. For the redundant manipulator, the joint angles are obtained through torque iteration instead of inverse kinematics to reduce calculation cost. Finally, the method was tested by a 7-DOF manipulator in the ROS framework. The obtained results indicate that the method in this method can realize dynamic obstacle avoidance and time cost reduction.

Calibration of a Manipulator With a Regularized Parameter Identification Method

As an inverse problem, the parameter identification of manipulators is essential to in-situ calibration. Since the ill-posed inverse kinematic model is sensitive to the measurement value, even tiny errors will make the geometric model of the manipulator wrongly identified. To overcome this problem, a Regularized Parameter Identification Method (RPIM) is proposed to calibrate the geometric model of the manipulator. The inverse kinematic model of a 6 DOF manipulator is modified by a Tikhonov regularization to overcome its ill-posed problem. The regularization parameter is optimized by the improved L curve method to adjust the initial model to a well-posed one that approximates the real situation. A calibration system is designed to evaluate the effectiveness of the suggested method. The position of the selected targets is tested by using a laser tracker. The experimental result shows that the absolute position errors of the manipulator are 2.533mm without calibration, 0.472mm by RPIM, 1.445mm by Least Square Method (LSM), 1.353mm by Gradient Descent (GD), and 0.956mm by Gauss-Newton (GN). It shows that the absolute position error of RPIM is reduced by 81.331% after calibration, which is superior to other methods.

Experiment and Design of Prescribed-Time Safety Filter for a 7-DOF Robot Manipulator Using CBF-QP

We formulate a prescribed-time safety filter for the case of a redundant manipulator performing a fixed-duration task. This quadratic programming based formulation yields a filter that is capable of avoiding multiple obstacles in a minimally invasive manner, while simultaneously allowing the nominal controller to converge to positions located on the boundary of the safe set by the end of the fixed-duration task. In order to demonstrate the efficacy of the proposed method, we performed a series of simulations and experiments on Baxter, a 7-DOF collaborative robot manipulator. Furthermore, when compared to the exponential safety filter, which is the state-of-the-art in current literature, our proposed method yielded consistently lower joint jerks. Thus, for tasks with a fixed duration, the proposed PTSf offers performance benefits over the exponential filters currently present in literature.

Generic Development of Bin Pick-and-Place System Based on Robot Operating System

Bin pick-and-place is an important topic in factory automation and warehouse automation. In this paper, a bin pick-and-place system based on robot operating system (ROS) is implemented to make a six-degree-of-freedom (6-DOF) robot manipulator to complete multiple pick-and-place tasks. The proposed system uses ROS to integrate an object perception module and a pick-and-place module, where the former uses an RGB-D camera to capture images inside the bin, and the latter controls a 6-DOF robot manipulator and two self-made vacuum tools. To estimate the pose of the target object, a YOLOv4 object detector is implemented, and an object sorting method is proposed to find the target object in the image. Then, a pose estimation method based on computer aided design (CAD) is proposed to estimate the pose of target object. To perform the object pick-and-place operations, a coordinate transformation node is designed to transfer the pose of the target object into the workspace. Then, a link distance-based bin collision avoidance method is proposed to avoid collisions. Finally, the angle of the 1-DOF vacuum tool and the picking and placement poses of the robot manipulator are obtained from the result of the bin collision avoidance and the pose of the target object. In this study, a total of ten ROS nodes are designed, and the solutions that make each function easier to implement and reproduce are proposed. In the experiments, we set up four experiments with two task types and two object types to verify the effectiveness of the implemented bin pick-and-place system.

Study on modeling and dynamic performance of a planar flexible parallel manipulator based on finite element method.

The application of a high-speed parallel manipulator necessitates the adoption of a lightweight design to reduce dead weight. However, this increases the elastic deformation of certain components, affecting the dynamic performance of the system. This study examined a 2-DOF planar flexible parallel manipulator. A dynamic model of the parallel manipulator composed of fully flexible links was established using a floating reference coordinate system and a combination of the finite element and augmented Lagrange multiplier methods. A dynamic analysis of the simplified model under three driving torque modes showed that the axial deformation was less than the transverse deformation by three orders of magnitude. Further, the kinematic and dynamic performance of the redundant drive was significantly better than that of the non-redundant drive, and the vibration was well suppressed in the redundant drive mode. In addition, the comprehensive performance of driving Mode 2 was better than that of the other two modes. Finally, the validity of the dynamic model was verified by modeling via Adams. The modular modeling method is conducive to the extension to other models and programming. Furthermore, the dynamic model of the established fully flexible link system can aid in optimizing the lightweight design and dynamic performance of the parallel manipulator.

Manipulator Collision Avoidance System Based on a 3D Potential Field With ISO 15066

Recently, various intelligent technological innovations are being applied to smart factories. As manipulators are widely used in smart factories, the manipulator and the workspace of humans overlap, and interest in cooperative robots and human safety has increased. In relation to this, a collision-avoidance control algorithm applicable in three dimensions (3D) and that also meets existing safety standards for humans and robots is required. In this paper, we propose a 3D potential field-based manipulator collision avoidance algorithm that meets the requirements of the ISO 15066 standard. This algorithm applies Speed and Separation Monitoring(SSM) according to the distance between the manipulator and the obstacle and controls the speed of the manipulator slowly as the risk is higher. This allows us to overcome the limitations that existing studies have not been conducted on 3D potential field-based obstacle avoidance and that it is difficult to apply to the field without considering ISO 15066. The proposed system was verified through simulation and experiments, and through comparison with the existing algorithm, we verified that SSM was well applied to the proposed system.

A Novel QP-Based Kinematic Redundancy Resolution Method With Joint Constraints Satisfaction

Kinematically redundant manipulators are advantageous for their increased dexterity and ability to fulfill some secondary requirements along with their primary task to follow the prescribed trajectory. The redundancy results in a non-trivial inverse kinematics problem (IK). Standard methods of redundancy resolution are based on the pseudoinverse of the Jacobian matrix of the manipulator. The excessive degrees of freedom are utilized to perform secondary tasks that are projected onto the null space of the Jacobian matrix. However, the joint constraints satisfaction cannot be easily ensured in this way—even though methods that account for the joint position limits are known, the constraints for joint velocities and especially accelerations are not straightforward to include. In this paper, a novel redundancy resolution method based on a less common quadratic programming (QP) approach is described. The proposed velocity-level IK method allows fulfilment of the joint constraints at the position, velocity, and acceleration levels. In the derived formulas, accelerations instead of the usual velocities are used—the discretized joint state equations allow the use of joint accelerations as decision variables in the QP problem. The developed algorithm is investigated in a series of numerical tests in which the kinematics of the KUKA LWR4+ redundant 7-DOF manipulator is exploited. The newly elaborated QP-based IK method is firstly compared with the classic pseudoinverse-based approach and then tested for its ability to keep the joint accelerations within the prescribed bounds. The prospects of the proposed approach are discussed in the concluding section.