3D Robot Research Articles

Electrical Heaters for Anti/De-Icing of Polymer Structures

The problem of icing for surfaces of engineering structures requires attention more and more every year. Active industrialization in permafrost zones is currently underway; marine transport in Arctic areas targets new goals; the requirements for aerodynamically critical surfaces of wind generators and aerospace products, serving at low temperatures, are increasing; and fiber-reinforced polymer composites find wide applicability in these structural applications demanding the problem of anti/de-icing to be addressed. The traditional manufacturing approaches are superimposed with the new technologies, such as 3D printers and robotics for laying heat wires or cheap and high-performance Thermal Sprayed methods for metallic cover manufacturing. Another next step in developing heaters for polymer structures is nano and micro additives to create electrically conductive heating networks within. In our study, we review and comparatively analyze the modern technologies of structure heating, based on resistive heating composites.

HTC-Grasp: A Hybrid Transformer-CNN Architecture for Robotic Grasp Detection

Accurately detecting suitable grasp areas for unknown objects through visual information remains a challenging task. Drawing inspiration from the success of the Vision Transformer in vision detection, the hybrid Transformer-CNN architecture for robotic grasp detection, known as HTC-Grasp, is developed to improve the accuracy of grasping unknown objects. The architecture employs an external attention-based hierarchical Transformer as an encoder to effectively capture global context and correlation features across the entire dataset. Furthermore, a channel-wise attention-based CNN decoder is presented to adaptively adjust the weight of the channels in the approach, resulting in more efficient feature aggregation. The proposed method is validated on the Cornell and the Jacquard dataset, achieving an image-wise detection accuracy of 98.3% and 95.8% on each dataset, respectively. Additionally, the object-wise detection accuracy of 96.9% and 92.4% on the same datasets are achieved based on this method. A physical experiment is also performed using the Elite 6Dof robot, with a grasping accuracy rate of 93.3%, demonstrating the proposed method’s ability to grasp unknown objects in real scenarios. The results of this study indicate that the proposed method outperforms other state-of-the-art methods.

Silver-Nanowire-Based Elastic Conductors: Preparation Processes and Substrate Adhesion.

The production of flexible electronic systems includes stretchable electrical interconnections and flexible electronic components, promoting the research and development of flexible conductors and stretchable conductive materials with large bending deformation or torsion resistance. Silver nanowires have the advantages of high conductivity, good transparency and flexibility in the development of flexible electronic products. In order to further prepare system-level flexible systems (such as autonomous full-software robots, etc.), it is necessary to focus on the conductivity of the system's composite conductor and the robustness of the system at the physical level. In terms of conductor preparation processes and substrate adhesion strategies, the more commonly used solutions are selected. Four kinds of elastic preparation processes (pretensioned/geometrically topological matrix, conductive fiber, aerogel composite, mixed percolation dopant) and five kinds of processes (coating, embedding, changing surface energy, chemical bond and force, adjusting tension and diffusion) to enhance the adhesion of composite conductors using silver nanowires as current-carrying channel substrates were reviewed. It is recommended to use the preparation process of mixed percolation doping and the adhesion mode of embedding/chemical bonding under non-special conditions. Developments in 3D printing and soft robots are also discussed.

2D/3D Wound Segmentation and Measurement Based on a Robot-Driven Reconstruction System

Chronic wounds, are a worldwide health problem affecting populations and economies as a whole. With the increase in age-related diseases, obesity, and diabetes, the costs of chronic wound healing will further increase. Wound assessment should be fast and accurate in order to reduce possible complications and thus shorten the wound healing process. This paper describes an automatic wound segmentation based on a wound recording system built upon a 7-DoF robot arm with an attached RGB-D camera and high-precision 3D scanner. The developed system represents a novel combination of 2D and 3D segmentation, where the 2D segmentation is based on the MobileNetV2 classifier and the 3D component is based on the active contour model, which works on the 3D mesh to further refine the wound contour. The end output is the 3D model of only the wound surface without the surrounding healthy skin and geometric parameters in the form of perimeter, area, and volume.

Robot 3D spatial motion measurement via vision-based method

Measuring the motion of a robot accurately is an important and integral part of evaluating the dynamic and static performance of the robot. The performance index of a robot, such as kinematic accuracy, bearing capacity, deformation, vibration, stability, and structural mode can all be calculated according to the motion displacement of the robot. Therefore, improving the robot motion measurement method, promoting the measurement accuracy, and enriching the measurement content have received considerable scholarly attention worldwide. In this paper, an approach based on binocular vision was proposed to measure the 3D spatial motion of a robot. In the process of reconstructing robot movement, a mathematical model that can facilitate the solving process and improve the accuracy of results was derived to build 3D coordinates information. A novel coordinate transformation method that is based on the singular value decomposition was drawn up to realize the transformation from camera coordinates to robot coordinates. Several experiments were carried out on the self-built three-degree-of-freedom rectangular coordinate robot platform. The marker was designed specially and glued to the end of robot, and a new train of thought was adopted to extract the marker’s feature point. The vision-based measurement results were compared with the actual coordinate value. Results of experiments demonstrated that the proposed method can successfully reconstruct the 3D spatial motion of a robot more exactly, which can meet the requirements of high-precision motion control, motion performance evaluation, and operation state evaluation.

Hierarchical Fuzzy Fault-Tolerant Controller Synthesis for Finite-Time Multitarget Surrounding of Networked Perturbed Mechanical Systems

This article investigates the hierarchical synthesis of fuzzy fault-tolerant controllers for networked perturbed mechanical systems to solve the problem of finite-time multitarget surrounding, namely, encircling multiple moving targets in a finite time. Given the system with actuator failures, faults, and perturbations (namely, parametric uncertainties and external disturbances), we propose a hierarchical framework for synthesizing a cascade-form controller to stabilize the system state in a finite time, as well as decouple and solve the abovementioned problem. Specifically, the methods of fuzzy logic systems and finite-time adaptive laws are included within the synthesized controller to approximate the unknown perturbations, and meanwhile to compensate the negative effects of the actuator failures and faults. By employing the theories of perturbations and fault-tolerant analysis, we derive the sufficient conditions on control gains for finite-time convergence of the closed-loop system. Finally, we carry out several simulation experiments on a network of six 2-DOF robots and four moving targets to verify the performance of the synthesis method and the obtained controller.

Coarse-to-Fine Hybrid 3D Mapping System With Co-Calibrated Omnidirectional Camera and Non-Repetitive LiDAR

This letter presents a novel 3D mapping robot with an omnidirectional field-of-view (FoV) sensor suite composed of a non-repetitive LiDAR and an omnidirectional camera. Thanks to the non-repetitive scanning nature of the LiDAR, an automatic targetless co-calibration method is proposed to simultaneously calibrate the intrinsic parameters for the omnidirectional camera and the extrinsic parameters for the camera and LiDAR, which is crucial for the required step in bringing color and texture information to the point clouds in surveying and mapping tasks. Comparisons and analyses are made to target-based intrinsic calibration and mutual information (MI)-based extrinsic calibration, respectively. With this co-calibrated sensor suite, the hybrid mapping robot integrates both the odometry-based mapping mode and stationary mapping mode. Meanwhile, we proposed a new workflow to achieve coarse-to-fine mapping, including efficient and coarse mapping in a global environment with odometry-based mapping mode; planning for viewpoints in the region-of-interest (ROI) based on the coarse map (relies on the previous work [1]); navigating to each viewpoint and performing finer and more precise stationary scanning and mapping of the ROI. The fine map is stitched with the global coarse map, which provides a more efficient and precise result than the conventional stationary approaches and the emerging odometry-based approaches, respectively.

Upgrading Gryphon PUMA Robot Arm Using ROS-MoveIt

Industrial and old lab robotic equipment can be upgraded and reused invery sophisticated applications such as robot arms if provided with an efficientmodular program or framework such as ROS (Robotics Operating system). In thispaper, we replace the old control system of the 5-DOF robot arm (completeGryphon Package 35-002-c) with the help of ROS MoveIt Motion PlanningFramework package to control motion planning and generate joints angles, URDF(Unified Robot Description format) used to create the robot model for ROS. Themicrocontroller is used to move the five stepping motors to move the end effectorin the desired pose: position and orientation using ROS and MoveIt capability.Thus, the robot should be upgradeable, and anyone who has ROS basic skills canimprove the arm functionality to use it in any desired application.

3D Printed Prosthetic Robot Arm with Grasping Detection System for Children

Prosthetic robot arms have been an alternative replacement for upper limb-related disability, especially among children as it helps them perform regular activities such as holding tools, eating, and drinking. This research aims to develop a 3D prosthetic robot arm using SolidWorks and 3D printer, using a low-cost and straightforward mechanism for children. This paper proposes a close loop control system with position control for the prosthetic robot arm to achieve an appropriate grasping force focusing on solid-shaped objects using PID control. The PID controller controls the system's response to perform in the most efficient path. Force-sensitive resistors (FSR) are attached to all fingers to measure the grasping force acting on objects with different surfaces, dimensions, and weights. The controller results showed improvement in the overshoot percentage of 0.902%, as overshoot is essential in preventing the grasped object's deformations. The analysis of the experiment shows that the mean grasping force and static coefficient friction of each object are different regardless of the material the object is made of and the object's mass. For example, a cube-shaped object made of wood requires 0.5288 N of grasping force to grasp the object firmly compared to a plastic-made cube that only requires 0.3245 N to hold the cube. On the other hand, the static coefficient friction for the wood cube is 3.1708 and 0.4725 for the plastic cube. Further research can be done by designing the prosthetic robot arm with independent motorized and multi-degree movement of fingers.

Stretchable e-skin and transformer enable high-resolution morphological reconstruction for soft robots

Many robotic tasks require knowledge of the exact 3D robot geometry. However, this remains extremely challenging in soft robotics because of the infinite degrees of freedom of soft bodies deriving from their continuum characteristics. Previous studies have achieved only low proprioceptive geometry resolution (PGR), thus suffering from loss of geometric details (for example, local deformation and surface information) and limited applicability. Here we report an intelligent stretchable capacitive e-skin to endow soft robots with high PGR (3,900) bodily awareness. We demonstrate that the proposed e-skin can finely capture a wide range of complex 3D deformations across the entire soft body through multi-position capacitance measurements. The e-skin signals can be directly translated to high-density point clouds portraying the complete geometry via a deep architecture based on transformer. This high PGR proprioception system providing millimetre-scale, local and global geometry reconstruction (2.322 ± 0.687 mm error on a 20 × 20 × 200 mm soft manipulator) can assist in solving fundamental problems in soft robotics, such as precise closed-loop control and digital twin modelling. Developing proprioception systems for flexible structures such as soft robots is a challenge. Hu et al. report a stretchable e-skin for soft robot proprioception. Combined with deep learning, the e-skin enables high-resolution 3D geometry reconstruction of the soft robot and can be applied in many scenarios, such as human–robot interaction.

Compliant‐based robotic 3D bin packing with unavoidable uncertainties

AbstractRobotic 3D bin packing (R‐3dBPP), aiming to place deformed cases in various sizes in the container without fences, is a comprehensive application that includes perception, planning, execution, and hardware design. Traditional studies assume that the context of the real space must be accurately perceived and represented. However, disjunctions between the planner and reality are unavoidable in R‐3dBPP, especially with low‐cost sensors. As far as the author knows, there is no practical solution. In this paper, the above assumption is discarded and the typical types of uncertainties prevalent to guide the design of the algorithms are formulated. A new online bin‐packing algorithm is proposed, keeping deformed boxes stacked in close contact with each other, so that the whole pallet is kept stable by friction between the boxes and the container. In order to meet the requirement of close contact under the non‐negligible errors from sensors and planners, a compliant‐based motion planning system is introduced. It replaces the precision‐based feedback with a compliant end‐effector and accompanying motion strategies. Last but not least, a complete online bin‐packing robot system is developed and the system's performance under the influence of the uncertainties mentioned above through simulation and physical experiments is evaluated.

Efficient implementation of Gaussian process–based predictive control by quadratic programming

AbstractThe paper addresses the problem of accelerating predictive control of non‐linear system models augmented with Gaussian processes (GP‐MPC). Due to the non‐linear and stochastic prediction model, predictive control of GP‐based models requires to solve a stochastic optimization problem. Different model simplification methods have to be applied to reformulate this problem to a deterministic, non‐linear optimization task that can be handled by a numerical solver. As these problems are still complex, especially with exact moment calculations, real‐time implementation of GP‐MPC is extremely challenging. The existing solutions accelerate the computations at the solver level by linearizing the non‐linear optimization problem and applying sequential convexification. In contrast, this paper proposes a novel GP‐MPC solution approach that without linearization formulates a series of surrogate quadratic programs (QP‐s) to iteratively obtain the solution of the original non‐linear optimization problem. The first step is embedding the non‐linear mean‐variance dynamics of the GP‐MPC prediction model in a linear parameter‐varying (LPV) structure and rewriting the constraints in parameter‐varying form. By fixing the scheduling trajectory at a known variation (based on previously computed or initial state‐input trajectories), optimization of the input sequence for the remaining varying linear model reduces to a linearly constrained quadratic program. After solving the QP, the non‐linear prediction model is simulated for the new control input sequence and new scheduling trajectories are updated. The procedure is iterated until the convergence of the scheduling, that is, the solution of the QP converges to the solution of the original non‐linear optimization problem. By designing a reference tracking controller for a 4DOF robot arm, we illustrate that the convergence is remarkably fast and the approach is computationally advantageous compared to current solutions. The proposed method enables the application of GP‐MPC algorithms even with exact moment matching on fast dynamical systems and requires only a QP solver.

Neural Joint Space Implicit Signed Distance Functions for Reactive Robot Manipulator Control

In this letter, we present an approach for learning a neural implicit signed distance function expressed in joint space coordinates, that efficiently computes distance-to-collisions for arbitrary robotic manipulator configurations. Computing such distances is a long standing problem in robotics as approximate representations of the robot and environment geometry can lead to overly conservative constraints, numerical instabilities and expensive computations – limiting real-time reactive control and task success. Leveraging GPU parallelization and the differentiable nature of the proposed distance function allows for fast calculation of gradients with respect to the neural network inputs, providing a continuous repulsive vector field directly in joint space. We show that the learned high-resolution collision representation can be used to achieve real-time reactive control by i) formulating it as a collision-avoidance constraint for a quadratic programming (QP) inverse kinematics (IK), and ii) introducing it as a collision cost in a sampling-based joint space model predictive controller (MPC). For a reaching benchmark task with a 7DoF robot and dynamic obstacles intentionally obstructing the robot's path we achieve average 250 Hz control frequency with QP-IK and 92 Hz with MPC, showing an accelerated performance of 15% for QP-IK and 40% for MPC over baseline distance computation techniques.

Affordable Robotic Mobile Mapping System Based on Lidar with Additional Rotating Planar Reflector.

This paper describes an affordable robotic mobile 3D mapping system. It is built with Livox Mid-40 lidar with a conic field of view extended by a custom rotating planar reflector. This 3D sensor is compared with the more expensive Velodyne VLP 16 lidar. It is shown that the proposed sensor reaches satisfactory accuracy and range. Furthermore, it is able to preserve the metric accuracy and non-repetitive scanning pattern of the unmodified sensor. Due to preserving the non-repetitive scan pattern, our system is capable of covering the entire field of view of 38.4 × 360 degrees, which is an added value of conducted research. We show the calibration method, mechanical design, and synchronization details that are necessary to replicate our system. This work extends the applicability of solid-state lidars since the field of view can be reshaped with minimal loss of measurement properties. The solution was part of a system that was evaluated during the 3rd European Robotics Hackathon in the Zwentendorf Nuclear Power Plant. The experimental part of the paper demonstrates that our affordable robotic mobile 3D mapping system is capable of providing 3D maps of a nuclear facility that are comparable to the more expensive solution.

Robotic 3D printing of concrete building components for residential buildings in Saudi Arabia

This article presents the development and validation of a design-to-fabrication framework aiming to improve the efficiency of fabricating reinforced concrete building components for housing projects in the Kingdom of Saudi Arabia by incorporating 3D concrete printing technology. In particular, the research presents an algorithmic framework to mass customise a typical Saudi Arabian free standing house by utilising parametric modelling, topology optimization (TO), finite element analysis (FEA), and robotic 3D printing tools and techniques. The framework was validated by the fabrication of optimised reinforced concrete columns and by testing their structural performance under the Saudi Building Code (SBC 304). The findings demonstrate the benefits and drawbacks of the proposed framework and compare it to current Saudi conventional construction approaches. The paper also addresses the need for mass customisation in the construction industry of the Kingdom of Saudi Arabia

Experimental Study of Robotic Polishing Process for Complex Violin Surface

This paper presents a robotic polishing process for complex violin surfaces to increase efficiency and minimize the cost and consumed time caused by using labor and traditional polishing machines. The polishing process is implemented based on modeling a smooth path, controlled contact force embedded with gravity compensation and material removal depth. A cubic Non-Uniform Rational Bases-Spline (NURBS) interpolation curve combined with an S-curve trajectory model is used to generate a smooth polishing path on a complex violin surface to achieve stable motion during the polishing process. An online admittance controller added to the fast gravity compensation algorithm maintains an accurate polishing force for equal removal depth on all polished surface areas. Then, based on Pythagorean theory, the removal depth model is calculated for the violin’s complex surface before and after polishing to estimate the accuracy of the polishing process. Experimental studies were conducted by polishing a wooden surface using the 6DOF robot manipulator to validate this methodology. The experimental results demonstrated that the robot had accurate polishing force based on the online admittance controller with gravity compensation. It also showed a precise proportional uniformity of removal depths at the different normal forces of 10, 15, and 20 N. The final results indicated that the proposed experimental polishing approach is accurate and polishes complex surfaces effectively.

Load-Oriented Nonplanar Additive Manufacturing Method for Optimized Continuous Carbon Fiber Parts.

The process of the additive manufacturing (AM) of carbon-fiber-reinforced polymer (CFRP) parts based on the process of fused deposition modeling (FDM) has seen considerable research in recent years, which amplifies the importance of adapted slicing and pathplanning methods. In particular, load-oriented techniques are of high interest when employing carbon fiber materials, as classical methods, such as tape-laying and laminating, struggle with highly curved and complex geometries and require the costly production of molds. While there have been some promising propositions in this field, most have restricted themselves to a planar slicing approach, which severely limits the ability to place the fibers along stress paths. In this paper, a nonplanar slicing approach is presented that utilizes principal stress directions to construct optimized nonplanar constituting layers on which pathplanning can be carried out. These layers are oriented such that the effect of the weak interlayer adhesion is minimized. Support material is adaptively generated to enable the use of arbitrary part geometry. Furthermore, a continuous pathplanning method and post-processor are applied to yield manufacturing instructions. The approach is verified for its viability of application through experimental investigation on a multi-axis robotic 3D printer. This constitutes an important step in allowing the fabrication of CFRP parts to further utilize the possibilities of additive manufacturing.

Eye-in-Hand Robotic Arm Gripping System Based on Machine Learning and State Delay Optimization.

This research focused on using RGB-D images and modifying an existing machine learning network architecture to generate predictions of the location of successfully grasped objects and to optimize the control system for state delays. A five-finger gripper designed to mimic the human palm was tested to demonstrate that it can perform more delicate missions than many two- or three-finger grippers. Experiments were conducted using the 6-DOF robot arm with the five-finger and two-finger grippers to perform at least 100 actual machine grasps, and compared to the results of other studies. Additionally, we investigated state time delays and proposed a control method for a robot manipulator. Many studies on time-delay systems have been conducted, but most focus on input and output delays. One reason for this emphasis is that input and output delays are the most commonly occurring delays in physical or electronic systems. An additional reason is that state delays increase the complexity of the overall control system. Finally, it was demonstrated that our network can perform as well as a deep network architecture with little training data and omitting steps, such as posture evaluation, and when combined with the hardware advantages of the five-finger gripper, it can produce an automated system with a gripping success rate of over 90%. This paper is an extended study of the conference paper.

Graph Wasserstein Autoencoder-Based Asymptotically Optimal Motion Planning With Kinematic Constraints for Robotic Manipulation

This paper presents a learning based motion planning method for robotic manipulation, aiming to solve the asymptotically-optimal motion planning problem with nonlinear kinematics in a complex environment. The core of the proposed method is based on a novel neural network model, i.e., graph wasserstein autoencoder (GraphWAE) network, which is used to represent the implicit sampling distributions of the configuration space (C-space) for sampling-based planning algorithms. Through learning the implicit distributions, we can guide the planning process to search or extend in the desired region to reduce the collision checks dramatically for fast and high-quality motion planning. The theoretical analysis and proofs are given to demonstrate the probabilistic completeness and asymptotic optimality of the proposed method. Numerical simulations and experiments are conducted to validate the effectiveness of the proposed method through a series of planning problems from 2D, 6D and 12D robot C-spaces in the challenging scenes. Results indicate that the proposed method can achieve better planning performance than the state-of-the-art planning algorithms. Note to Practitioners—The motivation of this work is to develop a fast and high-quality asymptotically optimal motion planning method for practical applications such as autonomous driving, robotic manipulation and others. Due to the time consumption caused by collision detection, current planning algorithms usually take much time to converge to the optimal motion path especially in the complicated environment. In this paper, we present a neural network model based on GraphWAE to learn the biasing sampling distributions as the sample generation source to further reduce or avoid collision checks of sampling-based planning algorithms. The proposed method is general and can be also deployed in other sampling-based planning algorithms for improving planning performance in different robot applications.

Range Extension for Large-Scale Robotic Precision 3-D Measurements in Vibration-Prone Environments

This paper presents a range extension concept for a high-precision robotic inline 3D measurement system. A scanning confocal chromatic sensor (SCCS) is integrated with a magnetically levitated and actuated measurement platform for the acquisition of 3D images with sub-micrometer resolution, enabled by means of active sample-tracking to compensate for relative motion between the SCCS and the sample. Based on an intermediate range extension approach, the lateral scan area is pre-extended to enable a robotic repositioning of the 3D measurement module, ensuring sufficient overlap regions between the individual 3D image frames. Experimental results show that the proposed concept extends the 3D measurement modules’ measurement range by about 80% of the lateral scan area within an acquisition time of about 90 s. Range-extended 3D measurements directly in a vibration-prone environment reveal that 97% of disturbing relative motion between the SCCS and the sample are compensated by the active sample-tracking approach. In this way, lab-like conditions for the SCCS are established directly in an industrial production line and, improving the measurement error by three orders of magnitude down to several tens of nanometers.