Robot Pose Research Articles

Topological map‐based approach for localization and mapping memory optimization

AbstractRobotics in agriculture faces several challenges, such as the unstructured characteristics of the environments, variability of luminosity conditions for perception systems, and vast field extensions. To implement autonomous navigation systems in these conditions, robots should be able to operate during large periods and travel long trajectories. For this reason, it is essential that simultaneous localization and mapping algorithms can perform in large‐scale and long‐term operating conditions. One of the main challenges for these methods is maintaining low memory resources while mapping extensive environments. This work tackles this issue, proposing a localization and mapping approach called VineSLAM that uses a topological mapping architecture to manage the memory resources required by the algorithm. This topological map is a graph‐based structure where each node is agnostic to the type of data stored, enabling the creation of a multilayer mapping procedure. Also, a localization algorithm is implemented, which interacts with the topological map to perform access and search operations. Results show that our approach is aligned with the state‐of‐the‐art regarding localization precision, being able to compute the robot pose in long and challenging trajectories in agriculture. In addition, we prove that the topological approach innovates the state‐of‐the‐art memory management. The proposed algorithm requires less memory than the other benchmarked algorithms, and can maintain a constant memory allocation during the entire operation. This consists of a significant innovation, since our approach opens the possibility for the deployment of complex 3D SLAM algorithms in real‐world applications without scale restrictions.

Capsule robot pose and mechanism state detection in ultrasound using attention-based hierarchical deep learning

Ingestible robotic capsules with locomotion capabilities and on-board sampling mechanism have great potential for non-invasive diagnostic and interventional use in the gastrointestinal tract. Real-time tracking of capsule location and operational state is necessary for clinical application, yet remains a significant challenge. To this end, we propose an approach that can simultaneously determine the mechanism state and in-plane 2D pose of millimeter capsule robots in an anatomically representative environment using ultrasound imaging. Our work proposes an attention-based hierarchical deep learning approach and adapts the success of transfer learning towards solving the multi-task tracking problem with limited dataset. To train the neural networks, we generate a representative dataset of a robotic capsule within ex-vivo porcine stomachs. Experimental results show that the accuracy of capsule state classification is 97%, and the mean estimation errors for orientation and centroid position are 2.0 degrees and 0.24 mm (1.7% of the capsule’s body length) on the hold-out test set. Accurate detection of the capsule while manipulated by an external magnet in a porcine stomach and colon is also demonstrated. The results suggest our proposed method has the potential for advancing the wireless capsule-based technologies by providing accurate detection of capsule robots in clinical scenarios.

Operational Impact Excitation Method for Milling Robot End Frequency Response Function Identification Under Movement State and Pose-Dependent Dynamic Compliance Analysis

Abstract Robot milling has become an important means of machining large structural parts, and the dynamic compliance of the robot end is the key factor affecting machining quality and efficiency. The dynamic characteristics of the milling robot end are different in movement state and static state and have significant pose dependence. In order to effectively evaluate the dynamic compliance of the robot end in the workspace under the movement state, the operational impact excitation method for robot joint relative frequency response function (FRF) identification based on operational modal analysis (OMA) used in machine tool is established, the inertial force generated by the joint acceleration and deceleration movements is used as the excitation force, the robot end relative dynamic compliance index (RERDCI) is proposed to evaluate the dynamic compliance of robot end in different poses, and verified by cutting experiments. Based on RERDCI, the effect of the robot pose on end dynamic compliance is analyzed, and some theoretical guidance for improvement of dynamic performance to resist vibration in milling is given.

Workspace Mapping Method Based on Edge Drifting for the Teleoperation System

The workspace mapping method for master-slave isomeric robots is a key research direction. In order to enhance the safety and stability of the operation of such robots, a position-joint hybrid mapping method based on edge drifting is proposed in this paper. First, teleoperation is divided into two phases, that is, task approach and task operation. During the task approach, the edge drifting algorithm is adopted to largely cover the workspace of a robot. Rate mapping is used to quickly control the movement of the slave robot to the target position. During task operation, a hybrid mapping algorithm integrating the Cartesian space and the joint space highly precisely maps both the position and pose of the slave robot to accurately follow the control command of the master end and achieve precise control. Lastly, the proposed method is validated on the experimental platform for master-slave isomeric teleoperation. The experimental results indicate that this mapping method allows the master-slave isomeric teleoperation system to cover a large workspace precisely with desirable precision in position and pose following.

Invariant Cubature Kalman Filtering-Based Visual-Inertial Odometry for Robot Pose Estimation

To maintain mechanistic stability while tracking the designated walking route, a robot must be cognizant of employed posture. Generally, visual-inertial odometer (VIO) is utilized for robot state estimation; however, the traditional cubature Kalman filter VIO (CKF-VIO) cannot transfer rotational uncertainty and compensate for the system’s processing error. To effectively improve the accuracy and stability of robot rigid body pose estimation, this paper proposes a matrix Lie group representation-based CKF framework which characterizes the uncertainty prompting in robotic motion while eliminating the VIO system internalization errors. The robot state, consisting of inertial measurement unit (IMU) pose, velocity, and 3-D landmarks’ positions, is deemed to be a single element of a high-dimensional Lie group SE2+p <xref ref-type="disp-formula" rid="deqn3" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(3)</xref> , while the accelerometers’ and gyrometers’ biases are appended to the state and estimated as well. The algorithm is validated by simulations with Monte Carlo and experiment. Results show that the CKF-VIO with a high-dimensional Lie group can improve the accuracy and consistency of robot pose estimation.

Robotic Navigation Based on Experiences and Predictive Map Inspired by Spatial Cognition

Humans and animals have environmental cognition and navigation abilities. These abilities are closely related to the spatial cognitive mechanism of brain. Based on this mechanism, we propose a novel robotic navigation framework based on experiences and predictive map inspired by spatial cognition to accurately construct environment experience and quickly plan path. The grid cell and the place cell are modeled to rapidly integrate self-motion cues. The multidimensional grid coding and one-shot learning rule are utilized for activating the place representation of robot pose. Visual cues provide information for relocation. These information are integrated through experiences which express the topology of the environment, enabling the robot to accurately achieve spatial cognition of complex environment. In order to realize the sequential decision making of hippocampus, the predictive map is introduced to quickly plan the experience sequence to target in dynamic environment. The successor representation model of robot's state is constructed through reinforcement learning. Combined with the goal-based reward function, the shortest path can be replanned to adapt to environment changes. The proposed method is tested in simulated maze, Kitti dataset and corridor environment. Compared with other bionic navigation methods, it has a faster computing speed and higher precision with bionic characteristics. Compared with visual navigation methods, the navigation tasks can robustly be accomplished without complex system design.

Singularity Analysis and Complete Methods to Compute the Inverse Kinematics for a 6-DOF UR/TM-Type Robot

Improving the strategies employed to control robotic arms is of great importance because of the increase in their use in advanced supervisory control strategies, such as digital twins. The inverse kinematic (IK) control of manipulators requires an IK solution and an awareness of the singular configurations. This work presents a complete IK calculation system with singularity analysis for the UR5 robotic arm created by Universal Robots. For a specific robot pose, different angle solution sets are obtained, and one of these solution sets has to be selected to achieve movement continuity and avoid singularities. Two methods for this double purpose are proposed: one calculates all the solution possibilities, and the other obtains only one solution set by following a sequence of decisions and calculations clearly stated by a finite state machine (FSM). Both methods are effective in managing singularities. The FSM-based method complements the IK solution procedure with advantages in the number of computations and performance by producing results that would not lead the joints to move abruptly. The results prove that the presented methods select an IK solution that does not result in a singular configuration, and that most of the time, they lead to the same valid IK solution.

BIM-based indoor mobile robot initialization for construction automation using object detection

In recent years, there has been increasing interest in robotic solutions to revolutionize the conventional construction industry. Despite various advances in developing mobile robotic solutions for construction automation. One key bottleneck towards a fully automated robotic solution in construction is the initialization of the mobile robot. Currently, most of the commercialized mobile construction robots are manually initialized before autonomous navigation can be performed at the construction sites for automated tasks. Even if the robot is initialized, the location information can be lost while navigating and re-initialization is required to resume the navigation. Any wrong initialization can cause failure in robot pose tracking and thus prevent the robot from performing the planned tasks. However, in indoor construction sites, GPS is not accessible, and indoor infrastructures, such as beacon devices are not available for robot initialization. In addition, construction environments are dynamic with significant change in scenes and structures for different construction blocks and floors, making pre-scanning of the environments and map matching difficult and time-consuming. An infrastructure-free and environment-independent robot initialization method is therefore required. In this paper, we propose an integrated Building Information Model (BIM)-based indoor robot initialization system using an object detector to automatically initialize the mobile robot when it is deployed at an unknown location. Convolutional neural network (CNN)-based object detection technique is used to detect and locate the visual features, which are widely distributed building components at construction sites. A feature matching algorithm is developed to correlate the acquired online information of detected features with geometric and semantic information retrieved from BIM. The robot location in the BIM coordinate frame is then estimated based on the feature association. Moreover, the proposed system aggregates the BIM information and the sensory information to supervise the online robot decision making, making the entire system fully automatic. The proposed system is validated through experiments in various environments including a university building and ongoing construction sites.

Kinematics and Stiffness Modeling of Soft Robot With a Concentric Backbone

Abstract Soft robots can undergo large elastic deformations and adapt to complex shapes. However, they lack the structural strength to withstand external loads due to the intrinsic compliance of fabrication materials (silicone or rubber). In this paper, we present a novel stiffness modulation approach that controls the robot’s stiffness on-demand without permanently affecting the intrinsic compliance of the elastomeric body. Inspired by concentric tube robots, this approach uses a Nitinol tube as the backbone, which can be slid in and out of the soft robot body to achieve robot pose or stiffness modulation. To validate the proposed idea, we fabricated a tendon-driven concentric tube (TDCT) soft robot and developed the model based on Cosserat rod theory. The model is validated in different scenarios by varying the joint-space tendon input and task-space external contact force. Experimental results indicate that the model is capable of estimating the shape of the TDCT soft robot with an average root-mean-square error (RMSE) of 0.90 (0.56% of total length) mm and average tip error of 1.49 (0.93% of total length) mm. Simulation studies demonstrate that the Nitinol backbone insertion can enhance the kinematic workspace and reduce the compliance of the TDCT soft robot by 57.7%. Two case studies (object manipulation and soft laparoscopic photodynamic therapy) are presented to demonstrate the potential application of the proposed design.

A SLAM Algorithm Based on Edge-Cloud Collaborative Computing

Simultaneous localization and mapping (SLAM) is a typical computing-intensive task. Based on its own computing power, a mobile robot has difficult meeting the real-time performance and accuracy requirements for the SLAM process at the same time. Benefiting from the rapid growth of the network data transmission rate, cloud computing technology begins to be applied in the robotics. There is the reliability problem caused by solely relying on cloud computing. To compensate for the insufficient airborne capacity, ensure the real-time performance and reliability, and improve the accuracy, a SLAM algorithm based on edge-cloud collaborative computing is proposed. The edge estimates the mobile robot pose and the local map using a square root unscented Kalman filter (SR-UKF). The cloud estimates the mobile robot pose and the global map using a distributed square root unscented particle filter (DSR-UPF). By using sufficient particles in the cloud, DSR-UPF can improve the SLAM accuracy. The cloud returns the particle with the largest posteriori probability to the edge, and the edge performs edge-cloud data fusion based on probability. Both the simulation and the experimental results show that the proposed algorithm can improve the estimation accuracy and reduce the execution time at the same time. By transferring the heavy computation from robots to the cloud, it can enhance the environmental adaptability of mobile robots.

Maintenance robot for remote assembly of protective jackets on live gas risers

Combining high-rise robotics research and polyurethane (PU) repair techniques presents new opportunities and unique challenges for utility pipe maintenance. Aiming at solving the complex pose alignment issue of robots and control PU curing, this paper automates the repair process by assembling protective jackets with PU foam. A wire-hoisted robot was equipped with anchors and a twin-gripper mechanism for workspace definition, along with a manipulator that positioned a pair of mirrored end effectors to perform surface polishing, jacket installation, and PU curing onto the pipe. Consequently, the robot pose was stabilized. The jacket pair was assembled onto the pipe with high precision, yielding high-quality repair, a success rate >96%, short cycle time, and low material wastage. This research may inspire innovative automation approaches for building maintenance, while utility companies can deploy the robot system for sustainable pipe service life.

The Localization Methods in Mobile Robot Navigation Technology: A Patent Review

Background: The mobile robot navigation technology includes two key issues: the localization and the path planning, of which the localization is the most basic part of mobile robot navigation, because the target point is meaningful only when mobile robot's pose is clarified, and path planning can be done on this basis. Therefore, it is of great significance to have a good localization method for mobile robots. Objective: This study aimed to summarize the existing localization methods in mobile robot navigation technology and introduce their classifications, characteristics, as well as the stage and trends of development. Methods: A retrospective review of various patents and literature related to localization methods in mobile robot navigation technology was performed. The characteristics, differences, and applications of different localization methods were also introduced. Results: The existing localization methods were analyzed and compared, and their typical characteristics are summarized. The main applications, as well as the pros and cons in the current development stage, were summarized and analyzed. In addition, the development trend of patents related to positioning methods is also discussed. Conclusion: This paper summarized three localization methods and provided numerous cases. It can be seen that most of the existing localization methods are dedicated to improving the positioning accuracy and efficiency of mobile robots, while there is less research on the robustness of mobile robot localization, which is not conducive to the research on robots with the unknown working environment and long moving distances. Therefore, related research patents and documents should be proposed. Current and future development trends are also looking forward.

Evaluation and prediction method of robot pose repeatability based on statistical distance

Pose repeatability is one of the most important performance specifications of robots, and its evaluation and prediction methods are of great significance to give full play to the precision characteristics of robot. In this paper, a pose repeatability evaluation method based on statistical distance and spatial probability ellipsoid is proposed, which can accurately and effectively describe the coupling effect between repeatability of various dimensionalities. On this basis, a prediction model of pose repeatability based on equivalent system is proposed. In this model, the pose repeatability of the end-effector is considered to be generated only by the equivalent repeatability of the actuated joint, and the latter is described as a function of the ideal actuated joint variables, which can accurately describe the variation of the pose repeatability caused by the uncertainties of the robot system in the whole workspace. Simulation and experimental study on a 6-DOF parallel robot are carried out to illustrate the effectiveness of the proposed approach.

Situational Graphs for Robot Navigation in Structured Indoor Environments

Mobile robots should be aware of their <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">situation</i> , comprising the deep understanding of their surrounding environment along with the estimation of its own state, to successfully make intelligent decisions and execute tasks autonomously in real environments. 3D scene graphs are an emerging field of research that propose to represent the environment in a joint model comprising geometric, semantic and relational/topological dimensions. Although 3D scene graphs have already been combined with SLAM techniques to provide robots with situational understanding, further research is still required to effectively deploy them on-board mobile robots. To this end, we present in this paper a novel, real-time, online built Situational Graph ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S-Graph</i> ), which combines in a single optimizable graph, the representation of the environment with the aforementioned three dimensions, together with the robot pose. Our method utilizes odometry readings and planar surfaces extracted from 3D LiDAR scans, to construct and optimize in real-time a three layered <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S-Graph</i> that includes (1) a robot tracking layer where the robot poses are registered, (2) a metric-semantic layer with features such as planar walls and (3) our novel topological layer constraining the planar walls using higher-level features such as corridors and rooms. Our proposal does not only demonstrate state-of-the-art results for pose estimation of the robot, but also contributes with a metric-semantic-topological model of the environment.

Enhanced Accuracy in Magnetic Actuation: Closed-loop Control of a Magnetic Agent with Low-Error Numerical Magnetic Model Estimation.

Magnetic actuation holds promise for wirelessly controlling small, magnetic surgical tools and may enable the next generation of ultra minimally invasive surgical robotic systems. Precise torque and force exertion are required for safe surgical operations and accurate state control. Dipole field estimation models perform well far from electromagnets but yield large errors near coils. Thus, manipulations near coils suffer from severe (10×) field modeling errors. We experimentally quantify closed-loop magnetic agent control performance by using both a highly erroneous dipole model and a more accurate numerical magnetic model to estimate magnetic forces and torques for any given robot pose in 2D. We compare experimental measurements with estimation errors for the dipole model and our finite element analysis (FEA) based model of fields near coils. With five different paths designed for this study, we demonstrate that FEA-based magnetic field modeling reduces positioning root-mean-square (RMS) errors by 48% to 79% as compared with dipole models. Models demonstrate close agreement for magnetic field direction estimation, showing similar accuracy for orientation control. Such improved magnetic modelling is crucial for systems requiring robust estimates of magnetic forces for positioning agents, particularly in force-sensitive environments like surgical manipulation.

Semi-Perspective Decoupled Heatmaps for 3D Robot Pose Estimation From Depth Maps

Knowing the exact 3D location of workers and robots in a collaborative environment enables several real applications, such as the detection of unsafe situations or the study of mutual interactions for statistical and social purposes. In this letter, we propose a non-invasive and light-invariant framework based on depth devices and deep neural networks to estimate the 3D pose of robots from an external camera. The method can be applied to any robot without requiring hardware access to the internal states. We introduce a novel representation of the predicted pose, namely <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Semi-Perspective Decoupled Heatmaps</i> (SPDH), to accurately compute 3D joint locations in world coordinates adapting efficient deep networks designed for the 2D Human Pose Estimation. The proposed approach, which takes as input a depth representation based on XYZ coordinates, can be trained on synthetic depth data and applied to real-world settings without the need for domain adaptation techniques. To this end, we present the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SimBa</i> dataset, based on both synthetic and real depth images, and use it for the experimental evaluation. Results show that the proposed approach, made of a specific depth map representation and the SPDH, overcomes the current state of the art.

Deep Learning-Based 3D Pose Reconstruction of an Underwater Soft Robotic Hand and Its Biomimetic Evaluation

Soft robotic hand shows considerable promise for various grasping applications. However, the sensing and reconstruction of the robot pose will cause limitation during the design and fabrication. In this work, we present a novel 3D pose reconstruction approach to analyze the grasping motion of a bidirectional soft robotic hand using experiment videos. The images from top, front, back, left and right views were collected using an one-camera-multiple-mirror (OCMM) imaging device. The experiment videos about underwater grasp were captured and further analyzed. The coordinate and orientation information of soft fingers are detected based on deep learning methods. A pretrained Faster RCNN model is used to detect the position of fingertips, while a pretrained U-Net classifier is applied to determine the side shape of the fingers. Based on the kinematic modelling, the corresponding coordinate and orientation databases are established. The 3D pose reconstructed results present a satisfactory performance and effectiveness. Based on efficacy coefficient method, the finger contribution of the bending angle and distance between fingertips of soft robotic hand was analyzed compared with that of human hand. The results show that the soft robot hand performs a human-like motion in both single-direction and bidirectional grasping.

Sigma-FP: Robot Mapping of 3D Floor Plans With an RGB-D Camera Under Uncertainty

This letter presents Sigma-FP, a novel 3D reconstruction method to obtain the floor plan of a multi-room environment from a sequence of RGB-D images captured by a wheeled mobile robot. For each input image, the planar patches of visible walls are extracted and subsequently characterized by a multivariate Gaussian distribution in the convenient Plane Parameter Space. Then, accounting for the probabilistic nature of the robot localization, we transform and combine the planar patches from the camera frame into a 3D global model, where the planar patches include both the plane estimation uncertainty and the propagation of the robot pose uncertainty. Additionally, processing depth data, we detect openings (doors and windows) in the wall, which are also incorporated in the 3D global model to provide a more realistic representation. Experimental results, in both real-world and synthetic environments, demonstrate that our method outperforms state-of-the-art methods, both in time and accuracy, while just relying on Atlanta world assumption.

Learning Stable Vector Fields on Lie Groups

Learning robot motions from demonstration requires models able to specify vector fields for the full robot pose when the task is defined in operational space. Recent advances in reactive motion generation have shown that learning adaptive, reactive, smooth, and stable vector fields is possible. However, these approaches define vector fields on a flat Euclidean manifold, while representing vector fields for orientations requires modeling the dynamics in non-Euclidean manifolds, such as Lie Groups. In this paper, we present a novel vector field model that can guarantee most of the properties of previous approaches i.e., stability, smoothness, and reactivity beyond the Euclidean space. In the experimental evaluation, we show the performance of our proposed vector field model to learn stable vector fields for full robot poses as SE(2) and SE(3) in both simulated and real robotics tasks.

A general approach for robot pose error compensation based on an equivalent joint motion error model

A general error modelling, measurement, and compensation approach for robot calibration was proposed based on the equivalent joint motion error model from the perspective of vector space. First, the end pose error was expressed as a linear combination of the twists and the equivalent motion errors of the actuated joints, and the latter were described as functions of the ideal configuration. On this basis, the pose error in operating space was decoupled in joint space, and the regression model corresponding to each actuated joint was established. Then, a statistical index with dimensional consistency was proposed to evaluate the predictive capability of the model, and the variation of the prediction accuracy in the presence of system noise was studied. Finally, experiments were conducted adopting the strategy of off-line identification and on-line compensation. After calibration, the average value of robot position and attitude errors can be reduced to 0.046 mm and 0.011 deg, respectively.