Robot Pose Research Articles

CERBERUS: Autonomous Legged and Aerial Robotic Exploration in the Tunnel and Urban Circuits of the DARPA Subterranean Challenge

Autonomous exploration of subterranean environments constitutes a major frontier for robotic systems, as underground settings present key challenges that can render robot autonomy hard to achieve. This problem has motivated the DARPA Subterranean Challenge, where teams of robots search for objects of interest in various underground environments. In response, we present the CERBERUS system-of-systems, as a unified strategy for subterranean exploration using legged and flying robots. Our proposed approach relies on ANYmal quadraped as primary robots, exploiting their endurance and ability to traverse challenging terrain. For aerial robots, we use both conventional and collision-tolerant multirotors to explore spaces too narrow or otherwise unreachable by ground systems. Anticipating degraded sensing conditions, we developed a complementary multimodal sensor-fusion approach, utilizing camera, LiDAR, and inertial data for resilient robot pose estimation. Individual robot pose estimates are refined by a centralized multi-robot map-optimization approach to improve the reported location accuracy of detected objects of interest in the DARPA-defined coordinate frame. Furthermore, a unified exploration path-planning policy is presented to facilitate the autonomous operation of both legged and aerial robots in complex underground networks. Finally, to enable communication among team agents and the base station, CERBERUS utilizes a ground rover with a high-gain antenna and an optical fiber connection to the base station and wireless “breadcrumb” nodes deployed by the legged robots. We report results from the CERBERUS system-of-systems deployment at the DARPA Subterranean Challenge’s Tunnel and Urban Circuit events, along with the current limitations and the lessons learned for the benefit of the community.

A novel Stereo-IMU system based on accurate pose parameterization of two-wheeled inverted pendulum (TWIP) robot in localization and mapping

PurposeThis paper aims to propose a method to solve the problem of localization and mapping of a two-wheeled inverted pendulum (TWIP) robot on the ground using the Stereo–inertial measurement unit (IMU) system. This method reparametrizes the pose according to the motion characteristics of TWIP and considers the impact of uneven ground on vision and IMU, which is more adaptable to the real environment.Design/methodology/approachWhen TWIP moves, it is constrained by the ground and swings back and forth to maintain balance. Therefore, the authors parameterize the robot pose as SE(2) pose plus pitch according to the motion characteristics of TWIP. However, the authors do not omit disturbances in other directions but perform error modeling, which is integrated into the visual constraints and IMU pre-integration constraints as an error term. Finally, the authors analyze the influence of the error term on the vision and IMU constraints during the optimization process. Compared to traditional algorithms, the algorithm is simpler and better adapt to the real environment.FindingsThe results of indoor and outdoor experiments show that, for the TWIP robot, the method has better positioning accuracy and robustness compared with the state-of-the-art.Originality/valueThe algorithm in this paper is proposed for the localization and mapping of a TWIP robot. Different from the traditional positioning method on SE(3), this paper parameterizes the robot pose as SE(2) pose plus pitch according to the motion of TWIP and the motion disturbances in other directions are integrated into visual constraints and IMU pre-integration constraints as error terms, which simplifies the optimization parameters, better adapts to the real environment and improves the accuracy of positioning.

Position and Pose Measurement of 3-PRS Ankle Rehabilitation Robot Based on Deep Learning

Introduction: The position and pose measurement of the rehabilitation robot plays a very important role in patient rehabilitation movement, and the non-contact real-time robot position and pose measurement is of great significance. Rehabilitation training is a relatively complicated process, so it is very important to detect the training process of the rehabilitation robot in real time and accuracy. The method of the deep learning has a very good effect on monitoring the rehabilitation robot state. Methods: The structure sketch and the 3D model of the 3-PRS ankle rehabilitation robot are established, and the mechanism kinematics is analyzed to obtain the relationship between the driving input - the three slider heights - and the position and pose parameters. The whole network of the position and pose measurement is composed of two stages: (1) measuring the slider heights using the CNN based on the robot image and (2) calculating the position and pose parameter using the BPNN based on the measured slider heights from the CNN. According to the characteristics of continuous variation of the slider heights, a regression CNN is proposed and established to measure the robot slider height. Based on the data calculated by using the inverse kinematics of the 3-PRS ankle rehabilitation robot, a BPNN is established to solve the forward kinematics for the position and pose. Results: The experimental results show that the regression CNN outputs the slider height and then the BPNN accurately outputs the corresponding position and pose. Eventually, the position and pose parameters are obtained from the robot image. Compared with the traditional robot position and pose measurement method, the proposed method has significant advantages. Conclusion: The proposed 3-PRS ankle rehabilitation position and pose method can not only shorten the experiment period and cost, but also get excellent timeliness and precision. The proposed approach can help the medical staff to monitor the status of the rehabilitation robot and help the patient rehabilitation in training. Discussion: The goal of the work is to construct a new position and pose detection network based on the combination of the regression convolutional neural network (CNN) and the back propagation neural network (BPNN). The main contribution is to measure the position and pose of the 3-PRS ankle rehabilitation robot in real time, which improves the measurement accuracy and the efficiency of the medical staff work.

A Factor Graph Approach to Simultaneous Robot-World and Hand-Eye Calibration Based on Lie Group

The existing robot-world and hand-eye calibration algorithms always treated the robot pose as a deterministic value. Its uncertainty is not taken into account, which affects the calibration accuracy when the robot’s position accuracy is low. In this paper, we proposed a factor graph approach to simultaneous robot-world and hand-eye calibration, which considers the measurement error of the robot joint encoder. And then, the nonlinear optimization of reprojection error minimization based on a product of exponentials (POE)-based model is implemented to solve the calibration problem based on lie groups. Through simulation and experiment, we validated that the proposed algorithm can get more accurate calibration results compared with the state algorithm.

3D-RPP: a novel 3D vision-based Pose Perception Approach for Industrial Robots

Enhanced by artificial intelligence, industrial robots are becoming more powerful, gaining a large variety of applications in intelligent factories. Pose perception, the aim of which is to obtain the joint coordinates of the robot in the camera coordinate system, has been proposed as a promising technology for multi-robot intelligent interaction. A large number of efforts have been made on studying robot pose perception. However, most of existing studies estimate the position of robot joints in 3D space by just using 2D color images, and perform pose perception by estimating the position of joint key points in the image, which could become infeasible in scenarios with various camera views and background environments. To address the issue, considering the information about the robot itself as a prior knowledge, we propose a novel approach for robot pose perception, named 3D-RPP. We adopt a 3D visual point cloud to estimate the rigid transformation of the camera coordinate system with respect to the robot base coordinate system, which effectively improves the accuracy of the obtained robot joint position in the camera coordinate system. We conduct extensive experiments on ROKAE xMate3 robot to investigate the performance of 3D-RPP, and the experimental results show that 3D-RPP could solve the pose perception problem well.

Multi-Stage Refinement Feature Matching Using Adaptive ORB Features for Robotic Vision Navigation

Feature matching is a crucial technique to estimate robot pose in the light of natural landmarks for a vision navigation system. It is difficult for the present feature matching approaches to balance the accuracy, robustness and efficiency for a vision-based robot in complex workspaces. A multi-stage KNN-TF-BF-EN-RANSAC (KTBER) refinement matching technique is proposed to combine with the adaptive ORB feature extraction method, in order to develop a novel KTBER_AORB model that can outperform the state-of-the-art feature extraction and matching techniques. Firstly, the adaptive ORB (AORB) features are extracted in the Laplacian Transformation of Gaussian (LTOG) pyramid by using a set of variable-radius templates. The template radius is adjusted in terms of different brightness and contrast conditions. Then, the Eigenvector Norm (EN) and Bidirectional Filtering (BF) techniques are devised to establish a novel KTBER model, which can match AORB feature pairs not only according to the local appearance-based similarity of feature descriptors in the Hamming space, but also by means of the similarity of local image structure and global grayscale information for feature pairs in the Euclidean space. Finally, a large number of feature matching experiments are conducted on five public datasets and our mobile robot dataset, in order to test the algorithm performances for different challenging scenarios, including illumination change, motion blur, image rotation, viewpoint change, and low texture. On the one hand, an intensive experimental analysis of our model with different configurations is made to check the specific effects of different algorithm elements. On the other hand, the KTBER_AORB model is further compared with four state-of-the-art techniques, e.g., SIFT, ORB-SLAM, MP-ORB, and LAAT. The experimental results demonstrate that the KTBER_AORB model outperforms other state-of-the-art feature matching techniques on accuracy and robustness, still preserving a higher efficiency than most counterparts.

Advanced quaternion unscented Kalman filter based on SLAM of mobile robot pose estimation

Advanced quaternion unscented Kalman filter based on SLAM of mobile robot pose estimation

Terrain Estimation with Least Squares and Virtual Model Control for Quadruped Robots

In this paper, a method to estimate the terrain based on footholds and least-squares is presented to make quadrupedal locomotion stable on the sloped terrains, with joint encoders, force sensors and inertial measurement unit (IMU) instead of visual sensors. This method which is tested successfully in simulation can predict two types of terrain, one for adjusting the robot pose and the other for adjusting the trajectory of swing phase. And to make full use of these new capabilities, general balance and locomotion controller is presented. The virtual model controller is embedded into architecture and it allows the robot to handle unexpected terrain. The quadrupedal locomotion in terrain with different inclination prove the robustness of this method.

Evaluation on Surgical Robotic Pose Repeatability Based on Laser Tracking Measurement

To provide accurate information for registration and safety evaluation of surgical robot, the pose repeatability measurement method was proposed. According to the terminal instrument of the master-slave surgical robot (such as high-frequency electric knife, ultrasonic knife), a suitable target ball fixture was designed. The node data at 10%, 50% and 100% rated speed were measured respectively. Through data analysis, the pose repeatability property of the tested samples at different speeds was obtained. It has high applicability and repeatability, and can meet the requirements of data traceability and registration testing.

Safe-Nav: learning to prevent PointGoal navigation failure in unknown environments

Training robots to safely navigate (Safe-Nav) in uncertain complex environments using the RGB-D sensor is quite challenging as it involves the performance of different tasks such as obstacle avoidance, optimal path planning, and control. Traditional navigation approaches cannot generate suitable paths which guarantee enough visible features. Recent learning-based methods are still not mature enough due to their proneness to collisions and prohibitive computational cost. This paper focuses on generating safe trajectories to the desired goal while avoiding collisions and tracking failure in unknown complex environments. We present Safe-Nav, a hierarchical framework composed of the visual simultaneous localization and mapping (SLAM) module, the global planner module and the local planner module. The visual SLAM module generates the navigation map and the robot pose. The global planner module plans a local waypoint on the real-time navigation map. In the local planner module, a deep-reinforcement-learning-based (DRL-based) policy is presented for taking safe actions towards local waypoints. Our DRL-based policy can learn different navigation skills (e.g., avoiding collisions and avoiding tracking failure) through specialized modes without any supervisory signals when the PointGoal-navigation-specied reward is provided. We have demonstrated the performance of our proposed Safe-Nav in the Habitat simulation environment. Our approach outperforms the recent learning-based method and conventional navigation approach with relative improvements of over 205% (0.55 vs. 0.18) and 139% (0.55 vs. 0.23) in the success rate, respectively.

Adaptive Path Following for a Differential Drive Robot with EKF-based Localization

For a mobile robot, uncertainties in its operating environment and system setup are common challenges. The robot may need to follow a new path in order to avoid collision with a previously unknown object in the environment. Additionally, the robot does not usually have precise knowledge of its own pose. To compensate for these uncertainties, we develop an adaptive path following scheme for a differential drive robot with localization capabilities. We employ a least-squares adaptive law to estimate an unknown circular path. With measurements to landmarks in the environment, we use an extended Kalman filter to estimate the robot's pose. Furthermore, a cascaded control scheme is used to track the estimated path. Simulation and experimental results are provided to verify the developed adaptive path following scheme and demonstrate its capabilities.

Lidar SLAM Based on Particle Filter and Graph Optimization for Substation Inspection

Simultaneous Localization and Mapping (SLAM) is the core technology of intelligent substation inspection robot. Because of lightweight computation, Rao-Blackwellized Particle Filter (RBPF) is widely used in two-dimensional SLAM. However, it suffers from poor positioning accuracy, low robustness and rapid cumulative errors despite recent improvement. This paper presents a lidar SLAM system based on RBPF and graph optimization that can adapt to unstructured operating environment of substation. Firstly, the diversity of particles is increased by rebuilding the resample algorithm to improve the robustness of the system, and high-quality poses are estimated in submaps. Secondly, the multi-submap system is established to construct odometry constraints (one pose corresponds to two submaps). Furthermore, loop detector is an important part of optimization algorithm, and the branch-bound method is used to reduce computation burden and accelerate the loop detection. Finally, global poses of robot are optimized by the whole odometry and loop constraints in real time. Experiment results show that the proposed method is more accurate than other methods, and can maintain and produce high-precision positioning and mapping in complex substation operation and maintenance environment. It provides a new idea for intelligent substation inspection and positioning method.

Monocular Visual-Inertial and Robotic-Arm Calibration in a Unifying Framework

Reliable and accurate calibration for camera, inertial measurement unit (IMU) and robot is a critical prerequisite for visual-inertial based robot pose estimation and surrounding environment perception. However, traditional calibrations suffer inaccuracy and inconsistency. To address these problems, this paper proposes a monocular visual-inertial and robotic-arm calibration in a unifying framework. In our method, the spatial relationship is geometrically correlated between the sensing units and robotic arm. The decoupled estimations on rotation and translation could reduce the coupled errors during the optimization. Additionally, the robotic calibration moving trajectory has been designed in a spiral pattern that enables full excitations on 6 DOF motions repeatably and consistently. The calibration has been evaluated on our developed platform. In the experiments, the calibration achieves the accuracy with rotation and translation RMSEs less than 0.7° and 0.01 m, respectively. The comparisons with state-of-the-art results prove our calibration consistency, accuracy and effectiveness.

Robot Pose Estimation Method Based on Image and Point Cloud Fusion with Dynamic Feature Elimination

Robot Pose Estimation Method Based on Image and Point Cloud Fusion with Dynamic Feature Elimination

Real-Time Robot Pose Correction on Curved Surface Employing 6-Axis Force/Torque Sensor

Research on robot trajectory correction on a curved surface has seen a surge in the last two decades to deal with issues in many machining processes. Due to the intricacies involved in complex machining, the real-time correction of robot end-effector position and orientation is highly challenging. The existing approaches are either computationally expensive or inadequate for high accuracy in desired trajectory tracking. Hence, this research explores and implements a novel sophisticated approach, barring minor complications, to empower industrial robots to adjust poses in real-time automatically. In contrast with other studies, the proposed technique does not require any prior geometric information of the workpiece, such as a CAD model or a 3D scan. Our proposed technique relies solely on force/torque sensor feedback information obtained from a 6-axis force/torque sensor installed at the end-effector. When the developed tool comes in contact with the curved surface, the force sensor transmits the data to the pose correction algorithm. The pose correction algorithm estimates the adjustment required to make the tool normal to the surface. The effectiveness and feasibility of the proposed scheme were validated through numerous experiments carried out with a 6-DOF industrial robot. The final results of the curved surface trajectory correction demonstrates that the contact-based robot pose correction has significantly improved. The estimated average error on depth (Z-axis), and angles (Rx,Ry) are 0.7 <i>mm</i>, 0.7&#x00B0;, and 0.9&#x00B0;, respectively.

Optimization of an autonomous robotic drilling system for the machining of aluminum aerospace alloys

This paper aims to identify the capability of a highly flexible industrial robot modified with a high-speed machine spindle for drilling of aluminum 6061-T6. With a focus on drilling feed rate, spindle speed, and pecking cycle, the hole surface roughness and exit burr heights were investigated using the Taguchi design methodology. A state of the art condition monitoring system was used to identify the vibrations experienced during drilling operation and to establish which robot pose had increased stiffness, and thus the optimum workspace for drilling. When benchmarked against a CNC machine the results show that the CNC was capable of producing the best surface finish and the lowest burr heights. However, the robot system matched and outperformed the CNC in several experiments and there is much scope for further optimization of the process. By identifying the optimum pose for drilling together with the idealized settings, the proposed drilling system is shown to be far more flexible than a CNC milling machine and when considering the optimized drilling of aerospace aluminum this robotic solution has the potential to drastically improve productivity.

An Adaptive Pose Fusion Method for Indoor Map Construction

The vision-based robot pose estimation and mapping system has the disadvantage of low pose estimation accuracy and poor local detail mapping effects, while the modeling environment has poor features, high dynamics, weak light, and multiple shadows, among others. To address these issues, we propose an adaptive pose fusion (APF) method to fuse the robot’s pose and use the optimized pose to construct an indoor map. Firstly, the proposed method calculates the robot’s pose by the camera and inertial measurement unit (IMU), respectively. Then, the pose fusion method is adaptively selected according to the motion state of the robot. When the robot is in a static state, the proposed method directly uses the extended Kalman filter (EKF) method to fuse camera and IMU data. When the robot is in a motive state, the weighted coefficient is determined according to the matching success rate of the feature points, and the weighted pose fusion (WPF) method is used to fuse camera and IMU data. According to the different states, a series of new poses of the robot are obtained. Secondly, the fusion optimized pose is used to correct the distance and azimuth angle of the laser points obtained by LiDAR, and a Gauss–Newton iterative matching process is used to match the corresponding laser points to construct an indoor map. Finally, a pose fusion experiment is designed, and the EuRoc data and the measured data are used to verify the effectiveness of this method. The experimental results confirm that this method provides higher pose estimation accuracy compared with the robust visual inertial odometry (ROVIO) and visual-inertial ORB-SLAM (VI ORB-SLAM) algorithms. Compared with the Cartographer algorithm, this method provides higher two-dimensional map modeling accuracy and modeling performance.

Unsupervised deep learning based ego motion estimation with a downward facing camera

Knowing the robot's pose is a crucial prerequisite for mobile robot tasks such as collision avoidance or autonomous navigation. Using powerful predictive models to estimate transformations for visual odometry via downward facing cameras is an understudied area of research. This work proposes a novel approach based on deep learning for estimating ego motion with a downward looking camera. The network can be trained completely unsupervised and is not restricted to a specific motion model. We propose two neural network architectures based on the Early Fusion and Slow Fusion design principle: “EarlyBird” and “SlowBird”. Both networks share a Spatial Transformer layer for image warping and are trained with a modified structural similarity index (SSIM) loss function. Experiments carried out in simulation and for a real world differential drive robot show similar and partially better results of our proposed deep learning based approaches compared to a state-of-the-art method based on fast Fourier transformation.

A Reliable Linearly Separable Optimization Method for SLAM Estimation based on Stereo Features

SLAM (Simultaneous Localization and Mapping) based on graphical representation is used to solve optimization non-linear problems. The solutions of graph-based SLAM have n noisy measurements and re-projection errors lead to uncertainty in robot pose and landmark position. The proposed system providesa novel two-stage architecture for visual SLAM that was designed and implemented. The new two-stage architecture resulted in an exactly linear graph optimization, by using a linear optimization method for graph optimization, so the problem now becomes linear. We take benefit of all camera measurements and needn’t ignore any frames. We enhance accuracy of large scale VSLM system, minimize processing time and decrease computational cost. We show that our system can be used with a higher accuracy and processing time than well-known systems based on bundle adjustment.

H-infinity optimised control of external inertial actuators for higher precision robotic machining

ABSTRACT Serial-link industrial robots that feature small footprints and large working volumes have been used widely in various applications. However, their low-stiffness-induced vibration hinders application in precision robotic machining processes. In robotic milling, the spindle may experience chatter, leading to unsatisfactory surface finishes. Moreover, the chatter frequency varies, depending on the robot pose and its underlying stiffness and inertia characteristics, which is likely to be outside of the controlled robot system bandwidth. One solution is to use inertial actuators close to the source of excitation on the robot to generate forces that counteract the vibration. This paper advances matters by employing optimised H ∞ control strategies for improved effectiveness and robustness to perform active vibration suppression. The strategies are independent of tool paths and take account of different robot poses, hence the variability of the robot dynamic characteristics. Performance is assessed against that of standard velocity feedback controllers in eccentric mass experiments and milling tests. The experimental results show that within a relatively large work-plane (500 mm × 500 mm), H ∞ controllers greatly improve machining capability and reduce machining errors by up to 85%.