High-precision Pose Estimation Research Articles

Cylinder object 6-DOF pose estimation via single perspective circle on cylindrical surface

Abstract Pose estimation is an important research topic in computer vision. In this paper, a novel method for six-degree-of-freedom (6-DOF) pose estimation of cylindrical objects via a single perspective circle on cylindrical surface is proposed. Unlike traditional methods that rely on planar circular features, a circular feature target which is affixed to the side of cylindrical objects is utilized. The geometric properties of the spatial curve formed by its contour are leveraged for pose estimation. Specifically, the imaging model of the spatial curve is established to derive the general equation of its projection curve. The initial pose is determined based on the perspective principle, and the final optimized pose is obtained with the least squares method. To avoid local optima, a verification and correction method is proposed to enhance estimation accuracy. Synthetic and physical experiments demonstrate that the proposed method can achieve real-time, high-precision pose estimation for cylindrical objects, exhibiting robustness against occlusion and noise interference.

High Precision Pose Estimation for Uncooperative Targets Based on Monocular Vision and 1D Laser Fusion

High Precision Pose Estimation for Uncooperative Targets Based on Monocular Vision and 1D Laser Fusion

Improved Multi-Sensor Fusion Dynamic Odometry Based on Neural Networks

High-precision simultaneous localization and mapping (SLAM) in dynamic real-world environments plays a crucial role in autonomous robot navigation, self-driving cars, and drone control. To address this dynamic localization issue, in this paper, a dynamic odometry method is proposed based on FAST-LIVO, a fast LiDAR (light detection and ranging)–inertial–visual odometry system, integrating neural networks with laser, camera, and inertial measurement unit modalities. The method first constructs visual–inertial and LiDAR–inertial odometry subsystems. Then, a lightweight neural network is used to remove dynamic elements from the visual part, and dynamic clustering is applied to the LiDAR part to eliminate dynamic environments, ensuring the reliability of the remaining environmental data. Validation of the datasets shows that the proposed multi-sensor fusion dynamic odometry can achieve high-precision pose estimation in complex dynamic environments with high continuity, reliability, and dynamic robustness.

CylinderTag: An Accurate and Flexible Marker for Cylinder-Shape Objects Pose Estimation Based on Projective Invariants.

High-precision pose estimation based on visual markers has been a thriving research topic in the field of computer vision. However, the suitability of traditional flat markers on curved objects is limited due to the diverse shapes of curved surfaces, which hinders the development of high-precision pose estimation for curved objects. Therefore, this paper proposes a novel visual marker called CylinderTag, which is designed for developable curved surfaces such as cylindrical surfaces. CylinderTag is a cyclic marker that can be firmly attached to objects with a cylindrical shape. Leveraging the manifold assumption, the cross-ratio in projective invariance is utilized for encoding in the direction of zero curvature on the surface. Additionally, to facilitate the usage of CylinderTag, we propose a heuristic search-based marker generator and a high-performance recognizer as well. Moreover, an all-encompassing evaluation of CylinderTag properties is conducted by means of extensive experimentation, covering detection rate, detection speed, dictionary size, localization jitter, and pose estimation accuracy. CylinderTag showcases superior detection performance from varying view angles in comparison to traditional visual markers, accompanied by higher localization accuracy. Furthermore, CylinderTag boasts real-time detection capability and an extensive marker dictionary, offering enhanced versatility and practicality in a wide range of applications. Experimental results demonstrate that the CylinderTag is a highly promising visual marker for use on cylindrical-like surfaces, thus offering important guidance for future research on high-precision visual localization of cylinder-shaped objects. The code is available at: https://github.com/wsakobe/CylinderTag.

Multi-Angle Models and Lightweight Unbiased Decoding-Based Algorithm for Human Pose Estimation

When a top-down method is taken to the task of human pose estimation, the accuracy of joint point localization is often limited by the accuracy of human detection. In addition, conventional algorithms commonly encode the image to generate a heat map before processing, but the systematic error in decoding the heat map back to the original image has an impact on the positioning. Therefore, to address the two problems, we propose an algorithm that uses multiple angle models to generate the human boxes and then performs lightweight decoding to recover the image. The new boxes can better fit humans and the recovery error can be reduced. First, we split the backbone network into three sub-networks, the first sub-network is responsible for generating the original human box, the second sub-network is responsible for generating a coarse pose estimation in the boxes, and the third sub-network is responsible for a high-precision pose estimation. In order to make the human box fit the human body better, with only a small number of interfering pixels inside the box, models of the human boxes with multiple rotation angles are generated. The results from the second sub-network are used to select the best human box. Using this human box as input to the third sub-network can significantly improve the accuracy of the pose estimation. Then to reduce the errors arising from image decoding, we propose a lightweight unbiased decoding strategy that differs from traditional methods by combining multiple possible offsets to select the direction and size of the final offset. On the MPII dataset and the COCO dataset, we compare the proposed algorithm with 11 state-of-the-art algorithms. The experimental results show that the algorithm achieves a large improvement in accuracy for a wide range of image sizes and different metrics.

D-LIOM: Tightly-Coupled Direct LiDAR-Inertial Odometry and Mapping

Simultaneous localization and mapping via LiDAR-Inertial fusion is a crucial technology in many automation-related applications. Recently, a number of approaches based on geometric features have evolved, yielding impressive results via tightly-coupled estimation. This sort of feature-based techniques, however, are inextricably linked to the scanning mechanism of the LiDAR, relying on stable feature detection, and thus are difficult to adapt to multi-LiDAR systems. A few direct solutions, on the other hand, register the raw point cloud with the built probability map, which is more computationally efficient and easy to be extended. But, the existing direct approaches are all loosely-coupled, lacking correction of the IMU biases, and thus only work well in 2D cases. To this end, we present D-LIOM, a tightly-coupled Direct LiDAR-Inertial Odometry and Mapping framework. In D-LIOM, a scan is directly registered to a probability submap, and the LiDAR odometry, the IMU pre-integration, and the gravity constraint are integrated to build a local factor graph in the submap's time window, allowing the system to perform real-time high-precision pose estimation. Furthermore, to eliminate accumulated errors in time, we detect loops and adjust the sparse pose graph based on mutual matching of projected 2D submaps, allowing D-LIOM to run stably in large-scale scenes. In addition, to improve its flexibility to varied sensor combinations, D-LIOM supports multi-LiDAR inputs and facilitates the initialization with a common 6-axis IMU. Extensive experiments demonstrate that D-LIOM largely outperforms the existing state-of-the-art counterparts in mapping effect and localization accuracy as well as with high time efficiency. Lastly, to ensure that our results are entirely reproducible, all necessary data and codes are made open-source available. One introduction video can also be found on the online website.

Relative Pose Estimation of Non-Cooperative Space Targets Using a TOF Camera

It is difficult to determine the accurate pose of non-cooperative space targets in on-orbit servicing (OOS). The visual camera is easily affected by the extreme light environment in space, and the scanning lidar will have motion distortion when the target moves at high speed. Therefore, we proposed a non-cooperative target pose-estimation system combining a registration and a mapping algorithm using a TOF camera. We first introduce the projection model of the TOF camera and proposed a new calibration method. Then, we introduce the three modules of the proposed method: the TOF data preprocessing module, the registration module and the model mapping module. We assembled the experimental platform to conduct semi-physical experiments; the results showed that the proposed method has the smallest translation error 8 mm and Euler angle error 1° compared with other classical methods. The total time consumption is about 100 ms, and the pose tracking frequency can reach 10 Hz. We can conclude that the proposed pose-estimation scheme can achieve the high-precision pose estimation of non-cooperative targets and meet the requirements necessary for aerospace applications.

An Uncertainty Weighted Non-Cooperative Target Pose Estimation Algorithm, Based on Intersecting Vectors

Aiming at the relative pose estimation of non-cooperative targets in space traffic management tasks, a two-step pose estimation method, based on spatially intersecting straight lines, is proposed, which mainly includes three aspects: (1) Use binocular vision to reconstruct the straight space line, and based on the direction vector of the straight line and the intersection of the straight line, solve the pose of the measured target in the measurement coordinate system, and obtain the initial value of the pose estimation. (2) Analyze the uncertainty of the spatial straight-line imaging, construct the uncertainty description matrix of the line, and filter the line features, accordingly. (3) Analyze the problems existing in the current linear distance measurement, construct the spatial linear back-projection error in the parametric coordinate space, and use the linear imaging uncertainty to weigh the projection error term to establish the optimization objective function of the pose estimation. Finally, the nonlinear optimization algorithm is used to iteratively solve the above optimization problem, to obtain high-precision pose estimation results. The experimental results show that the two-step pose estimation algorithm, proposed in this paper, can effectively achieve a high-precision and robust pose estimation for non-cooperative spatial targets. When the measurement distance is 10 m, the position accuracy can reach 10 mm, and the attitude measurement accuracy can reach 1°, which meets the pose estimation accuracy requirements of space traffic management.

R-LIO: Rotating Lidar Inertial Odometry and Mapping

In this paper, we propose a novel simultaneous localization and mapping algorithm, R-LIO, which combines rotating multi-line lidar and inertial measurement unit. R-LIO can achieve real-time and high-precision pose estimation and map-building. R-LIO is mainly composed of four sequential modules, namely nonlinear motion distortion compensation module, frame-to-frame point cloud matching module based on normal distribution transformation by self-adaptive grid, frame-to-submap point cloud matching module based on line and surface feature, and loop closure detection module based on submap-to-submap point cloud matching. R-LIO is tested on public datasets and private datasets, and it is compared quantitatively and qualitatively to the four well-known methods. The test results show that R-LIO has a comparable localization accuracy to well-known algorithms as LIO-SAM, FAST-LIO2, and Faster-LIO in non-rotating lidar data. The standard algorithms cannot function normally with rotating lidar data. Compared with non-rotating lidar data, R-LIO can improve localization and mapping accuracy in rotating lidar data.

Pose estimation of a fast tumbling space noncooperative target using the time-of-flight camera

This article proposes a pose estimation method for a fast tumbling space noncooperative target. The core idea of this method is to extract the target’s body-fixed coordinate system by using the geometric characteristics of the target’s point cloud and then by the body-fixed coordinate system to realize pose initialization and pose tracking of the target. In the extraction of the body-fixed coordinate system, a point cloud of the target, which can be obtained by a time-of-flight camera, can be divided into small plane point clouds firstly; then the geometric information of these plane point clouds can be utilized to extract the target’s descriptive structures, such as the target surfaces and the solar panel supports; and finally the body-fixed coordinate system can be determined by the geometric characteristics of these structures. The body-fixed coordinate system obtained above can be used to determine the pose of consecutive point clouds of the target, that is, to realize the pose initialization and the pose tracking, and accumulated bias often emerges in the pose tracking. To mitigate the accumulated bias, a pose graph optimization method is adopted. In the end of this article, the performance of the proposed method is evaluated by numerical simulations. Simulation results show that when the distance between the target and the chaser is 10 m, the errors of the estimation results of the target’s attitude and position are 0.025° and 0.026 m, respectively. This means that the proposed method can achieve high-precision pose estimation of the noncooperative target.

An RGB-D-Based Cross-Field of View Pose Estimation System for a Free Flight Target in a Wind Tunnel

Estimating the real-time pose of a free flight aircraft in a complex wind tunnel environment is extremely difficult. Due to the high dynamic testing environment, complicated illumination condition, and the unpredictable motion of target, most general pose estimating methods will fail. In this paper, we introduce a cross-field of view (FOV) real-time pose estimation system, which provides high precision pose estimation of the free flight aircraft in the wind tunnel environment. Multiview live RGB-D streams are used in the system as input to ensure the measurement area can be fully covered. First, a multimodal initialization method is developed to measure the spatial relationship between the RGB-D camera and the aircraft. Based on all the input multimodal information, a so-called cross-FOV model is proposed to recognize the dominating sensor and accurately extract the foreground region in an automatic manner. Second, we develop an RGB-D-based pose estimation method for a single target, by which the 3D sparse points and the pose of the target can be simultaneously obtained in real time. Many experiments have been conducted, and an RGB-D image simulation based on 3D modeling is implemented to verify the effectiveness of our algorithm. Both the real scene’s and simulation scene’s experimental results demonstrate the effectiveness of our method.

INTEGRATING DEPTH AND IMAGE SEQUENCES FOR PLANETARY ROVER MAPPING USING RGB-D SENSOR

Abstract. RGB-D camera allows the capture of depth and color information at high data rates, and this makes it possible and beneficial integrate depth and image sequences for planetary rover mapping. The proposed mapping method consists of three steps. First, the strict projection relationship among 3D space, depth data and visual texture data is established based on the imaging principle of RGB-D camera, then, an extended bundle adjustment (BA) based SLAM method with integrated 2D and 3D measurements is applied to the image network for high-precision pose estimation. Next, as the interior and exterior elements of RGB images sequence are available, dense matching is completed with the CMPMVS tool. Finally, according to the registration parameters after ICP, the 3D scene from RGB images can be registered to the 3D scene from depth images well, and the fused point cloud can be obtained. Experiment was performed in an outdoor field to simulate the lunar surface. The experimental results demonstrated the feasibility of the proposed method.

Depth-Based Hand Pose Estimation: Methods, Data, and Challenges

Hand pose estimation has matured rapidly in recent years. The introduction of commodity depth sensors and a multitude of practical applications have spurred new advances. We provide an extensive analysis of the state-of-the-art, focusing on hand pose estimation from a single depth frame. To do so, we have implemented a considerable number of systems, and have released software and evaluation code. We summarize important conclusions here: (1) Coarse pose estimation appears viable for scenes with isolated hands. However, high precision pose estimation [required for immersive virtual reality and cluttered scenes (where hands may be interacting with nearby objects and surfaces) remain a challenge. To spur further progress we introduce a challenging new dataset with diverse, cluttered scenes. (2) Many methods evaluate themselves with disparate criteria, making comparisons difficult. We define a consistent evaluation criteria, rigorously motivated by human experiments. (3) We introduce a simple nearest-neighbor baseline that outperforms most existing systems. This implies that most systems do not generalize beyond their training sets. This also reinforces the under-appreciated point that training data is as important as the model itself. We conclude with directions for future progress.

A high accuracy method for pose estimation based on rotation parameters

In this paper, A high-precision pose estimation method based on rotation parameters is proposed. By introducing the parameterization of rotation matrix, we first formulate a nonlinear least-squares cost function whose optimality conditions constitute a system of three third-order polynomials. Then, based on Gröbner basis method, we solve the solution of polynomial equation directly, and then the attitude is calculated by back-substitution method. More importantly, one of the main advantages of our method is scalability, because the order of the polynomial equations we solve is independent of the number of points involved in the estimation. Finally, we compare the performance of our algorithm with the leading PnP method, both in simulations and experiments, and show that the proposed method achieves the accuracy of approaching the OPnP (optimal solution to the PnP problem).

A survey on Visual-Based Localization: On the benefit of heterogeneous data

We are surrounded by plenty of information about our environment. From these multiple sources, numerous data could be extracted: set of images, 3D model, coloured points cloud... When classical localization devices failed (e.g. GPS sensor in cluttered environments), aforementioned data could be used within a localization framework. This is called Visual Based Localization (VBL). Due to numerous data types that can be collected from a scene, VBL encompasses a large amount of different methods. This paper presents a survey about recent methods that localize a visual acquisition system according to a known environment. We start by categorizing VBL methods into two distinct families: indirect and direct localization systems. As the localization environment is almost always dynamic, we pay special attention to methods designed to handle appearances changes occurring in a scene. Thereafter, we highlight methods exploiting heterogeneous types of data. Finally, we conclude the paper with a discussion on promising trends that could permit to a localization system to reach high precision pose estimation within an area as large as possible.

RGB-D SLAM Based on Extended Bundle Adjustment with 2D and 3D Information.

In the study of SLAM problem using an RGB-D camera, depth information and visual information as two types of primary measurement data are rarely tightly coupled during refinement of camera pose estimation. In this paper, a new method of RGB-D camera SLAM is proposed based on extended bundle adjustment with integrated 2D and 3D information on the basis of a new projection model. First, the geometric relationship between the image plane coordinates and the depth values is constructed through RGB-D camera calibration. Then, 2D and 3D feature points are automatically extracted and matched between consecutive frames to build a continuous image network. Finally, extended bundle adjustment based on the new projection model, which takes both image and depth measurements into consideration, is applied to the image network for high-precision pose estimation. Field experiments show that the proposed method has a notably better performance than the traditional method, and the experimental results demonstrate the effectiveness of the proposed method in improving localization accuracy.