Relative Pose Estimation Research Articles

Multi-type feature fusion visual navigation for asteroid landing

In order to address the challenges encountered in visual navigation for asteroid landing using traditional point features, such as significant recognition and extraction errors, low computational efficiency, and limited navigation accuracy, a novel approach for multi-type fusion visual navigation is proposed. This method aims to overcome the limitations of single-type features and enhance navigation accuracy. Analytical criteria for selecting multi-type features are introduced, which simultaneously improve computational efficiency and system navigation accuracy. Concerning pose estimation, both absolute and relative pose estimation methods based on multi-type feature fusion are proposed, and multi-type feature normalization is established, which significantly improves system navigation accuracy and lays the groundwork for flexible application of joint absolute-relative estimation. The feasibility and effectiveness of the proposed method are validated through simulation experiments through 4769 Castalia.

MURP: Multi-Agent Ultra-Wideband Relative Pose Estimation With Constrained Communications in 3D Environments

MURP: Multi-Agent Ultra-Wideband Relative Pose Estimation With Constrained Communications in 3D Environments

Real-Time LiDAR–Inertial Simultaneous Localization and Mesh Reconstruction

In this paper, a novel LiDAR–inertial-based Simultaneous Localization and Mesh Reconstruction (LI-SLAMesh) system is proposed, which can achieve fast and robust pose tracking and online mesh reconstruction in an outdoor environment. The LI-SLAMesh system consists of two components, including LiDAR–inertial odometry and a Truncated Signed Distance Field (TSDF) free online reconstruction module. Firstly, to reduce the odometry drift errors we use scan-to-map matching, and inter-frame inertial information is used to generate prior relative pose estimation for later LiDAR-dominated optimization. Then, based on the motivation that the unevenly distributed residual terms tend to degrade the nonlinear optimizer, a novel residual density-driven Gauss–Newton method is proposed to obtain the optimal pose estimation. Secondly, to achieve fast and accurate 3D reconstruction, compared with TSDF-based mapping mechanism, a more compact map representation is proposed, which only maintains the occupied voxels and computes the vertices’ SDF values of each occupied voxels using an iterative Implicit Moving Least Squares (IMLS) algorithm. Then, marching cube is performed on the voxels and a dense mesh map is generated online. Extensive experiments are conducted on public datasets. The experimental results demonstrate that our method can achieve significant localization and online reconstruction performance improvements. The source code will be made public for the benefit of the robotic community.

Relative pose estimation from panoramic images using a hybrid neural network architecture

Camera-based relative pose estimation (RPE) localizes a mobile robot given a view at the current position and an image at a reference location. Matching the landmarks between views is critical to localization quality. Common challenges are appearance changes, for example due to differing illumination. Indirect RPE methods extract high-level features that provide invariance against appearance changes but neglect the remaining image data. This can lead to poor pose estimates in scenes with little detail. Direct RPE methods mitigate this issue by operating on the pixel level with only moderate preprocessing, but invariances have to be achieved by different means. We propose to attain illumination invariance for the direct RPE algorithm MinWarping by integrating it with a convolutional neural network for image preprocessing, creating a hybrid architecture. We optimize network parameters using a metric on RPE quality, backpropagating through MinWarping and the network. We focus on planar movement, panoramic images, and indoor scenes with varying illumination conditions; a novel dataset for this setup is recorded and used for analysis. Our method compares favourably against the previous best preprocessing method for MinWarping, edge filtering, and against a modern deep-learning-based indirect RPE pipeline. Analysis of the trained hybrid architecture indicates that neglecting landmarks in a direct RPE framework can improve estimation quality in scenes with occlusion and few details.

Cross-Spectral Navigation with Sensor Handover for Enhanced Proximity Operations with Uncooperative Space Objects

Close-proximity operations play a crucial role in emerging mission concepts, such as Active Debris Removal or small celestial bodies exploration. When approaching a non-cooperative target, the increased risk of collisions and reduced reliance on ground intervention necessitate autonomous on-board relative pose (position and attitude) estimation. Although navigation strategies relying on monocular cameras which operate in the visible (VIS) spectrum have been extensively studied and tested in flight for navigation applications, their accuracy is heavily related to the target’s illumination conditions, thus limiting their applicability range. The novelty of the paper is the introduction of a thermal-infrared (TIR) camera to complement the VIS one to mitigate the aforementioned issues. The primary goal of this work is to evaluate the enhancement in navigation accuracy and robustness by performing VIS-TIR data fusion within an Extended Kalman Filter (EKF) and to assess the performance of such navigation strategy in challenging illumination scenarios. The proposed navigation architecture is tightly coupled, leveraging correspondences between a known uncooperative target and feature points extracted from multispectral images. Furthermore, handover from one camera to the other is introduced to enable seamlessly operations across both spectra while prioritizing the most significant measurement sources. The pipeline is tested on Tango spacecraft synthetically generated VIS and TIR images. A performance assessment is carried out through numerical simulations considering different illumination conditions. Our results demonstrate that a combined VIS-TIR navigation strategy effectively enhances operational robustness and flexibility compared to traditional VIS-only navigation chains.

Str-L Pose: integrating point and structured line for relative pose estimation in dual graph

Str-L Pose: integrating point and structured line for relative pose estimation in dual graph

Unmanned aerial vehicle landing localization technology based on visual dual-channel uniqueness coding

Unmanned aerial vehicle (UAV) mainly uses GPS (Global Positioning System) for positioning, but it is difficult to locate in areas where GPS signal is missing or being disturbed. Therefore, the relative positioning technology based on visual method is also widely used in UAV landing, but due to the influence of cross wind and the motion of the landing plane, the traditional visual landing sign and positioning method is prone to lose the positioning target at the end stage of the landing. Therefore, this article proposes a landing sign based on double-layer unique coding, which can realize continuous positioning in the horizontal direction and accurate positioning of large-scale changes in the vertical direction. Firstly, real-time image acquisition and image processing are carried out through the airborne camera. Based on the flight altitude and encoding extraction effect of the UAV, the red or blue channel component is extracted, and then the image is preprocessed, edge detection, line detection, coding grid extraction, etc. As long as any 5 × 5 small squares of the whole pattern can be extracted, the perspective-n-point algorithm is used to obtain the relative pose estimation between the UAV and the landing sign. Experiments show that the landing sign and recognition algorithm effectively improve the UAV landing reliability.

Addressing the challenges of loop detection in agricultural environments

AbstractWhile visual Simultaneous Localization and Mapping systems are well studied and achieve impressive results in indoor and urban settings, natural, outdoor, and open‐field environments are much less explored and still present relevant research challenges. Visual navigation and local mapping have shown a relatively good performance in open‐field environments. However, globally consistent mapping and long‐term localization still depend on the robustness of loop detection and closure, for which the literature is scarce. In this work, we propose a novel method to pave the way towards robust loop detection in open fields, particularly in agricultural settings, based on local feature search and stereo geometric refinement, with a final stage of relative pose estimation. Our method consistently achieves good loop detections, with a median error of 15 cm. We aim to characterize open fields as a novel environment for loop detection, understanding the limitations and problems that arise when dealing with them. Code is available at: https://github.com/CIFASIS/StereoLoopDetector

A method of dense point cloud SLAM based on improved YOLOV8 and fused with ORB-SLAM3 to cope with dynamic environments

With the development of society and the advancement of technology, intelligent robots have been widely used in various fields. At the same time, Simultaneous Localization and Mapping (SLAM) technology is a key technology in the research field of intelligent robots. However, in dynamic environments, achieving accurate and robust visual SLAM remains a major challenge. In this paper, we propose a method based on improved YOLOv8 fused with ORB-SLAM3 to address dense point cloud SLAM in dynamic environments. Our proposed method successfully integrates real-time object detection and image segmentation technologies of YOLOv8 into the ORB-SLAM3 framework, achieving high-precision and robust visual SLAM in dynamic environments. In the YOLOv8 framework, we use a balanced convolution method, GSConv, instead of some traditional convolution layers (Conv), which balances accuracy with computational load. Based on the GSConv convolution method, we adopt a new feature fusion module, VoVGSCSP, to replace traditional C2f feature fusion modules, thereby improving the Neck structure of YOLOv8 and achieving a lightweight network model. We compare our proposed method with ORB-SLAM3 and some computer vision algorithms on the TUM dataset. Experimental data confirms that our method outperforms existing visual SLAM algorithms in dynamic environments. In fast-moving dynamic environments, the RMSE of absolute pose estimation of our method is 96.28% lower than that of ORB-SLAM3, and the RMSE of relative pose estimation is 51.57% lower than that of ORB-SLAM3. The experimental results demonstrate that our method significantly improves the accuracy of pose estimation in dynamic environments and greatly enhances the performance compared to ORB-SLAM3.

Globally Optimal Relative Pose and Scale Estimation from Only Image Correspondences with Known Vertical Direction

Installing multi-camera systems and inertial measurement units (IMUs) in self-driving cars, micro aerial vehicles, and robots is becoming increasingly common. An IMU provides the vertical direction, allowing coordinate frames to be aligned in a common direction. The degrees of freedom (DOFs) of the rotation matrix are reduced from 3 to 1. In this paper, we propose a globally optimal solver to calculate the relative poses and scale of generalized cameras with a known vertical direction. First, the cost function is established to minimize algebraic error in the least-squares sense. Then, the cost function is transformed into two polynomials with only two unknowns. Finally, the eigenvalue method is used to solve the relative rotation angle. The performance of the proposed method is verified on both simulated and KITTI datasets. Experiments show that our method is more accurate than the existing state-of-the-art solver in estimating the relative pose and scale. Compared to the best method among the comparison methods, the method proposed in this paper reduces the rotation matrix error, translation vector error, and scale error by 53%, 67%, and 90%, respectively.

Accelerating Globally Optimal Consensus Maximization in Geometric Vision.

Branch-and-bound-based consensus maximization stands out due to its important ability of retrieving the globally optimal solution to outlier-affected geometric problems. However, while the discovery of such solutions caries high scientific value, its application in practical scenarios is often prohibited by its computational complexity growing exponentially as a function of the dimensionality of the problem at hand. In this work, we convey a novel, general technique that allows us to branch over an n-1 dimensional space for an n-dimensional problem. The remaining degree of freedom can be solved globally optimally within each bound calculation by applying the efficient interval stabbing technique. While each individual bound derivation is harder to compute owing to the additional need for solving a sorting problem, the reduced number of intervals and tighter bounds in practice lead to a significant reduction in the overall number of required iterations. Besides an abstract introduction of the approach, we present applications to four fundamental geometric computer vision problems: camera resectioning, relative camera pose estimation, point set registration, and rotation and focal length estimation. Through our exhaustive tests, we demonstrate significant speed-up factors at times exceeding two orders of magnitude, thereby increasing the viability of globally optimal consensus maximizers in online application scenarios.

ConvMatch: Rethinking Network Design for Two-View Correspondence Learning.

Multilayer perceptron (MLP) has become the de facto backbone in two-view correspondence learning, for it can extract effective deep features from unordered correspondences individually. However, the problem of natively lacking context information limits its performance although many context-capturing modules are appended in the follow-up studies. In this paper, from a novel perspective, we design a correspondence learning network called ConvMatch that for the first time can leverage a convolutional neural network (CNN) as the backbone, inherently capable of context aggregation. Specifically, with the observation that sparse motion vectors and a dense motion field can be converted into each other with interpolating and sampling, we regularize the putative motion vectors by estimating the dense motion field implicitly, then rectify the errors caused by outliers in local areas with CNN, and finally obtain correct motion vectors from the rectified motion field. Moreover, we propose global information injection and bilateral convolution, to fit the overall spatial transformation better and accommodate the discontinuities of the motion field in case of large scene disparity. Extensive experiments reveal that ConvMatch consistently outperforms state-of-the-arts for relative pose estimation, homography estimation, and visual localization.

Learning‐based monocular visual‐inertial odometry with SE2(3) $S{E}_{2}(3)$‐EKF

AbstractLearning‐based visual odometry (VO) becomes popular as it achieves a remarkable performance without manually crafted image processing and burdensome calibration. Meanwhile, the inertial navigation can provide a localization solution to assist VO when the VO produces poor state estimation under challenging visual conditions. Therefore, the combination of learning‐based technique and classical state estimation method can further improve the performance of pose estimation. In this paper, we propose a learning‐based visual‐inertial odometry (VIO) algorithm, which consists of an end‐to‐end VO network and an ‐Extended Kalman Filter (EKF). The VO network mainly combines a convolutional neural network with a recurrent neural network, taking advantage of two consecutive monocular images to produce relative pose estimation with associated uncertainties. The ‐EKF, which is proved to overcome the inconsistency issues of VIO, propagates inertial measurement unit kinematics‐based states, and fuses relative measurements and uncertainties from the VO network in its update step. The extensive experimental results on the KITTI data set and the EuRoC data set demonstrate the superior performance of the proposed method compared to other related methods.

A branch-and-bound based globally optimal solution to 2D multi-robot relative pose estimation problems

The relative pose estimation problem is important for multi-robot systems when they execute a variety of tasks cooperatively, such as cooperative sensing, multi-robot search and rescue, formation control. The finding of reliable solutions to this problem is essential, however, algorithms that guarantee to provide globally optimal solutions are still in demand. In this paper, an alternative dimensionality reduced objective function is proposed for the planar ground multi-robot relative pose estimation problem, and a new branch-and-bound (BnB) based method is developed, through which initialization is not required and the globally optimal solution is guaranteed under arbitrary noise levels. We demonstrate that the globally optimal solutions obtained via the algorithm proposed in this paper are superior than that obtained by existing classic methods such as SE-Sync algorithm when noise levels are high. Moreover, the method proposed is simple and easy to implement.

Graph Context Transformation Learning for Progressive Correspondence Pruning

Most of existing correspondence pruning methods only concentrate on gathering the context information as much as possible while neglecting effective ways to utilize such information. In order to tackle this dilemma, in this paper we propose Graph Context Transformation Network (GCT-Net) enhancing context information to conduct consensus guidance for progressive correspondence pruning. Specifically, we design the Graph Context Enhance Transformer which first generates the graph network and then transforms it into multi-branch graph contexts. Moreover, it employs self-attention and cross-attention to magnify characteristics of each graph context for emphasizing the unique as well as shared essential information. To further apply the recalibrated graph contexts to the global domain, we propose the Graph Context Guidance Transformer. This module adopts a confident-based sampling strategy to temporarily screen high-confidence vertices for guiding accurate classification by searching global consensus between screened vertices and remaining ones. The extensive experimental results on outlier removal and relative pose estimation clearly demonstrate the superior performance of GCT-Net compared to state-of-the-art methods across outdoor and indoor datasets.

Transformer Network-Aided Relative Pose Estimation for Non-cooperative Spacecraft Using Vision Sensor

The objective of the proposed work is to perform monocular vision-based relative 6-DOF pose estimation of the non-cooperative target spacecraft relative to the chaser satellite in rendezvous operations. In this work, the convolutional neural network (CNN) is replaced by the high-resolution transformer network to predict the feature points of the target satellite. The self-attention mechanism inside the transformer provides the advantage of overcoming the inadequacies of the translation equivariance, 2D neighborhood awareness, and long-range dependencies in CNN. First, the 3D model of the target satellite is reconstructed using the inverse direct linear transform (IDLT) method. Then, the pose estimation pipeline is developed with a learning-based image-processing subsystem and geometric optimization of the pose solver. The image-processing subsystem performs target localization using CNN-based architecture. Then, the key points detection network performs regression to predict 2D key points using the transformer-based network. Afterward, the predicted key points based on their confidence scores are projected onto the corresponding 3D points, and the pose value is computed using the efficient perspective-n-point method. The pose is refined using the non-linear iterative Gauss–Newton method. The proposed architecture is trained and tested on the spacecraft pose estimation dataset and it shows superior accuracy both in translation and rotation values. The architecture has shown robustness against the drastically changing clutter background and light conditions in the space images due to the self-attention mechanism. Moreover, this method consumes less computation resources by using fewer floating-point operations and trainable parameters with low input image resolution.

A self-supervised monocular odometry with visual-inertial and depth representations

Background:Self-supervised monocular visual-odometry (VO) methods have demonstrated impressive results in relative pose estimation, but they encounter challenges when visual information is unreliable. To address this, we introduce a self-supervised visual-inertial odometry (VIO) system. Methods:Our method adopts a trinocular assumption, where three consecutive frames are treated as if captured simultaneously by three cameras. This approach expands the field of view and enhances robustness in visual constraints for 3D structures. Additionally, we incorporate Inertial Measurement Unit (IMU) data into the system. This integration, achieved by an attention-based fusion strategy, allows our visual-inertial odometry (VIO) system to adaptively select the most reliable representations for accurate relative pose estimation. Results:Experiments on public and self-collected datasets show that the proposed method is state-of-the-art in visual odometry. Conclusion:The encouraging results show that it is feasible to apply this learning-based VIO system in autonomous cars and robots.

Circular-Feature-Based Pose Estimation of Noncooperative Satellite Using Time-of-Flight Sensor

This paper develops an end-to-end circular-feature-based pose estimation using the time-of-flight (TOF) camera for capturing serviced satellite. First, the docking ring is chosen as the recognized object for pose measurement. Second, a new ellipse detection method is proposed to detect the docking ring. Third, the circular feature is used to estimate the six-degrees-of-freedom pose with the aid of a reflective block. Combined with the range at each pixel location from TOF, the pose duality can be eliminated, and the roll angle is recovered through the geometric constraint. Ground physical experiments validate the effectiveness and efficiency of the proposed method. The on-orbit application demonstrates that the proposed algorithm can accurately and robustly estimate the relative pose of the noncooperative satellite, providing reliable navigation information for the chaser spacecraft with the relative pose estimation errors less than 2 deg and 5 cm.

Ca2Lib: Simple and Accurate LiDAR-RGB Calibration Using Small Common Markers

Modern visual perception techniques often rely on multiple heterogeneous sensors to achieve accurate and robust estimates. Knowledge of their relative positions is a mandatory prerequisite to accomplish sensor fusion. Typically, this result is obtained through a calibration procedure that correlates the sensors’ measurements. In this context, we focus on LiDAR and RGB sensors that exhibit complementary capabilities. Given the sparsity of LiDAR measurements, current state-of-the-art calibration techniques often rely on complex or large calibration targets to resolve the relative pose estimation. As such, the geometric properties of the targets may hinder the calibration procedure in those cases where an ad hoc environment cannot be guaranteed. This paper addresses the problem of LiDAR-RGB calibration using common calibration patterns (i.e., A3 chessboard) with minimal human intervention. Our approach exploits the flatness of the target to find associations between the sensors’ measurements, leading to robust features and retrieval of the solution through nonlinear optimization. The results of quantitative and comparative experiments with other state-of-the-art approaches show that our simple schema performs on par or better than existing methods that rely on complex calibration targets.

A fast, lightweight deep learning vision pipeline for autonomous UAV landing support with added robustness

Despite massive development in aerial robotics, precise and autonomous landing in various conditions is still challenging. This process is affected by many factors, such as terrain shape, weather conditions, and the presence of obstacles. This paper describes a deep learning-accelerated image processing pipeline for accurate detection and relative pose estimation of the UAV with respect to the landing pad. Moreover, the system provides increased safety and robustness by implementing human presence detection and error estimation for both landing target detection and pose computation. Human presence and landing pad location are performed by estimating the presence probability via segmentation. This is followed by the landing pad keypoints’ location regression algorithm, which, in addition to coordinates, provides the uncertainty of presence for each defined landing pad landmark. To perform the aforementioned tasks, a set of lightweight neural network models was selected and evaluated. The resulting measurements of the system’s performance and accuracy are presented for each component individually and for the whole processing pipeline. The measurements are performed using onboard embedded UAV hardware and confirm that the method can provide accurate, low-latency feedback information for safe landing support.