Pose Estimation Error Research Articles

A LiDAR SLAM Algorithm Considering Dynamic Extraction of Feature Points in Underground Coal Mine

Abstract. To address the absence of GNSS signals in underground coal mines and the susceptibility of mainstream LiDAR SLAM to degradation due to insufficient feature constraints, this paper proposes a tightly-coupled SLAM algorithm incorporating LiDAR and IMU for use in such environments. The paper initially introduces a dynamic feature point extraction strategy, where the number of corner feature points can be dynamically adjusted based on the occurrence of degradation in the underground coal mine environment. This approach constructs a constraint matrix with rich and robust feature information, enhancing pose estimation accuracy in environments with inadequate feature constraints, mitigating degradation effects, and reducing global cumulative error. Subsequently, the consistent building of the underground coal mine environment is achieved through back-end factor graph optimization. Finally, to validate the effectiveness of the method, experimental analysis is conducted in an underground coal mine. The results demonstrate that LeGO_LOAM exhibits poor robustness in underground coal mines and fails to construct a globally consistent pose estimation and map. Conversely, the pose estimation error of the proposed method in this paper is 50.93% lower in the plane direction and 42.13% lower in the elevation direction compared to LIO_SAM. This underscores the method's significance as crucial technical support for the intelligent perception and positioning of robots in underground coal mines.

Enhancing Inter-AUV Perception: Adaptive 6-DOF Pose Estimation with Synthetic Images for AUV Swarm Sensing

The capabilities of AUV mutual perception and localization are crucial for the development of AUV swarm systems. We propose the AUV6D model, a synthetic image-based approach to enhance inter-AUV perception through 6D pose estimation. Due to the challenge of acquiring accurate 6D pose data, a dataset of simulated underwater images with precise pose labels was generated using Unity3D. Mask-CycleGAN technology was introduced to transform these simulated images into realistic synthetic images, addressing the scarcity of available underwater data. Furthermore, the Color Intermediate Domain Mapping strategy is proposed to ensure alignment across different image styles at pixel and feature levels, enhancing the adaptability of the pose estimation model. Additionally, the Salient Keypoint Vector Voting Mechanism was developed to improve the accuracy and robustness of underwater pose estimation, enabling precise localization even in the presence of occlusions. The experimental results demonstrated that our AUV6D model achieved millimeter-level localization precision and pose estimation errors within five degrees, showing exceptional performance in complex underwater environments. Navigation experiments with two AUVs further verified the model’s reliability for mutual 6D pose estimation. This research provides substantial technical support for more complex and precise collaborative operations for AUV swarms in the future.

Fixed-Wing UAV Pose Estimation Using a Self-Organizing Map and Deep Learning

In many Unmanned Aerial Vehicle (UAV) operations, accurately estimating the UAV’s position and orientation over time is crucial for controlling its trajectory. This is especially important when considering the landing maneuver, where a ground-based camera system can estimate the UAV’s 3D position and orientation. A Red, Green, and Blue (RGB) ground-based monocular approach can be used for this purpose, allowing for more complex algorithms and higher processing power. The proposed method uses a hybrid Artificial Neural Network (ANN) model, incorporating a Kohonen Neural Network (KNN) or Self-Organizing Map (SOM) to identify feature points representing a cluster obtained from a binary image containing the UAV. A Deep Neural Network (DNN) architecture is then used to estimate the actual UAV pose based on a single frame, including translation and orientation. Utilizing the UAV Computer-Aided Design (CAD) model, the network structure can be easily trained using a synthetic dataset, and then fine-tuning can be done to perform transfer learning to deal with real data. The experimental results demonstrate that the system achieves high accuracy, characterized by low errors in UAV pose estimation. This implementation paves the way for automating operational tasks like autonomous landing, which is especially hazardous and prone to failure.

Robust visual SLAM algorithm based on target detection and clustering in dynamic scenarios.

We propose a visual Simultaneous Localization and Mapping (SLAM) algorithm that integrates target detection and clustering techniques in dynamic scenarios to address the vulnerability of traditional SLAM algorithms to moving targets. The proposed algorithm integrates the target detection module into the front end of the SLAM and identifies dynamic objects within the visual range by improving the YOLOv5. Feature points associated with the dynamic objects are disregarded, and only those that correspond to static targets are utilized for frame-to-frame matching. This approach effectively addresses the camera pose estimation in dynamic environments, enhances system positioning accuracy, and optimizes the visual SLAM performance. Experiments on the TUM public dataset and comparison with the traditional ORB-SLAM3 algorithm and DS-SLAM algorithm validate that the proposed visual SLAM algorithm demonstrates an average improvement of 85.70 and 30.92% in positioning accuracy in highly dynamic scenarios. In comparison to the DynaSLAM system using MASK-RCNN, our system exhibits superior real-time performance while maintaining a comparable ATE index. These results highlight that our pro-posed SLAM algorithm effectively reduces pose estimation errors, enhances positioning accuracy, and showcases enhanced robustness compared to conventional visual SLAM algorithms.

Robust Visual SLAM in Dynamic Environment Based on Motion Detection and Segmentation

Abstract In this study, we propose a robust and accurate SLAM method for dynamic environments. Our approach combines sparse optical flow with epipolar geometric constraints to detect motion, determining whether a priori dynamic objects are moving. By integrating semantic segmentation with this motion detection, we can effectively remove dynamic keypoints, eliminating the influence of dynamic objects. This dynamic object removal technique is integrated into ORB-SLAM2, en-hancing its robustness and accuracy for localization and mapping. Experimental results on the TUM dataset demonstrate that our proposed system significantly reduces pose estimation error compared to ORB-SLAM2. Specifically, the RMSE and standard deviation (S.D.) of ORB-SLAM2 are reduced by up to 97.78% and 97.91%, respectively, in highly dynamic se-quences, markedly improving robustness in dynamic environments. Furthermore, compared to other similar SLAM methods, our method reduces RMSE and S.D. by up to 69.26% and 73.03%, respectively. Dense semantic maps generated by our method also closely align with the ground truth.

Multi-mode vehicle pose estimation under different GNSS conditions

The integrated navigation system combining the global navigation satellite system (GNSS) and inertial navigation system (INS) is a crucial method for pose estimation in the field of autonomous driving technologies. Nevertheless, the accuracy of pose estimation is severely compromised when GNSS signals are obstructed or disrupted. To address this issue, this study introduces a multi-mode pose estimation framework designed to ensure accurate pose estimation even under unstable GNSS conditions. By integrating vehicle kinematics model that considers steering characteristics (VKMSC) and the convolutional neural network-long short-term memory (CNN-LSTM) neural network (NN) model into various estimation modes, the framework enhances the robustness of the integrated navigation system against signal interference. The system dynamically selects the optimal estimation strategy based on the degree of GNSS signal disruption. The proposed method has been validated through real-vehicle experiments, which demonstrate its efficacy in providing precise pose estimation across a spectrum of interference scenarios. Under the multipath and non-line-of-sight (MP/NLOS) mode, compared to the integrated navigation system and the fusion of traditional vehicle kinematic models, the proposed method improved positional estimation accuracy by 61.8 % and 19.7 %, respectively. In GNSS outage mode, the proposed method increased the estimation accuracy by 36.5 % and 12.0 %, respectively, compared to the INS navigation system assisted by the VKMSC and CNN-LSTM network model. The proposed method effectively reduces pose estimation errors in the integrated navigation system during interference and suppresses data fluctuations, thereby enhancing the system's precision and robustness.

A Learnable Viewpoint Evolution Method for Accurate Pose Estimation of Complex Assembled Product

Balancing adaptability, reliability, and accuracy in vision technology has always been a major bottleneck limiting its application in appearance assurance for complex objects in high-end equipment production. Data-driven deep learning shows robustness to feature diversity but is limited by interpretability and accuracy. The traditional vision scheme is reliable and can achieve high accuracy, but its adaptability is insufficient. The deeper reason is the lack of appropriate architecture and integration strategies between the learning paradigm and empirical design. To this end, a learnable viewpoint evolution algorithm for high-accuracy pose estimation of complex assembled products under free view is proposed. To alleviate the balance problem of exploration and optimization in estimation, shape-constrained virtual–real matching, evolvable feasible region, and specialized population migration and reproduction strategies are designed. Furthermore, a learnable evolution control mechanism is proposed, which integrates a guided model based on experience and is cyclic-trained with automatically generated effective trajectories to improve the evolution process. Compared to the 1.69°,55.67 mm of the state-of-the-art data-driven method and the 1.28°,77.67 mm of the classic strategy combination, the pose estimation error of complex assembled product in this study is 0.23°,23.71 mm, which proves the effectiveness of the proposed method. Meanwhile, through in-depth exploration, the robustness, parameter sensitivity, and adaptability to the virtual–real appearance variations are sequentially verified.

PETS-Nets: Joint Pose Estimation and Tissue Segmentation of Fetal Brains Using Anatomy-Guided Networks.

Fetal Magnetic Resonance Imaging (MRI) is challenged by fetal movements and maternal breathing. Although fast MRI sequences allow artifact free acquisition of individual 2D slices, motion frequently occurs in the acquisition of spatially adjacent slices. Motion correction for each slice is thus critical for the reconstruction of 3D fetal brain MRI. In this paper, we propose a novel multi-task learning framework that adopts a coarse-to-fine strategy to jointly learn the pose estimation parameters for motion correction and tissue segmentation map of each slice in fetal MRI. Particularly, we design a regression-based segmentation loss as a deep supervision to learn anatomically more meaningful features for pose estimation and segmentation. In the coarse stage, a U-Net-like network learns the features shared for both tasks. In the refinement stage, to fully utilize the anatomical information, signed distance maps constructed from the coarse segmentation are introduced to guide the feature learning for both tasks. Finally, iterative incorporation of the signed distance maps further improves the performance of both regression and segmentation progressively. Experimental results of cross-validation across two different fetal datasets acquired with different scanners and imaging protocols demonstrate the effectiveness of the proposed method in reducing the pose estimation error and obtaining superior tissue segmentation results simultaneously, compared with state-of-the-art methods.

Circular-Feature-Based Pose Estimation of Noncooperative Satellite Using Time-of-Flight Sensor

This paper develops an end-to-end circular-feature-based pose estimation using the time-of-flight (TOF) camera for capturing serviced satellite. First, the docking ring is chosen as the recognized object for pose measurement. Second, a new ellipse detection method is proposed to detect the docking ring. Third, the circular feature is used to estimate the six-degrees-of-freedom pose with the aid of a reflective block. Combined with the range at each pixel location from TOF, the pose duality can be eliminated, and the roll angle is recovered through the geometric constraint. Ground physical experiments validate the effectiveness and efficiency of the proposed method. The on-orbit application demonstrates that the proposed algorithm can accurately and robustly estimate the relative pose of the noncooperative satellite, providing reliable navigation information for the chaser spacecraft with the relative pose estimation errors less than 2 deg and 5 cm.

RC-SLAM: Road Constrained Stereo Visual SLAM System Based on Graph Optimization.

Intelligent vehicles are constrained by road, resulting in a disparity between the assumed six degrees of freedom (DoF) motion within the Visual Simultaneous Localization and Mapping (SLAM) system and the approximate planar motion of vehicles in local areas, inevitably causing additional pose estimation errors. To address this problem, a stereo Visual SLAM system with road constraints based on graph optimization is proposed, called RC-SLAM. Addressing the challenge of representing roads parametrically, a novel method is proposed to approximate local roads as discrete planes and extract parameters of local road planes (LRPs) using homography. Unlike conventional methods, constraints between the vehicle and LRPs are established, effectively mitigating errors arising from assumed six DoF motion in the system. Furthermore, to avoid the impact of depth uncertainty in road features, epipolar constraints are employed to estimate rotation by minimizing the distance between road feature points and epipolar lines, robust rotation estimation is achieved despite depth uncertainties. Notably, a distinctive nonlinear optimization model based on graph optimization is presented, jointly optimizing the poses of vehicle trajectories, LPRs, and map points. The experiments on two datasets demonstrate that the proposed system achieved more accurate estimations of vehicle trajectories by introducing constraints between the vehicle and LRPs. The experiments on a real-world dataset further validate the effectiveness of the proposed system.

A Measure of Semantic Class Difference of Point Reprojection Pairs in Camera Pose Estimation

In this paper, we propose a new measure that evaluates the semantic errors of camera poses in visual odometry (VO) and visual simultaneous localization and mapping (VSLAM). Traditionally, VO/VSLAM methods have used photometric images to estimate camera poses, but they suffer from varying illumination and viewpoint changes. Thus, methods using semantic images have been an alternative to increase consistency, as semantic information has shown its robustness even in hostile environments. Our measure compares semantic classes of map point reprojection pairs between images to improve the camera pose estimation accuracy in VO/VSLAM. To evaluate the difference between semantic classes, we adopt the normalized information distance (NID) from information theory. Furthermore, we suggest a weight parameter to balance the existing error of VO/VSLAM with the semantic error introduced by our approach. Our experimental results, obtained from the VKITTI and KITTI benchmark datasets, show that the proposed semantic error measure reduces both the relative pose error (RPE) and absolute trajectory error (ATE) of camera pose estimation compared to the existing photometric image-based errors of indirect and direct VO/VSLAM.

Pose estimation via structure-depth information from monocular endoscopy images sequence.

Image-based endoscopy pose estimation has been shown to significantly improve the visualization and accuracy of minimally invasive surgery (MIS). This paper proposes a method for pose estimation based on structure-depth information from a monocular endoscopy image sequence. Firstly, the initial frame location is constrained using the image structure difference (ISD) network. Secondly, endoscopy image depth information is used to estimate the pose of sequence frames. Finally, adaptive boundary constraints are used to optimize continuous frame endoscopy pose estimation, resulting in more accurate intraoperative endoscopy pose estimation. Evaluations were conducted on publicly available datasets, with the pose estimation error in bronchoscopy and colonoscopy datasets reaching 1.43 mm and 3.64 mm, respectively. These results meet the real-time requirements of various scenarios, demonstrating the capability of this method to generate reliable pose estimation results for endoscopy images and its meaningful applications in clinical practice. This method enables accurate localization of endoscopy images during surgery, assisting physicians in performing safer and more effective procedures.

PosturePose: Optimized Posture Analysis for Semi-Supervised Monocular 3D Human Pose Estimation.

One motivation for studying semi-supervised techniques for human pose estimation is to compensate for the lack of variety in curated 3D human pose datasets by combining labeled 3D pose data with readily available unlabeled video data-effectively, leveraging the annotations of the former and the rich variety of the latter to train more robust pose estimators. In this paper, we propose a novel, fully differentiable posture consistency loss that is unaffected by camera orientation and improves monocular human pose estimators trained with limited labeled 3D pose data. Our semi-supervised monocular 3D pose framework combines biomechanical pose regularization with a multi-view posture (and pose) consistency objective function. We show that posture optimization was effective at decreasing pose estimation errors when applied to a 2D-3D lifting network (VPose3D) and two well-studied datasets (H36M and 3DHP). Specifically, the proposed semi-supervised framework with multi-view posture and pose loss lowered the mean per-joint position error (MPJPE) of leading semi-supervised methods by up to 15% (-7.6 mm) when camera parameters of unlabeled poses were provided. Without camera parameters, our semi-supervised framework with posture loss improved semi-supervised state-of-the-art methods by 17% (-15.6 mm decrease in MPJPE). Overall, our pose models compete favorably with other high-performing pose models trained under similar conditions with limited labeled data.

Deep Learning-Based Oyster Packaging System

With the deepening understanding of the nutritional value of oysters by consumers, oysters as high-quality seafood are gradually entering the market. Raw edible oyster production lines mainly rely on manual sorting and packaging, which hinders the improvement of oyster packaging efficiency and quality, and it is easy to cause secondary oyster pollution and cross-contamination, which results in the waste of oysters. To enhance the production efficiency, technical level, and hygiene safety of the raw aquatic products production line, this study proposes and constructs a deep learning-based oyster packaging system. The system achieves intelligence and automation of the oyster packaging production line by integrating the deep learning algorithm, machine vision technology, and mechanical arm control technology. The oyster visual perception model is established by deep learning object detection techniques to realize fast and real-time detection of oysters. Using a simple online real-time tracking (SORT) algorithm, the grasping position of the oyster can be predicted, which enables dynamic grasping. Utilizing mechanical arm control technology, an automatic oyster packaging production line was designed and constructed to realize the automated grasping and packaging of raw edible oysters, which improves the efficiency and quality of oyster packaging. System tests showed that the absolute error in oyster pose estimation was less than 7 mm, which allowed the mechanical claw to consistently grasp and transport oysters. The static grasping and packing of a single oyster took about 7.8 s, and the success rate of grasping was 94.44%. The success rate of grasping under different transportation speeds was above 68%.

A multi-sensor fusion framework with tight coupling for precise positioning and optimization

In the dynamic landscape of artificial intelligence and robotics, the pursuit of accurate positioning in mobile robots has intensified. This research addresses the limitations of single-sensor SLAM (Simultaneous Localization and Mapping) techniques in complex settings by harnessing the collective strengths of LiDAR (Light Detection And Ranging), Camera, IMU (Inertial Measurement Unit), and GNSS (Global Navigation Satellite System) sensors. The proposed multi-sensor tightly-coupled SLAM framework is an integration of point-line feature-based laser–visual–inertial odometry, visual–laser fusion loop closure detection, and factor graph-based back-end optimization. Within the visual–inertial subsystem, an advanced LSD (Line Segment Detector) feature extraction strategy is introduced, incorporating point-line fusion to enhance visual line features. Additionally, the laser point cloud is projected onto the camera coordinate system, establishing depth associations with visual attributes. Strengthening the robustness of the visual–inertial subsystem in low-texture environments, camera poses undergo optimization through a sliding-window bundle adjustment method. In the laser-inertial subsystem, IMU preintegration mitigates laser point cloud distortion. Extracting edge and plane features, coupled with frame-to-local-map matching, enhances matching efficiency while streamlining computational intricacies. This amalgamation forms the basis of the laser–visual–inertial odometry fusion system. To overcome the limitations of standalone visual and laser-based loop closure detection, a dual-loop closure method utilizing visual–laser fusion is proposed. Leveraging the DBoW2 bag-of-words model, complemented by temporal–spatial consistency checks, enhances detection efficiency and accuracy. The integration of GNSS factors imparts global constraints for expansive outdoor scenarios. Employing factor graph-based back-end optimization, the refinement of laser–visual–inertial odometry factors, visual–inertial odometry factors, IMU preintegration factors, loop closure factors, and GNSS factors culminates in precise global pose estimation and high-fidelity point cloud maps. Through rigorous evaluation of the M2DGR dataset and a mobile robot platform, the proposed methodology emerges as an exemplar of performance, showcasing superiority over the state-of-the-art LIO-SAM technique. Achieving a reduction of 2.86m and 3.23m in the root mean square error of absolute pose estimation across divergent environments, this approach exhibits remarkable efficacy in outdoor scenarios, thereby elevating the precision and resilience of SLAM algorithms for mobile robots.

JoyPose: Jointly learning evolutionary data augmentation and anatomy-aware global–local representation for 3D human pose estimation

Video-based 3D human pose estimation is an important yet challenging task for many human-involved pattern recognition systems. Existing deep learning-based 3D human pose estimation methods are faced with the problems of lacking large-scale training data and lacking effective solutions to represent the complicated human body structure. To this end, this paper proposes a jointly learning framework entitled JoyPose that simultaneously leverages both human pose data augmentation and human pose estimation. In particular, JoyPose consists of an evolutionary data augmentation module and an anatomy-aware global–local pose feature representation module for 3D human pose estimation. The evolution for data augmentation is guided by a reinforcement learning strategy in a probabilistic way according to pose estimation loss. The anatomy-aware global–local pose feature representation module separately captures global features and local features according to anatomical and kinematic patterns observed from pose estimation errors across different human joints. The performance of the final human pose estimation is leveraged by both data augmentation and anatomy-aware global–local feature representation. Extensive experiments on three real-world datasets demonstrate the superiority and robustness against state-of-the-art methods.

A 3D pose estimation framework for preterm infants hospitalized in the Neonatal Unit

AbstractInfant pose estimation is crucial in different clinical applications, including preterm automatic general movements assessment. Recent infant pose estimation methods are limited by a lack of real clinical data and are mainly focused on 2D detection. We introduce a stereoscopic system for infants’ 3D pose estimation, based on fine-tuning state-of-the-art 2D human pose estimation networks on a large, real, and manually annotated dataset of infants’ images. Our dataset contains over 88k images, collected from 175 videos from 53 premature infants born <33 weeks of gestational age (GA), acquired within the Neonatology department of the Centre Hospitalier Universitaire de Saint Etienne, France, between 32 and 41 weeks of GA. This framework significantly reduced the pose estimation error compared to existing 2D infant pose estimation networks. It achieved a mean error of 1.72 cm on 18000 stereoscopic images in the 3D pose estimation task. This framework is the first 3D pose estimation tool dedicated to preterm infants hospitalized in the Neonatal Unit that does not depend on any visual markers or infrared cameras.

Robot Programming from Fish Demonstrations.

Fish are capable of learning complex relations found in their surroundings, and harnessing their knowledge may help to improve the autonomy and adaptability of robots. Here, we propose a novel learning from demonstration framework to generate fish-inspired robot control programs with as little human intervention as possible. The framework consists of six core modules: (1) task demonstration, (2) fish tracking, (3) analysis of fish trajectories, (4) acquisition of robot training data, (5) generating a perception-action controller, and (6) performance evaluation. We first describe these modules and highlight the key challenges pertaining to each one. We then present an artificial neural network for automatic fish tracking. The network detected fish successfully in 85% of the frames, and in these frames, its average pose estimation error was less than 0.04 body lengths. We finally demonstrate how the framework works through a case study focusing on a cue-based navigation task. Two low-level perception-action controllers were generated through the framework. Their performance was measured using two-dimensional particle simulations and compared against two benchmark controllers, which were programmed manually by a researcher. The fish-inspired controllers had excellent performance when the robot was started from the initial conditions used in fish demonstrations (>96% success rate), outperforming the benchmark controllers by at least 3%. One of them also had an excellent generalisation performance when the robot was started from random initial conditions covering a wider range of starting positions and heading angles (>98% success rate), again outperforming the benchmark controllers by 12%. The positive results highlight the utility of the framework as a research tool to form biological hypotheses on how fish navigate in complex environments and design better robot controllers on the basis of biological findings.

Fixed-Wing Unmanned Aerial Vehicle 3D-Model-Based Tracking for Autonomous Landing

The vast increase in the available computational capability has allowed the application of Particle-Filter (PF)-based approaches for monocular 3D-model-based tracking. These filters depend on the computation of a likelihood function that is usually unavailable and can be approximated using a similarity metric. We can use temporal filtering techniques between filter iterations to achieve better results when dealing with this suboptimal approximation, which is particularly important when dealing with the Unmanned Aerial Vehicle (UAV) model symmetry. The similarity metric evaluation time is another critical concern since we usually want a real-time implementation. We explored, tested, and compared with the same dataset two different types of PFs, (i) an Unscented Bingham Filter (UBiF) and (ii) an Unscented Bingham–Gauss Filter (UBiGaF), using pose optimization in both implementations. Using optimization steps between iterations increases the convergence capability of the filter and decreases the obtained error. A new tree-based similarity metric approach is also explored based on the Distance Transform (DT), allowing a faster evaluation of the possibilities without losing accuracy. The results showed that the obtained pose estimation error is compatible with the automatic landing requirements.

Hierarchical Estimation-Based LiDAR Odometry With Scan-to-Map Matching and Fixed-Lag Smoothing

LiDAR odometry (LO) has gained popularity in recent years due to accurate depth measurement and robustness to illumination. However, existing solutions to LO mainly rely on scan-to-map matching in which only current pose is optimized. To further reduce the accumulated error of pose estimation, the fixed-lag smoothing that optimizes fixed-size poses simultaneously by matching corresponding point features of multiple frames in visual odometry provides an important reference. The integration of fixed-lag smoothing with LO still needs further exploration. In this paper, a general fixed-lag smoothing module is proposed, which can be appended to existing LO framework to improve the consistency of trajectory. Also, a fast scan-to-map matching module based on sparse features and efficient feature management is developed to guarantee the real-time performance. Besides, the feature-centric feature management strategy is adopted in both scan-to-map matching and fixed-lag smoothing modules, which makes the proposed LO more efficient. On this basis, a hierarchical estimation-based LiDAR odometry is presented, where low-level scan-to-map matching estimates pose of each frame by aligning associated features in the frame and corresponding surrounding map with high efficiency, and high-level fixed-lag smoothing further optimizes keyframe poses in a sliding window by matching associated features among multiple frames with high accuracy. As a result, a fast and accurate pose estimation is achieved, which is verified by experiments on the KITTI dataset, Newer College dataset, and an actual outdoor scenario.