Odometry Estimation Research Articles

Stereo Orientation Prior for UAV Robust and Accurate Visual Odometry

This article presents a novel outlier rejection approach for feature-based visual odometry. The proposed approach is based on an empirical observation that shows that some 2D–3D correspondences with very low reprojection error can cause a high error in pose estimation. This work exploits such observations for odometry when a stereo camera is available. We argue that an explicit pose error measure is desired over that of implicit reprojection – whenever the former is possible – to classify correspondences into inliers versus outliers for the robust long-term odometry. To explicitly measure the plausible pose error, we derive bounds on the individual pose parameters with the help of the known orientation of stereo cameras. In this process, we formulate our bounds using the sum-of-square polynomials, which allow us to test whether a given correspondence satisfies any solution within the expected bounds. If the correspondence does not satisfy bounds for any parameter, it is considered to be an outlier. We implemented and tested the proposed method for the unmanned aerial vehicle (UAV) indoor navigation. The experiments from both benchmark evaluations (EuRoC and KITTI) and UAV onboard tests indicate that the inlier group refined by the proposed method significantly improves odometry estimation compared to the traditional outlier rejection methods. In fact, the proposed method performs as accurately as inertial measurement unit (IMU) aided methods in the state of the art. ROS project page (open-source code with demo video) is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/arclab-hku/SOPVO</uri>

Generalized LOAM: LiDAR Odometry Estimation With Trainable Local Geometric Features

This letter presents a LiDAR odometry estimation framework called Generalized LOAM. Our proposed method is generalized in that it can seamlessly fuse various local geometric shapes around points to improve the position estimation accuracy compared to the conventional LiDAR odometry and mapping (LOAM) method. To utilize continuous geometric features for LiDAR odometry estimation, we incorporate tiny neural networks into a generalized iterative closest point (GICP) algorithm. These neural networks improve the data association metric and the matching cost function using local geometric features. Experiments with the KITTI benchmark demonstrate that our proposed method reduces relative trajectory errors compared to the other LiDAR odometry estimation methods.

Assessment of CNN-Based Models for Odometry Estimation Methods with LiDAR

The problem of simultaneous localization and mapping (SLAM) in mobile robotics currently remains a crucial issue to ensure the safety of autonomous vehicles’ navigation. One approach addressing the SLAM problem and odometry estimation has been through perception sensors, leading to V-SLAM and visual odometry solutions. Furthermore, for these purposes, computer vision approaches are quite widespread, but LiDAR is a more reliable technology for obstacles detection and its application could be broadened. However, in most cases, definitive results are not achieved, or they suffer from a high computational load that limits their operation in real time. Deep Learning techniques have proven their validity in many different fields, one of them being the perception of the environment of autonomous vehicles. This paper proposes an approach to address the estimation of the ego-vehicle positioning from 3D LiDAR data, taking advantage of the capabilities of a system based on Machine Learning models, analyzing possible limitations. Models have been used with two real datasets. Results provide the conclusion that CNN-based odometry could guarantee local consistency, whereas it loses accuracy due to cumulative errors in the evaluation of the global trajectory, so global consistency is not guaranteed.

Information-Theoretic Odometry Learning

In this paper, we propose a unified information theoretic framework for learning-motivated methods aimed at odometry estimation, a crucial component of many robotics and vision tasks such as navigation and virtual reality where relative camera poses are required in real time. We formulate this problem as optimizing a variational information bottleneck objective function, which eliminates pose-irrelevant information from the latent representation. The proposed framework provides an elegant tool for performance evaluation and understanding in information-theoretic language. Specifically, we bound the generalization errors of the deep information bottleneck framework and the predictability of the latent representation. These provide not only a performance guarantee but also practical guidance for model design, sample collection, and sensor selection. Furthermore, the stochastic latent representation provides a natural uncertainty measure without the needs for extra structures or computations. Experiments on two well-known odometry datasets demonstrate the effectiveness of our method.

PSE-Match: A Viewpoint-Free Place Recognition Method With Parallel Semantic Embedding

Accurate localization on the autonomous driving cars is essential for autonomy and driving safety, especially for complex urban streets and search-and-rescue subterranean environments where high-accurate GPS is not available. However current odometry estimation may introduce the drifting problems in long-term navigation without robust global localization. The main challenges involve scene divergence under the interference of dynamic environments and effective perception of observation and object layout variance from different viewpoints. To tackle these challenges, we present PSE-Match, a viewpoint-free place recognition method based on parallel semantic analysis of isolated semantic attributes from 3D point-cloud models. Compared with the original point cloud, the observed variance of semantic attributes is smaller. PSE-Match incorporates a divergence place learning network to capture different semantic attributes parallelly through the spherical harmonics domain. Using both existing benchmark datasets and two in-field collected datasets, our experiments show that the proposed method achieves above 70% average recall with top one retrieval and above 95% average recall with top ten retrieval cases. And PSE-Match has also demonstrated an obvious generalization ability with limited training dataset.

Lightweight spatial attentive network for vehicular visual odometry estimation in urban environments

Lightweight spatial attentive network for vehicular visual odometry estimation in urban environments

Robust Self-Supervised LiDAR Odometry Via Representative Structure Discovery and 3D Inherent Error Modeling

The correct ego-motion estimation basically relies on the understanding of correspondences between adjacent LiDAR scans. However, given the complex scenarios and the low-resolution LiDAR, finding reliable structures for identifying correspondences can be challenging. In this letter, we delve into structure reliability for accurate self-supervised ego-motion estimation and aim to alleviate the influence of unreliable structures in training, inference and mapping phases. We improve the self-supervised LiDAR odometry substantially from three aspects: 1) A two-stage odometry estimation network is developed, where we obtain the ego-motion by estimating a set of sub-region transformations and averaging them with a motion voting mechanism, to encourage the network focusing on representative structures. 2) The inherent alignment errors, which cannot be eliminated via ego-motion optimization, are down-weighted in losses based on the 3D point covariance estimations. 3) The discovered representative structures and learned point covariances are incorporated in the mapping module to improve the robustness of map construction. Our two-frame odometry outperforms the previous state of the arts by 16%/12% in terms of translational/rotational errors on the KITTI dataset and performs consistently well on the Apollo-Southbay datasets. We can even rival the fully supervised counterparts with our mapping module and more unlabeled training data.

An online semantic mapping system for extending and enhancing visual SLAM

We present a real-time semantic mapping approach for mobile vision systems with a 2D to 3D object detection pipeline and rapid data association for generated landmarks. Besides the semantic map enrichment the associated detections are further introduced as semantic constraints into a simultaneous localization and mapping (SLAM) system for pose correction purposes. This way, we are able generate additional meaningful information that allows to achieve higher-level tasks, while simultaneously leveraging the view-invariance of object detections to improve the accuracy and the robustness of the odometry estimation. We propose tracklets of locally associated object observations to handle ambiguous and false predictions and an uncertainty-based greedy association scheme for an accelerated processing time. Our system reaches real-time capabilities with an average iteration duration of 65 ms and is able to improve the pose estimation of a state-of-the-art SLAM by up to 68% on a public dataset. Additionally, we implemented our approach as a modular ROS package that makes it straightforward for integration in arbitrary graph-based SLAM methods.

Modular Multi-Domain Aware Autonomous Surface Vehicle for Inspection

A growing interest in ocean exploration for scientific and commercial research has been shown, mainly due to the technological developments for maritime and offshore industries. The use of Autonomous surface vehicles (ASV) have a promising role to revolutionize the transportation, monitorization, operation and maintenance areas, allowing to perform distinct task from offshore assets inspection to harbor patrolling. This work presents SENSE, an autonomouS vEssel for multi-domaiN inSpection and maintEnance. It provides an open-source hardware and software architecture that is easy to replicate for both research institutes and industry. This is a multi-purpose vehicle capable of acquiring multi-domain data for inspecting and reconstructing maritime infrastructures. SENSE provides a research platform which can increase the situational awareness capabilities for ASVs. SENSE full configuration provides multimodal sensory data acquired from both domains using a Light Detection And Ranging (LiDAR), a stereoscopic camera, and a multibeam echosounder. In addition, it supplies navigation information obtained from a real-time kinematic satellite navigation system and inertial measurement units. Moreover, the tests performed at the harbor of Marina de Leça, at Porto, Portugal, resulted in a dataset which captures a fully operational harbor. It illustrates several conditions on maritime scenarios, such as undocking and docking examples, crossings with other vehicles and distinct types of moored vessels. The data available represents both domains of the maritime scenario, being the first public dataset acquired for multi-domain observation using a single vehicle. This paper also provides examples of applications for navigation and inspection on multi-domain scenarios, such as odometry estimation, bathymetric surveying and multi-domain mapping.

Frame-to-Frame Visual Odometry Estimation Network With Error Relaxation Method

Estimating frame-to-frame (F2F) visual odometry with monocular images has significant problems of propagated accumulated drift. We propose a learning-based approach for F2F monocular visual odometry estimation with novel and simple methods that consider the coherence of camera trajectories without any post-processing. The proposed network consists of two stages: initial estimation and error relaxation. In the first stage, the network learns disparity images to extract features and predicts relative camera pose between adjacent two frames through the attention, rotation, and translation networks. Then, loss functions are proposed in the error relaxation stage to reduce the local drift, increasing consistency under dynamic driving scenes. Moreover, our skip-ordering scheme shows the effectiveness of dealing with sequential data. Experiments with the KITTI benchmark dataset show that our proposed network outperforms other approaches with higher and more stable performance.

Masked GAN for Unsupervised Depth and Pose Prediction With Scale Consistency.

Previous work has shown that adversarial learning can be used for unsupervised monocular depth and visual odometry (VO) estimation, in which the adversarial loss and the geometric image reconstruction loss are utilized as the mainly supervisory signals to train the whole unsupervised framework. However, the performance of the adversarial framework and image reconstruction is usually limited by occlusions and the visual field changes between the frames. This article proposes a masked generative adversarial network (GAN) for unsupervised monocular depth and ego-motion estimations. The MaskNet and Boolean mask scheme are designed in this framework to eliminate the effects of occlusions and impacts of visual field changes on the reconstruction loss and adversarial loss, respectively. Furthermore, we also consider the scale consistency of our pose network by utilizing a new scale-consistency loss, and therefore, our pose network is capable of providing the full camera trajectory over a long monocular sequence. Extensive experiments on the KITTI data set show that each component proposed in this article contributes to the performance, and both our depth and trajectory predictions achieve competitive performance on the KITTI and Make3D data sets.

Low-Bandwidth and Compute-Bound RGB-D Planar Semantic SLAM.

Visual simultaneous location and mapping (SLAM) using RGB-D cameras has been a necessary capability for intelligent mobile robots. However, when using point-cloud map representations as most RGB-D SLAM systems do, limitations in onboard compute resources, and especially communication bandwidth can significantly limit the quantity of data processed and shared. This article proposes techniques that help address these challenges by mapping point clouds to parametric models in order to reduce computation and bandwidth load on agents. This contribution is coupled with a convolutional neural network (CNN) that extracts semantic information. Semantics provide guidance in object modeling which can reduce the geometric complexity of the environment. Pairing a parametric model with a semantic label allows agents to share the knowledge of the world with much less complexity, opening a door for multi-agent systems to perform complex tasking, and human–robot cooperation. This article takes the first step towards a generalized parametric model by limiting the geometric primitives to a planar surface and providing semantic labels when appropriate. Two novel compression algorithms for depth data and a method to independently fit planes to RGB-D data are provided, so that plane data can be used for real-time odometry estimation and mapping. Additionally, we extend maps with semantic information predicted from sparse geometries (planes) by a CNN. In experiments, the advantages of our approach in terms of computational and bandwidth resources savings are demonstrated and compared with other state-of-the-art SLAM systems.

DEEPLIO: DEEP LIDAR INERTIAL SENSOR FUSION FOR ODOMETRY ESTIMATION

Abstract. Having a good estimate of the position and orientation of a mobile agent is essential for many application domains such as robotics, autonomous driving, and virtual and augmented reality. In particular, when using LiDAR and IMU sensors as the inputs, most existing methods still use classical filter-based fusion methods to achieve this task. In this work, we propose DeepLIO, a modular, end-to-end learning-based fusion framework for odometry estimation using LiDAR and IMU sensors. For this task, our network learns an appropriate fusion function by considering different modalities of its input latent feature vectors. We also formulate a loss function, where we combine both global and local pose information over an input sequence to improve the accuracy of the network predictions. Furthermore, we design three sub-networks with different modules and architectures derived from DeepLIO to analyze the effect of each sensory input on the task of odometry estimation. Experiments on the benchmark dataset demonstrate that DeepLIO outperforms existing learning-based and model-based methods regarding orientation estimation and shows a marginal position accuracy difference.

Odometry Estimation from Sparse LiDAR Point Cloud Constrained by Image Feature Correspondence

Estimation of translation between consecutive frames, i.e., odometry, plays an important role in autonomous navigation. This paper presents an odometry estimation method using sparse LiDAR points and image feature points. In case of sparse LiDAR measurements, it is difficult to accurately estimate depth at image feature points. Image feature points with low-accuracy depth cause misconvergence in odometry optimization. To improve the robustness to the misconvergence, a new method with a Gaussian process that estimates not only the depth at image feature points but also the variance is proposed. By using this variance, it estimates the residual of image features in the world coordinate with depth, or in the image coordinate without depth. This allows more accurate and robust estimation than conventional methods in case of sparse LiDAR points. In an experiment with simulated sparse LiDAR points from the KITTI dataset, the proposed method is confirmed to estimate the odometry more accurately than conventional methods.

Lightweight 3-D Localization and Mapping for Solid-State LiDAR

The LIght Detection And Ranging (LiDAR) sensor has become one of the most important perceptual devices due to its important role in simultaneous localization and mapping (SLAM). Existing SLAM methods are mainly developed for mechanical LiDAR sensors, which are often adopted by large scale robots. Recently, the solid-state LiDAR is introduced and becomes popular since it provides a cost-effective and lightweight solution for small scale robots. Compared to mechanical LiDAR, solid-state LiDAR sensors have higher update frequency and angular resolution, but also have smaller field of view (FoV), which is very challenging for existing LiDAR SLAM algorithms. Therefore, it is necessary to have a more robust and computationally efficient SLAM method for this new sensing device. To this end, we propose a new SLAM framework for solid-state LiDAR sensors, which involves feature extraction, odometry estimation, and probability map building. The proposed method is evaluated on a warehouse robot and a hand-held device. In the experiments, we demonstrate both the accuracy and efficiency of our method using an Intel L515 solid-state LiDAR. The results show that our method is able to provide precise localization and high quality mapping. We made the source codes public at https://github.com/wh200720041/SSL_SLAM.

Intensity-SLAM: Intensity Assisted Localization and Mapping for Large Scale Environment

Simultaneous Localization And Mapping (SLAM) is a task to estimate the robot location and to reconstruct the environment based on observation from sensors such as LIght Detection And Ranging (LiDAR) and camera. It is widely used in robotic applications such as autonomous driving and drone delivery. Traditional LiDAR-based SLAM algorithms mainly leverage the geometric features from the scene context, while the intensity information from LiDAR is ignored. Some recent deep-learning-based SLAM algorithms consider intensity features and train the pose estimation network in an end-to-end manner. However, they require significant data collection effort and their generalizability to environments other than the trained one remains unclear. In this paper we introduce intensity features to a SLAM system. And we propose a novel full SLAM framework that leverages both geometry and intensity features. The proposed SLAM involves both intensity-based front-end odometry estimation and intensity-based back-end optimization. Thorough experiments are performed including both outdoor autonomous driving and indoor warehouse robot manipulation. The results show that the proposed method outperforms existing geometric-only LiDAR SLAM methods.

Multi-domain inspection of offshore wind farms using an autonomous surface vehicle

The offshore wind power industry is an emerging and exponentially growing sector, which calls to a necessity for a cyclical monitoring and inspection to ensure the safety and efficiency of the wind farm facilities. Thus, the emersed (aerial) and immersed (underwater) scenarios must be reconstructed to create a more complete and reliable map that maximizes the observability of all the offshore structures from the wind turbines to the cable arrays, presenting a multi domain scenario.This work proposes the use of an Autonomous Surface Vehicle (ASV) to map both domains simultaneously. As such, it will produce a multi-domain map through the fusion of navigational sensors, GPS and IMU, to localize the vehicle and aid the registration process for the perception sensors, 3D Lidar and Multibeam echosounder sonar. The performed experiments demonstrate the ability of the multi-domain mapping architecture to provide an accurate reconstruction of both scenarios into a single representation using the odometry system as the initial seed to further improve the map with data filtering and registration processes. An error of 0.049 m for the odometry estimation is observed with the GPS/IMU fusion for simulated data and 0.07 m for real field tests. The multi-domain map methodology requires an average of 300 ms per iteration to reconstruct the environment, with an error of at most 0.042 m in simulation.

Co-Planar Parametrization for Stereo-SLAM and Visual-Inertial Odometry

This work proposes a novel SLAM framework for stereo and visual inertial odometry estimation. It builds an efficient and robust parametrization of co-planar points and lines which leverages specific geometric constraints to improve camera pose optimization in terms of both efficiency and accuracy. %reduce the size of the Hessian matrix in the optimization. The pipeline consists of extracting 2D points and lines, predicting planar regions and filtering the outliers via RANSAC. Our parametrization scheme then represents co-planar points and lines as their 2D image coordinates and parameters of planes. We demonstrate the effectiveness of the proposed method by comparing it to traditional parametrizations in a novel Monte-Carlo simulation set. Further, the whole stereo SLAM and VIO system is compared with state-of-the-art methods on the public real-world dataset EuRoC. Our method shows better results in terms of accuracy and efficiency than the state-of-the-art. The code is released at https://github.com/LiXin97/Co-Planar-Parametrization.

An improved method for odometry estimation based on EKF and Temporal Convolutional Network

Compared with wheeled robots, the walking direction and distance of biped robots are difficult to accurately control, and the body of the robot will periodically shake during walking. During the walking of the biped robot, the movement of the torso drives the head camera to shake, which causes great interference to the slam algorithm, and leads to deviations in the vision-based mileage estimation method. This paper mainly combines the vision positioning method and the sensor data on the biped robot and estimates the position information of the robot through data fusion. This method makes full use of the sensor data on the robot to improve the estimation accuracy of the odometer.

DeepTIO: A Deep Thermal-Inertial Odometry With Visual Hallucination

Visual odometry shows excellent performance in a wide range of environments. However, in visually-denied scenarios (e.g. heavy smoke or darkness), pose estimates degrade or even fail. Thermal cameras are commonly used for perception and inspection when the environment has low visibility. However, their use in odometry estimation is hampered by the lack of robust visual features. In part, this is as a result of the sensor measuring the ambient temperature profile rather than scene appearance and geometry. To overcome this issue, we propose a Deep Neural Network model for thermal-inertial odometry (DeepTIO) by incorporating a visual hallucination network to provide the thermal network with complementary information. The hallucination network is taught to predict fake visual features from thermal images by using Huber loss. We also employ selective fusion to attentively fuse the features from three different modalities, i.e thermal, hallucination, and inertial features. Extensive experiments are performed in hand-held and mobile robot data in benign and smoke-filled environments, showing the efficacy of the proposed model.