Visual Odometry Research Articles

Robust Neural Visual Inertial Odometry With Deep Velocity Constraint

Robust Neural Visual Inertial Odometry With Deep Velocity Constraint

4Seasons: Benchmarking Visual SLAM and Long-Term Localization for Autonomous Driving in Challenging Conditions

AbstractIn this paper, we present a novel visual SLAM and long-term localization benchmark for autonomous driving in challenging conditions based on the large-scale 4Seasons dataset. The proposed benchmark provides drastic appearance variations caused by seasonal changes and diverse weather and illumination conditions. While significant progress has been made in advancing visual SLAM on small-scale datasets with similar conditions, there is still a lack of unified benchmarks representative of real-world scenarios for autonomous driving. We introduce a new unified benchmark for jointly evaluating visual odometry, global place recognition, and map-based visual localization performance which is crucial to successfully enable autonomous driving in any condition. The data has been collected for more than one year, resulting in more than 300 km of recordings in nine different environments ranging from a multi-level parking garage to urban (including tunnels) to countryside and highway. We provide globally consistent reference poses with up to centimeter-level accuracy obtained from the fusion of direct stereo-inertial odometry with RTK GNSS. We evaluate the performance of several state-of-the-art visual odometry and visual localization baseline approaches on the benchmark and analyze their properties. The experimental results provide new insights into current approaches and show promising potential for future research. Our benchmark and evaluation protocols will be available at https://go.vision.in.tum.de/4seasons.

Landmark-aware autonomous odometry correction and map pruning for planetary rovers

Planetary rover autonomous localization is paramount for a planetary surface exploration mission. However, existing methods demonstrate limited localization accuracy, mostly due to the unstructured texture characterization of planetary surface. In response, this study presents a novel Neural Radiance Field (NeRF) driven visual odometry correction method that allows for high-precision 6-DoF rover pose estimation and local map pruning. First, an innovative image saliency evaluation approach, combining binarization and feature detection, is introduced to meticulously select landmarks that are conducive to rover re-localization. Subsequently, we conduct 3D reconstruction and rendering of the chosen landmarks based on a-priori knowledge of planetary surface images and their Neural Radiance Field (NeRF) models. High-precision odometry correction is achieved through the optimization of photometric loss between NeRF rending images and real images. Simultaneously, the odometry correction mechanism is employed in an autonomous manner to refine the NeRF model of the corresponding landmark, leading to an improved local map and gradually enhanced rover localization accuracy. Numerical simulation and experiment trials are carried out to evaluate the performance of the proposed method, results of which demonstrate state-of-the-art rover re-localization accuracy and local map pruning.

Towards robust visual odometry by motion blur recovery

IntroductionMotion blur, primarily caused by rapid camera movements, significantly challenges the robustness of feature point tracking in visual odometry (VO).MethodsThis paper introduces a robust and efficient approach for motion blur detection and recovery in blur-prone environments (e.g., with rapid movements and uneven terrains). Notably, the Inertial Measurement Unit (IMU) is utilized for motion blur detection, followed by a blur selection and restoration strategy within the motion frame sequence. It marks a substantial improvement over traditional visual methods (typically slow and less effective, falling short in meeting VO’s realtime performance demands). To address the scarcity of datasets catering to the image blurring challenge in VO, we also present the BlurVO dataset. This publicly available dataset is richly annotated and encompasses diverse blurred scenes, providing an ideal environment for motion blur evaluation.

RD-VIO: Robust Visual-Inertial Odometry for Mobile Augmented Reality in Dynamic Environments.

It is typically challenging for visual or visual-inertial odometry systems to handle the problems of dynamic scenes and pure rotation. In this work, we design a novel visual-inertial odometry (VIO) system called RD-VIO to handle both of these two problems. First, we propose an IMU-PARSAC algorithm which can robustly detect and match keypoints in a two-stage process. In the first state, landmarks are matched with new keypoints using visual and IMU measurements. We collect statistical information from the matching and then guide the intra-keypoint matching in the second stage. Second, to handle the problem of pure rotation, we detect the motion type and adapt the deferred-triangulation technique during the data-association process. We make the pure-rotational frames into the special subframes. When solving the visual-inertial bundle adjustment, they provide additional constraints to the pure-rotational motion. We evaluate the proposed VIO system on public datasets and online comparison. Experiments show the proposed RD-VIO has obvious advantages over other methods in dynamic environments.

PL-EVIO: Robust Monocular Event-Based Visual Inertial Odometry With Point and Line Features

PL-EVIO: Robust Monocular Event-Based Visual Inertial Odometry With Point and Line Features

Fine-MVO: Toward Fine-Grained Feature Enhancement for Self-Supervised Monocular Visual Odometry in Dynamic Environments

Fine-MVO: Toward Fine-Grained Feature Enhancement for Self-Supervised Monocular Visual Odometry in Dynamic Environments

Improved Multi-Sensor Fusion Dynamic Odometry Based on Neural Networks

High-precision simultaneous localization and mapping (SLAM) in dynamic real-world environments plays a crucial role in autonomous robot navigation, self-driving cars, and drone control. To address this dynamic localization issue, in this paper, a dynamic odometry method is proposed based on FAST-LIVO, a fast LiDAR (light detection and ranging)–inertial–visual odometry system, integrating neural networks with laser, camera, and inertial measurement unit modalities. The method first constructs visual–inertial and LiDAR–inertial odometry subsystems. Then, a lightweight neural network is used to remove dynamic elements from the visual part, and dynamic clustering is applied to the LiDAR part to eliminate dynamic environments, ensuring the reliability of the remaining environmental data. Validation of the datasets shows that the proposed multi-sensor fusion dynamic odometry can achieve high-precision pose estimation in complex dynamic environments with high continuity, reliability, and dynamic robustness.

Direct RGB-D visual odometry with point features

Direct RGB-D visual odometry with point features

RGB-D visual odometry by constructing and matching features at superpixel level

Abstract Visual odometry (VO) is a key technology for estimating camera motion from captured images. In this paper, we propose a novel RGB-D visual odometry by constructing and matching features at the superpixel level that represents better adaptability in different environments than state-of-the-art solutions. Superpixels are content-sensitive and perform well in information aggregation. They could thus characterize the complexity of the environment. Firstly, we designed the superpixel-based feature SegPatch and its corresponding 3D representation MapPatch. By using the neighboring information, SegPatch robustly represents its distinctiveness in various environments with different texture densities. Due to the inclusion of depth measurement, the MapPatch constructs the scene structurally. Then, the distance between SegPatches is defined to characterize the regional similarity. We use the graph search method in scale space for searching and matching. As a result, the accuracy and efficiency of matching process are improved. Additionally, we minimize the reprojection error between the matched SegPatches and estimate camera poses through all these correspondences. Our proposed VO is evaluated on the TUM dataset both quantitatively and qualitatively, showing good balance to adapt to the environment under different realistic conditions.

Multi-Robot Collaborative Mapping with Integrated Point-Line Features for Visual SLAM.

Simultaneous Localization and Mapping (SLAM) enables mobile robots to autonomously perform localization and mapping tasks in unknown environments. Despite significant progress achieved by visual SLAM systems in ideal conditions, relying solely on a single robot and point features for mapping in large-scale indoor environments with weak-texture structures can affect mapping efficiency and accuracy. Therefore, this paper proposes a multi-robot collaborative mapping method based on point-line fusion to address this issue. This method is designed for indoor environments with weak-texture structures for localization and mapping. The feature-extraction algorithm, which combines point and line features, supplements the existing environment point feature-extraction method by introducing a line feature-extraction step. This integration ensures the accuracy of visual odometry estimation in scenes with pronounced weak-texture structure features. For relatively large indoor scenes, a scene-recognition-based map-fusion method is proposed in this paper to enhance mapping efficiency. This method relies on visual bag of words to determine overlapping areas in the scene, while also proposing a keyframe-extraction method based on photogrammetry to improve the algorithm's robustness. By combining the Perspective-3-Point (P3P) algorithm and Bundle Adjustment (BA) algorithm, the relative pose-transformation relationships of multi-robots in overlapping scenes are resolved, and map fusion is performed based on these relative pose relationships. We evaluated our algorithm on public datasets and a mobile robot platform. The experimental results demonstrate that the proposed algorithm exhibits higher robustness and mapping accuracy. It shows significant effectiveness in handling mapping in scenarios with weak texture and structure, as well as in small-scale map fusion.

Hybrid Visual Odometry Algorithm Using a Downward-Facing Monocular Camera

The increasing interest in developing robots capable of navigating autonomously has led to the necessity of developing robust methods that enable these robots to operate in challenging and dynamic environments. Visual odometry (VO) has emerged in this context as a key technique, offering the possibility of estimating the position of a robot using sequences of onboard cameras. In this paper, a VO algorithm is proposed that achieves sub-pixel precision by combining optical flow and direct methods. This approach uses only a downward-facing, monocular camera, eliminating the need for additional sensors. The experimental results demonstrate the robustness of the developed method across various surfaces, achieving minimal drift errors in calculation.

Visual Odometry in GPS-Denied Zones for Fixed-Wing Unmanned Aerial Vehicle with Reduced Accumulative Error Based on Satellite Imagery

In this paper, we present a method for estimating GPS coordinates from visual information captured by a monocular camera mounted on a fixed-wing tactical Unmanned Aerial Vehicle at high altitudes (up to 3000 m) in GPS-denied zones. The main challenge in visual odometry using aerial images is the computation of the scale due to irregularities in the elevation of the terrain. That is, it is not possible to accurately convert from pixels in the image to meters in space, and the error accumulates. The contribution of this work is a reduction in the accumulated error by comparing the images from the camera with satellite images without requiring the dynamic model of the vehicle. The algorithm has been tested in real-world flight experiments at altitudes above 1000 m and in missions over 17 km. It has been proven that the algorithm prevents an increase in the accumulated error.

Pose Estimation Based on Bidirectional Visual–Inertial Odometry with 3D LiDAR (BV-LIO)

Due to the limitation of a single sensor such as only camera or only LiDAR, the Visual SLAM detects few effective features in the case of poor lighting or no texture. The LiDAR SLAM will also degrade in an unstructured environment and open spaces, which reduces the accuracy of pose estimation and the quality of mapping. In order to solve this problem, on account of the high efficiency of Visual odometry and the high accuracy of LiDAR odometry, this paper investigates the multi-sensor fusion of bidirectional visual–inertial odometry with 3D LiDAR for pose estimation. This method can couple the IMU with the bidirectional vision respectively, and the LiDAR odometry is obtained assisted by the bidirectional visual inertial. The factor graph optimization is constructed, which effectively improves the accuracy of pose estimation. The algorithm in this paper is compared with LIO-LOAM, LeGO-LOAM, VINS-Mono, and so on using challenging datasets such as KITTI and M2DGR. The results show that this method effectively improves the accuracy of pose estimation and has high application value for mobile robots.

Detection-first tightly-coupled LiDAR-Visual-Inertial SLAM in dynamic environments

To address the challenges posed by the dynamic environment for Simultaneous Localization and Mapping (SLAM), a detection-first tightly-coupled LiDAR-Visual-Inertial SLAM incorporating lidar, camera, and inertial measurement unit (IMU) is proposed. Firstly, the point cloud clustering with semantic labels are obtained by fusing image and point cloud information. Then, a tracking algorithm is applied to obtain the information of the motion state of the targets. Afterwards, the tracked dynamic targets are utilized to eliminate extraneous feature points. Finally, a factor graph is used to jointly optimize the IMU pre-integration, and tightly couple the laser odometry and visual odometry within the system. To validate the performance of the proposed SLAM framework, both public datasets (KITTI and UrbanNav) and actual scene data are tested. The experimental results show that compared with LeGO-LOAM, LIO-SAM and LVI-SAM for public dataset, the root mean squared error (RMSE) of proposed algorithm is decreased by 44.56 % (4.47 m) and 4.15 % (4.62 m) in high dynamic scenes and normal scenes, respectively. Through actual scene data, the proposed algorithm mitigates the impact of dynamic objects on map building directly.

Mix-VIO: A Visual Inertial Odometry Based on a Hybrid Tracking Strategy.

In this paper, we proposed Mix-VIO, a monocular and binocular visual-inertial odometry, to address the issue where conventional visual front-end tracking often fails under dynamic lighting and image blur conditions. Mix-VIO adopts a hybrid tracking approach, combining traditional handcrafted tracking techniques with Deep Neural Network (DNN)-based feature extraction and matching pipelines. The system employs deep learning methods for rapid feature point detection, while integrating traditional optical flow methods and deep learning-based sparse feature matching methods to enhance front-end tracking performance under rapid camera motion and environmental illumination changes. In the back-end, we utilize sliding window and bundle adjustment (BA) techniques for local map optimization and pose estimation. We conduct extensive experimental validations of the hybrid feature extraction and matching methods, demonstrating the system's capability to maintain optimal tracking results under illumination changes and image blur.

A Fast and Light-weight Non-Iterative Visual Odometry with RGB-D Cameras

A Fast and Light-weight Non-Iterative Visual Odometry with RGB-D Cameras

Visible Light Positioning With Visual Odometry: A Single Luminaire Based Positioning Algorithm

Visible Light Positioning With Visual Odometry: A Single Luminaire Based Positioning Algorithm

An Improved Strategy for Active Visual Odometry Based on Robust Adaptive Unscented Kalman Filter

An Improved Strategy for Active Visual Odometry Based on Robust Adaptive Unscented Kalman Filter

Catadioptric omnidirectional thermal odometry in dynamic environment

This paper presents a catadioptric omnidirectional thermal odometry (COTO) system that estimates the six degrees of freedom (DoF) pose of a camera using only omnidirectional thermal images in visually degraded, fast-motion, and dynamic environments. First, we design and fabricate a central hyperbolic catadioptric omnidirectional thermal camera that captures surrounding thermal images with 360° horizontal field of view (FoV), and improve the omnidirectional camera model and calibration method to obtain high-precision camera intrinsic parameter. Second, we propose the epipolar curve constraint combining with omnidirectional thermal object detection to significantly reduce the interference of moving objects on pose estimation. Third, the implemented COTO pipeline consists of photometric calibration, dynamic region removal, tracking and mapping to overcome the drawbacks of photometric inconsistency and large distortion in omnidirectional thermal images. Experiments have been conducted on a total of 17 sequences of Lab, Outdoor and Driving, amounting to more than 60,000 omnidirectional thermal images of real environments. The experimental results indicate that the proposed COTO system has excellent localization accuracy and unparalleled robustness over the current state-of-the-art methods. The average localization accuracy measured by the absolute trajectory error (ATE) is less than 15 cm from the ground truth in both Lab and Outdoor sequences. In addition, COTO was the only system with complete and successful tracking in all sequences. The system can be used as an innovative localization solution, particularly in challenging environments with changes in ambient light, rapid vehicle motion, and moving object interference, which can be a difficult problem for visual odometry to solve.