Visual Odometry Algorithm Research Articles

Hybrid Visual Odometry Algorithm Using a Downward-Facing Monocular Camera

The increasing interest in developing robots capable of navigating autonomously has led to the necessity of developing robust methods that enable these robots to operate in challenging and dynamic environments. Visual odometry (VO) has emerged in this context as a key technique, offering the possibility of estimating the position of a robot using sequences of onboard cameras. In this paper, a VO algorithm is proposed that achieves sub-pixel precision by combining optical flow and direct methods. This approach uses only a downward-facing, monocular camera, eliminating the need for additional sensors. The experimental results demonstrate the robustness of the developed method across various surfaces, achieving minimal drift errors in calculation.

Resilient Simultaneous Localization and Mapping Fusing Ultra Wide Band Range Measurements and Visual Odometry

In this paper we consider a Simultaneous Localization and Mapping (SLAM) problem for a moving agent using Visual Odometry (VO) while measuring the range from a set of Ultra Wide Band (UWB) antennas, deployed in unknown position in the environment. The solution approach is based on a switching observer which, under standard working conditions, for each observed UWB, uses a two dimensional Extended Kalman Filter (EKF) providing an estimate of the range and bearing of the observed UWB with respect to the agent. This information is then used in a Robust EKF algorithm which solves the SLAM problem with performances that, even before closing the loop, are comparable to the ones that a VO algorithm (namely ORB-SLAM2) would obtain only after closing the loop. Moreover, a resilient module is added to the algorithm to evaluate the reliability of the position estimate of each observed UWB. When the Visual Odometry is not available, the switching observer uses an auxiliary EKF to provide an estimate of the agent position. This makes the proposed approach robust with respect to several kinds of unmodeled disturbances, like multipath effects, and automatically adapts to sensor failures with resilience (e.g. when Visual Odometry or UWB measurements are not available).

Robust Monocular Visual Inertial Odometry in γ Radioactive Environments using Edge-based Point Features

Abstract In the γ radioactive environment, high-energy photons induce degradation of the image sensor, which effects feature detection and tracking in the Visual Inertial Odometry (VIO) algorithm and deteriorates its localization performance. To address this issue, in this work, we propose a monocular VIO method using edge-based point features. To mitigate the effects of radiation noise, firstly, in the image preprocessing module, the median filter is used for real-time image denoising. Secondly, in the data association module, both Shi-Tomasi and edge-based point features are detected. The edge-based point feature is the endpoint or corner point in the salient edge map, which is more robust to radiation noise. Then, the bi-directional motion parallaxes and the RANdom SAmple Consensus (RANSAC) method are exploited to reject outliers. Finally, the point features measurements and Inertial Measurement Unit (IMU) pre-integration measurements are added into a tightly-coupled sliding window optimization VIO framework for localization estimation. The proposed method is verified by synthetic and real γ radioactive environment datasets. The experimental results show that the proposed method achieves more accurate and robust localization than the state-of-the-art VIO approaches in the γ radioactive environments.

MIXO: Mixture Of Experts-Based Visual Odometry for Multicamera Autonomous Systems

Visual-inertial odometry is a fundamental technology exploited by autonomous vehicles and mobile robots to determine their position in unknown environments. Indeed, many techniques have been proposed for monocular and stereo cameras. State-of-the-art autonomous vehicles, however, are equipped with multiple cameras covering the entire vehicle environment. We present here MIxture of eXperts Odometry (MIXO), a data-driven, machine learning-based technique that loosely combines odometry outputs from multiple cameras to obtain a more accurate and robust global estimate. It stems from the intuition that each camera provides an optimal vantage point in specific driving scenarios. In MIXO, each camera (or expert) is individually processed by a state-of-the-art visual odometry algorithm (e.g., ORB-SLAM2). Then, the odometry estimates are mixed by a gating network, which selects the locally optimal experts in the current operational conditions and weights their contributions accordingly. MIXO is a lightweight module that can be easily implemented on top of any visual odometry algorithm. Experimental results on real-life data from autonomous vehicles show that MIXO achieves more robust and accurate results than any single camera, reducing the absolute rotational and translation error by 38% and 15%, respectively.

Development and testing of a navigation solution for Autonomous Underwater Vehicles based on stereo vision

Autonomous Underwater Vehicles (AUVs) employed for inspection and monitoring applications require a reliable estimate of their navigation status to successfully accomplish the scheduled mission. Considering that they are typically endowed with vision systems to collect images of the surveyed environment, approaches based on Visual Odometry (VO) allow to utilise the optical payload also to perform robot navigation. Despite a large body of prior research, the topic of navigation based on stereo vision is still an open problem in the marine domain. Within this context, this paper proposes an underwater navigation framework utilising a stereo camera as a linear velocity sensor. The purpose is to assess the potentiality of a stereo vision system to indirectly measure the linear velocity of an AUV, and provide this information to the navigation framework of the vehicle. The proposed strategy consists in a visual inertial odometry solution fusing linear velocity estimates, derived from optical information, with attitude and depth measurements to retrieve the robot navigation status. Such linear velocity estimates are computed exploiting a stereo VO algorithm, relying on a 3D-to-2D approach, whose source code is made available online. The developed strategy is evaluated on real underwater datasets acquired during monitoring surveys performed by an AUV equipped with Doppler Velocity Log (DVL), bottom-looking stereo camera, inertial unit, and depth sensor. The results, obtained through a comparison with the DVL taken as benchmark, confirm the approach as feasible and exploitable in underwater inspection and monitoring contexts.

Human Visual Attention Mechanism-Inspired Point-and-Line Stereo Visual Odometry for Environments with Uneven Distributed Features

Visual odometry is critical in visual simultaneous localization and mapping for robot navigation. However, the pose estimation performance of most current visual odometry algorithms degrades in scenes with unevenly distributed features because dense features occupy excessive weight. Herein, a new human visual attention mechanism for point-and-line stereo visual odometry, which is called point-line-weight-mechanism visual odometry (PLWM-VO), is proposed to describe scene features in a global and balanced manner. A weight-adaptive model based on region partition and region growth is generated for the human visual attention mechanism, where sufficient attention is assigned to position-distinctive objects (sparse features in the environment). Furthermore, the sum of absolute differences algorithm is used to improve the accuracy of initialization for line features. Compared with the state-of-the-art method (ORB-VO), PLWM-VO show a 36.79% reduction in the absolute trajectory error on the Kitti and Euroc datasets. Although the time consumption of PLWM-VO is higher than that of ORB-VO, online test results indicate that PLWM-VO satisfies the real-time demand. The proposed algorithm not only significantly promotes the environmental adaptability of visual odometry, but also quantitatively demonstrates the superiority of the human visual attention mechanism.

Improving the Computational Efficiency of ROVIO

ROVIO is one of the state-of-the-art monocular visual inertial odometry algorithms. It uses an Iterative Extended Kalman Filter (IEKF) to align visual features and update the vehicle state simultaneously by including the feature locations in the state vector of the IEKF. This algorithm is single-core intensive, which allows the other cores to be used for other algorithms, such as object detection and path optimization. However, the computational cost of the algorithm grows rapidly with the maximum number of features to track. Each feature adds three new states (a 2D bearing vector and inverse depth), leading to bigger matrix multiplications that are computationally expensive. The main computational load of ROVIO is the iterative update step of the IEKF. In this work, we reduce the average computational cost of ROVIO by [Formula: see text] on an NVIDIA Jetson TX2, without affecting the accuracy of the algorithm. This computational gain is mainly achieved by exploiting the sparse matrices in ROVIO. Furthermore, we reduce the computational peaks by pre-selecting new features based on their already calculated FAST score. The combination of both modifications allows us to run ROVIO on the computationally restricted Raspberry Pi Zero 2[Formula: see text]W.

A robust method for approximate visual robot localization in feature-sparse sewer pipes.

Buried sewer pipe networks present many challenges for robot localization systems, which require non-standard solutions due to the unique nature of these environments: they cannot receive signals from global positioning systems (GPS) and can also lack visual features necessary for standard visual odometry algorithms. In this paper, we exploit the fact that pipe joints are equally spaced and develop a robot localization method based on pipe joint detection that operates in one degree-of-freedom along the pipe length. Pipe joints are detected in visual images from an on-board forward facing (electro-optical) camera using a bag-of-keypoints visual categorization algorithm, which is trained offline by unsupervised learning from images of sewer pipe joints. We augment the pipe joint detection algorithm with drift correction using vision-based manhole recognition. We evaluated the approach using real-world data recorded from three sewer pipes (of lengths 30, 50 and 90m) and benchmarked against a standard method for visual odometry (ORB-SLAM3), which demonstrated that our proposed method operates more robustly and accurately in these feature-sparse pipes: ORB-SLAM3 completely failed on one tested pipe due to a lack of visual features and gave a mean absolute error in localization of approximately 12%-20% on the other pipes (and regularly lost track of features, having to re-initialize multiple times), whilst our method worked successfully on all tested pipes and gave a mean absolute error in localization of approximately 2%-4%. In summary, our results highlight an important trade-off between modern visual odometry algorithms that have potentially high precision and estimate full six degree-of-freedom pose but are potentially fragile in feature sparse pipes, versus simpler, approximate localization methods that operate in one degree-of-freedom along the pipe length that are more robust and can lead to substantial improvements in accuracy.

Boosting auxiliary task guidance: a probabilistic approach

This work aims to introduce a novel approach for auxiliary task guidance (ATG). In this approach, our goal is to achieve effective guidance from a suitable auxiliary task by utilizing the uncertainty in calculated gradients for a mini-batch of samples. Our method calculates a probabilistic fitness factor of the auxiliary task gradient for each of the shared weights to guide the main task at every training step of mini-batch gradient descent. We have shown that this proposed factor incorporates task specific confidence of learning to manipulate ATG in an effective manner. For studying the potency of the method, monocular visual odometry (VO) has been chosen as an application. Substantial experiments have been done on the KITTI VO dataset for solving monocular VO with a simple convolutional neural network (CNN) architecture. Corresponding results show that our ATG method significantly boosts the performance of supervised learning for VO. It also out performs state-of-the-art (SOTA) auxiliary guided methods we applied for VO. The proposed method is able to achieve decent scores (in some cases competitive)compared to existing SOTA supervised monocular VO algorithms, while keeping an exceptionally low parameter space in supervised regime.

A REVIEW OF VISUAL ODOMETRY FOR UAV AUTONOMOUS NAVIGATION

Visual odometry (VO) has become a popular technique for autonomous naviga-tion of unmanned aerial vehicles (UAVs) in GPS-denied environments. Although, the concept of VO is not new, it has been successfully implemented for on-ground vehi-cles only. There are several product level solutions of autonomous navigation, which rely on VO or contain VO as an important subpart. Meanwhile, a proper system of VO for UAVs is not yet achieved and continues to be a subject of an active research. In this paper, we show the comparison of the latest VO techniques for UAV nav-igation. We begin by describing the basic principles of VO, then review the different types of VO algorithms commonly used in UAV navigation, such as monocular, stereo, and RGB-D approaches. For each type of algorithm, we discuss the advantages and limitations, and provide examples of recent research and applications. Finally, we dis-cuss some of the emerging trends and future directions in visual odometry for UAV navigation, such as Visual inertial odometry approaches and real-time implementa-tions. This review paper provides a comprehensive overview of the current state-of-the-art in visual odometry for UAV autonomous navigation, highlighting the ad-vantages and limitations of different types of algorithms and identifying key challeng-es and future research directions.

ViT VO - A Visual Odometry technique Using CNN-Transformer Hybrid Architecture

Localization is one of the main tasks involved in the operation of autonomous agents (e.g., vehicle, robot etc.). It allows them to be able to track their paths and properly detect and avoid obstacles. Visual Odometry (VO) is one of the techniques used for agent localization. VO involves estimating the motion of an agent using the images taken by cameras attached to it. Conventional VO algorithms require specific workarounds for challenges posed by the working environment and the captured sensor data. On the other hand, Deep Learning approaches have shown tremendous efficiency and accuracy in tasks that require high degree of adaptability and scalability. In this work, a novel deep learning model is proposed to perform VO tasks for space robotic applications. The model consists of an optical flow estimation module which abstracts away scene-specific details from the input video sequence and produces an intermediate representation. The CNN module which follows next learn relative poses from the optical flow estimates. The final module is a state-of-the-art Vision Transformer, which learn absolute pose from the relative pose learnt by the CNN module. The model is trained on the KITTI dataset and has obtained a promising accuracy of approximately 2%. It has outperformed the baseline model, MagicVO, in a few sequences in the dataset.

A Generic Image Processing Pipeline for Enhancing Accuracy and Robustness of Visual Odometry.

The accuracy of pose estimation from feature-based Visual Odometry (VO) algorithms is affected by several factors such as lighting conditions and outliers in the matched features. In this paper, a generic image processing pipeline is proposed to enhance the accuracy and robustness of feature-based VO algorithms. The pipeline consists of three stages, each addressing a problem that affects the performance of VO algorithms. The first stage tackles the lighting condition problem, where a filter called Contrast Limited Adaptive Histogram Equalization (CLAHE) is applied to the images to overcome changes in lighting in the environment. The second stage uses the Suppression via Square Covering (SSC) algorithm to ensure the features are distributed properly over the images. The last stage proposes a novel outliers rejection approach called the Angle-based Outlier Rejection (AOR) algorithm to remove the outliers generated in the feature matching process. The proposed pipeline is generic and modular and can be integrated with any type of feature-based VO (monocular, RGB-D, or stereo). The efficiency of the proposed pipeline is validated using sequences from KITTI (for stereo VO) and TUM (for RGB-D VO) datasets, as well as experimental sequences using an omnidirectional mobile robot (for monocular VO). The obtained results showed the performance gained by enhancing the accuracy and robustness of the VO algorithms without compromising on the computational cost using the proposed pipeline. The results are substantially better as opposed to not using the pipeline.

Contrast Maximization-Based Feature Tracking for Visual Odometry with an Event Camera

As a new type of vision sensor, the dynamic and active-pixel vision sensor (DAVIS) outputs image intensity and asynchronous event streams in the same pixel array. We present a novel visual odometry algorithm based on the DAVIS in this paper. The Harris detector and the Canny detector are utilized to extract an initialized tracking template from the image sequence. The spatio-temporal window is selected by determining the life cycle of the asynchronous event streams. The alignment on timestamps is achieved by tracking the motion relationship between the template and events within the window. A contrast maximization algorithm is adopted for the estimation of the optical flow. The IMU data are used to calibrate the position of the templates during the update process that is exploited to estimate camera trajectories via the ICP algorithm. In the end, the proposed visual odometry algorithm is evaluated in several public object tracking scenarios and compared with several other algorithms. The tracking results show that our visual odometry algorithm can achieve better accuracy and lower latency tracking trajectory than other methods.

Hybrid sparse monocular visual odometry with online photometric calibration

Most monocular visual Simultaneous Localization and Mapping (vSLAM) and visual odometry (VO) algorithms focus on either feature-based methods or direct methods. Hybrid (semi-direct) approach is less studied although it is equally important. In this paper, a hybrid sparse visual odometry (HSO) algorithm with online photometric calibration is proposed for monocular vision. HSO introduces two novel measures, that is, direct image alignment with adaptive mode selection and image photometric description using ratio factors, to enhance the robustness against dramatic image intensity changes and motion blur. Moreover, HSO is able to establish pose constraints between keyframes far apart in time and space by using KLT tracking enhanced with a local-global brightness consistency. The convergence speed of candidate map points is adopted as the basis for keyframe selection, which strengthens the coordination between the front end and the back end. Photometric calibration is elegantly integrated into the VO system working in tandem: (1) Photometric interference from the camera, such as vignetting and changes in exposure time, is accurately calibrated and compensated in HSO, thereby improving the accuracy and robustness of VO. (2) On the other hand, VO provides pre-calculated data for the photometric calibration algorithm, which reduces resource consumption and improves the estimation accuracy of photometric parameters. Extensive experiments are performed on various public datasets to evaluate the proposed HSO against the state-of-the-art monocular vSLAM/VO and online photometric calibration methods. The results show that the proposed HSO achieves superior performance on VO and photometric calibration in terms of accuracy, robustness, and efficiency, being comparable with the state-of-the-art VO/vSLAM systems. We open source HSO for the benefit of the community.

Feature-Based Correspondence Filtering Using Structural Similarity Index for Visual Odometry

The stereo correspondence problem is one of the most pre-eminent problems in a stereo vision system. With the right correspondence, a stereo vision system can help cap over diverse problems, while on the other hand, a wrong correspondence can be costly. While the performance of a feature-based correspondence approach is exceptional, the method can still produce wrong correspondences. This work presents an amalgam of feature-based and correlation-based correspondence, where the local pixels around a feature pair are compared using Structural SIMilarity index (SSIM), enhancing the correspondences, and a semantic-based filtering module, which further filters the obtained corresponding features using semantic data whenever detected in both the stereo image pair. While approaches in the literature are focused towards finding better features and their representation, the proposed approach advocates that correlation-based verification of the features can filter out bad correspondences, and in addition, aided by semantic-level filtering. These two modules establish the novelty of the work. The proposed correspondence matching algorithm is used to solve the problem of Visual Odometry to let a low-cost robot compute its pose in a novel environment. The experimental results show adequate filtering of wrong feature correspondence wherein, different environments with different lighting conditions were also considered. The proposed approach outperformed numerous state-of-the-art approaches available in the literature. The visual odometry algorithm using the proposed correspondence matching is compared against classical methods and a deep learning method, and it is observed that the proposed approach delivers lower trajectory error values in most scenarios on the KITTI dataset sequences.

Visual odometry algorithm based on geometric prior for dynamic environments

Simultaneous localization and mapping (SLAM) is considered to be an important way for some smart devices to perform automatic path planning, and many successful SLAM systems have been developed in the past few years. Most existing approaches rely heavily on static world assumptions, and such strong assumptions limit the application of most vSLAM (visual SLAM) in complex dynamic reality environments, where dynamic objects often lead to incorrect data association in tracking, which reduces the overall accuracy and robustness of the system and causes tracking crashes. The dynamic objects in the map may change over time; thus, distinguishing dynamic information in a scene is challenging. In order to solve the interference problem of dynamic objects, most point-based visual odometry algorithms have concentrated on feature matching or direct pixel intensity matching, disregarding an ordinary but crucial image entity: geometric information. In this article, we put forward a novel visual odometry algorithm based on dynamic point detection methods called geometric prior and constraints. It removes the moving objects by combining the spatial geometric information of the image and depends on the remaining features to estimate the position of the camera. To the best of our knowledge, our proposed algorithm achieves superior performance over existing methods on a variety of public datasets.

Adaptive compensation visual odometry in dynamic scenarios

In dynamic scenarios, dynamic participants break the static assumptions of the visual odometry (VO) algorithm. Hence, dynamic participants are typically removed and only static participants are used as motion references. When semantic information is used to eliminate participants with dynamic semantic labels in the scene, the dynamic degree of the scene affects system robustness and poses estimation accuracy. In this paper, we propose a VO system that employs adaptive compensation for semantic information and utilizes an adaptive compensation model for sparse static feature map construction to make the compensation process a flexible, independent thread. Moreover, a layered extraction fusion framework is proposed to uniformly sample equi-probability feature points both globally and locally. This framework combines multi-level weights of scene mapping and spatial-temporal priority information of self-organization. Finally, in the candidate pixel extraction stage, system efficiency is improved through region matching candidate pixel detection, extraction and fusion semantic constraints. Experiments are done by using the TUM RGBD datasets. Comparisons with many conventional visual mileage calculation methods reveal that the absolute and relative trajectory errors of camera motion are significantly reduced in most scenes with different dynamic degrees. Thus, compared to previous algorithms, the proposed algorithm is more robust and precise in dynamic scenes.

MSC-VO: Exploiting Manhattan and Structural Constraints for Visual Odometry

Visual odometry algorithms tend to degrade when facing low-textured scenes —from e.g. human-made environments—, where it is often difficult to find a sufficient number of point features. Alternative geometrical visual cues, such as lines, which can often be found within these scenarios, can become particularly useful. Moreover, these scenarios typically present structural regularities, such as parallelism or orthogonality, and hold the Manhattan World assumption. Under these premises, in this work, we introduce MSC-VO, an RGB-D -based visual odometry approach that combines both point and line features and leverages, if exist, those structural regularities and the Manhattan axes of the scene. Within our approach, these structural constraints are initially used to estimate accurately the 3D position of the extracted lines. These constraints are also combined next with the estimated Manhattan axes and the reprojection errors of points and lines to refine the camera pose by means of local map optimization. Such a combination enables our approach to operate even in the absence of the aforementioned constraints, allowing the method to work for a wider variety of scenarios. Furthermore, we propose a novel multi-view Manhattan axes estimation procedure that mainly relies on line features. MSC-VO is assessed using several public datasets, outperforming other state-of-the-art solutions, and comparing favourably even with some SLAM methods.

Navigation of frameless fixation for gamma\xa0knife radiosurgery using fixed augmented reality

Augmented reality (AR) offers a new medical treatment approach. We aimed to evaluate frameless (mask) fixation navigation using a 3D-printed patient model with fixed-AR technology for gamma knife radiosurgery (GKRS). Fixed-AR navigation was developed using the inside-out method with visual inertial odometry algorithms, and the flexible Quick Response marker was created for object-feature recognition. Virtual 3D-patient models for AR-rendering were created via 3D-scanning utilizing TrueDepth and cone-beam computed tomography (CBCT) to generate a new GammaKnife Icon™ model. A 3D-printed patient model included fiducial markers, and virtual 3D-patient models were used to validate registration accuracy. Registration accuracy between initial frameless fixation and re-fixation navigated fixed-AR was validated through visualization and quantitative method. The quantitative method was validated through set-up errors, fiducial marker coordinates, and high-definition motion management (HDMM) values. A 3D-printed model and virtual models were correctly overlapped under frameless fixation. Virtual models from both 3D-scanning and CBCT were enough to tolerate the navigated frameless re-fixation. Although the CBCT virtual model consistently delivered more accurate results, 3D-scanning was sufficient. Frameless re-fixation accuracy navigated in virtual models had mean set-up errors within 1 mm and 1.5° in all axes. Mean fiducial marker differences from coordinates in virtual models were within 2.5 mm in all axes, and mean 3D errors were within 3 mm. Mean HDMM difference values in virtual models were within 1.5 mm of initial HDMM values. The variability from navigation fixed-AR is enough to consider repositioning frameless fixation without CBCT scanning for treating patients fractionated with large multiple metastases lesions (> 3 cm) who have difficulty enduring long beam-on time. This system could be applied to novel GKRS navigation for frameless fixation with reduced preparation time.

Robust and efficient edge-based visual odometry

Visual odometry, which aims to estimate relative camera motion between sequential video frames, has been widely used in the fields of augmented reality, virtual reality, and autonomous driving. However, it is still quite challenging for state-of-the-art approaches to handle low-texture scenes. In this paper, we propose a robust and efficient visual odometry algorithm that directly utilizes edge pixels to track camera pose. In contrast to direct methods, we choose reprojection error to construct the optimization energy, which can effectively cope with illumination changes. The distance transform map built upon edge detection for each frame is used to improve tracking efficiency. A novel weighted edge alignment method together with sliding window optimization is proposed to further improve the accuracy. Experiments on public datasets show that the method is comparable to state-of-the-art methods in terms of tracking accuracy, while being faster and more robust.