Visual Odometry Algorithm Research Articles

Stereo-Based Visual Odometry for Autonomous Robot Navigation

Mobile robots should possess accurate self-localization capabilities in order to be successfully deployed in their environment. A solution to this challenge may be derived from visual odometry (VO), which is responsible for estimating the robot's pose by analysing a sequence of images. The present paper proposes an accurate, computationally-efficient VO algorithm relying solely on stereo vision images as inputs. The contribution of this work is twofold. Firstly, it suggests a non-iterative outlier detection technique capable of efficiently discarding the outliers of matched features. Secondly, it introduces a hierarchical motion estimation approach that produces refinements to the global position and orientation for each successive step. Moreover, for each subordinate module of the proposed VO algorithm, custom non-iterative solutions have been adopted. The accuracy of the proposed system has been evaluated and compared with competent VO methods along DGPS-assessed benchmark routes. Experimental results of relevance to rough terrain routes, including both simulated and real outdoors data, exhibit remarkable accuracy, with positioning errors lower than 2%.

Quantitative Evaluation of Stereo Visual Odometry for Autonomous Vessel Localisation in Inland Waterway Sensing Applications.

Autonomous survey vessels can increase the efficiency and availability of wide-area river environment surveying as a tool for environment protection and conservation. A key challenge is the accurate localisation of the vessel, where bank-side vegetation or urban settlement preclude the conventional use of line-of-sight global navigation satellite systems (GNSS). In this paper, we evaluate unaided visual odometry, via an on-board stereo camera rig attached to the survey vessel, as a novel, low-cost localisation strategy. Feature-based and appearance-based visual odometry algorithms are implemented on a six degrees of freedom platform operating under guided motion, but stochastic variation in yaw, pitch and roll. Evaluation is based on a 663 m-long trajectory (>15,000 image frames) and statistical error analysis against ground truth position from a target tracking tachymeter integrating electronic distance and angular measurements. The position error of the feature-based technique (mean of ±0.067 m) is three times smaller than that of the appearance-based algorithm. From multi-variable statistical regression, we are able to attribute this error to the depth of tracked features from the camera in the scene and variations in platform yaw. Our findings inform effective strategies to enhance stereo visual localisation for the specific application of river monitoring.

Image-Aided Navigation Using Cooperative Binocular Stereopsis

This paper proposes a novel method for estimating the positions of two vehicles in a global reference frame based on synchronized image and navigational information. The proposed technique leverages one vehicle's ability to localize itself relative to another using image data, enabling motion estimation from tracking common features. The visual odometry algorithm of this work uses the optimal vehicle motion over a single time interval estimated from the positions of common features in a bundle adjustment algorithm as a delayed state extended Kalman filter (EKF) measurement. The algorithm achieves accurate motion estimation and is a potential alternative to map-based simultaneous localization and mapping (SLAM) algorithms. Published 2015 This article is a U.S. Government work and is in the public domain in the USA.

Fusion of visual odometry and inertial navigation system on a smartphone

The paper presents the monocular visual odometry, inertial navigation system and the fusion of both these localization approaches. The visual odometry algorithm consists of four other algorithms, namely the camera calibration algorithm, KLT algorithm, algorithm for the estimation of rigid transformation and RANSAC algorithm. The inertial navigation system is based on a pedometer and digital compass. Both visual odometry and the inertial navigation system can determine the incremental movements and the positions of a robot or a pedestrian according to the world coordinate system. In order to get an even more robust and accurate localization system, the advantages of each mentioned localization approaches were combined by using the Extended Kalman Filter. The algorithms were fully implemented on a smartphone, where they were divided into several threads that could be performed simultaneously on multiple processor cores. The proposed system, which fuses information from the camera and inertial sensors, can convert the smartphone into a powerful mobile sensor unit or the so-called virtual sensor that returns relative position in relation to the starting point. This virtual sensor can be used as an advanced sensor unit on mobile robots or as part of a smartphone application which requires personal navigation system. The operation of the localization system is proved by experimental results which were obtained by attaching a smartphone on a pedestrian who walked along the reference trajectory drawn on the floor. In the experiments the described system showed big potential in many aspects since very good results were obtained.

Verification of visual odometry algorithms with an OpenGL-based software tool

We present a software tool called a stereovision egomotion sequence generator that was developed for testing visual odometry (VO) algorithms. Various approaches to single and multicamera VO algorithms are reviewed first, and then a reference VO algorithm that has served to demonstrate the program’s features is described. The program offers simple tools for defining virtual static three-dimensional scenes and arbitrary six degrees of freedom motion paths within such scenes and output sequences of stereovision images, disparity ground-truth maps, and segmented scene images. A simple script language is proposed that simplifies tests of VO algorithms for user-defined scenarios. The program’s capabilities are demonstrated by testing a reference VO technique that employs stereoscopy and feature tracking.

Cloud-Based Collaborative 3D Mapping in Real-Time With Low-Cost Robots

This paper presents an architecture, protocol, and parallel algorithms for collaborative 3D mapping in the cloud with low-cost robots. The robots run a dense visual odometry algorithm on a smartphone-class processor. Key-frames from the visual odometry are sent to the cloud for parallel optimization and merging with maps produced by other robots. After optimization the cloud pushes the updated poses of the local key-frames back to the robots. All processes are managed by Rapyuta, a cloud robotics framework that runs in a commercial data center. This paper includes qualitative visualization of collaboratively built maps, as well as quantitative evaluation of localization accuracy, bandwidth usage, processing speeds, and map storage.

Monocular Visual Odometry on a Smartphone

The paper presents the monocular visual odometry, which is the sequential estimation process of camera motions depending on the perceived movements of pixels in the image sequence. The visual odometry accurately determines the incremental movements and the positions of the camera according to the world coordinate system. The algorithm consists of four other algorithms, namely camera calibration algorithm, KLT algorithm, algorithm for estimation of rigid transformation and RANSAC algorithm. Since the visual odometry algorithm was developed with the purpose to be a part of an indoor localization application for pedestrians and especially the blind, it was fully implemented on a smartphone.

基于地面特征的移动机器人单目视觉里程计算法

基于地面特征的移动机器人单目视觉里程计算法

Error Analysis for Visual Odometry on Indoor, Wheeled Mobile Robots With 3-D Sensors

The objective of this paper is to improve the visual odometry performance through the analysis of the sensor noise and the propagation of an error through the entire visual odometry system. The visual odometry algorithm is implemented on an indoor, wheeled mobile robot (WMR) constrained to planar motion, and uses an integrated color-depth (RGB-D) camera, and a one-point (1-pt), 3 degree-of-freedom inverse kinematic solution, enabling a closed-form bound on the propagated error. There are three main contributions of this paper. First, feature location errors for the RGB-D camera are quantified. Second, these feature location errors are propagated through the entire visual odometry algorithm. Third, the visual odometry performance is improved by using the predicted error to weight individual 1-pt solutions. The error bounds and the improved visual odometry scheme are experimentally verified on a WMR. Using the error-weighting scheme, the proposed visual odometry algorithm achieves the performance of approximately $\hbox{1.5}\%$ error, without the use of iterative, outlier-rejection tools.

RGB-D Indoor Plane-based 3D-Modeling using Autonomous Robot

Abstract. 3D model of indoor environments provide rich information that can facilitate the disambiguation of different places and increases the familiarization process to any indoor environment for the remote users. In this research work, we describe a system for visual odometry and 3D modeling using information from RGB-D sensor (Camera). The visual odometry method estimates the relative pose of the consecutive RGB-D frames through feature extraction and matching techniques. The pose estimated by visual odometry algorithm is then refined with iterative closest point (ICP) method. The switching technique between ICP and visual odometry in case of no visible features suppresses inconsistency in the final developed map. Finally, we add the loop closure to remove the deviation between first and last frames. In order to have a semantic meaning out of 3D models, the planar patches are segmented from RGB-D point clouds data using region growing technique followed by convex hull method to assign boundaries to the extracted patches. In order to build a final semantic 3D model, the segmented patches are merged using relative pose information obtained from the first step.

Incorporating a wheeled vehicle model in a new monocular visual odometry algorithm for dynamic outdoor environments.

This paper presents a monocular visual odometry algorithm that incorporates a wheeled vehicle model for ground vehicles. The main innovation of this algorithm is to use the single-track bicycle model to interpret the relationship between the yaw rate and side slip angle, which are the two most important parameters that describe the motion of a wheeled vehicle. Additionally, the pitch angle is also considered since the planar-motion hypothesis often fails due to the dynamic characteristics of wheel suspensions and tires in real-world environments. Linearization is used to calculate a closed-form solution of the motion parameters that works as a hypothesis generator in a RAndom SAmple Consensus (RANSAC) scheme to reduce the complexity in solving equations involving trigonometric. All inliers found are used to refine the winner solution through minimizing the reprojection error. Finally, the algorithm is applied to real-time on-board visual localization applications. Its performance is evaluated by comparing against the state-of-the-art monocular visual odometry methods using both synthetic data and publicly available datasets over several kilometers in dynamic outdoor environments.

A temporally consistent grid-based visual odometry framework for multi-core architectures

Most recent visual odometry algorithms based on sparse feature matching are computationally efficient methods that can be executed in real time on desktop computers. However, further efforts are required to reduce computational complexity in order to integrate these solutions in embedded platforms with low power consumption. This paper presents a spacetime framework that can be applied to most stereo visual odometry algorithms greatly reducing their computational complexity. Moreover, it enables exploiting multi-core architectures available in most modern computing platforms. According to the tests performed on publicly available datasets and an experimental driverless car, the proposed framework significantly reduces the computational complexity of a visual odometry algorithm while improving the accuracy of the results.

1-Point Ransac Based Robust Visual Odometry

Many of the current visual odometry algorithms suffer from some extreme limitations such as requiring a high amount of computation time, complex algorithms, and not working in urban environments. In this paper, we present an approach that can solve all the above problems using a single camera. Using a planar motion assumption and Ackermann's principle of motion, we construct the vehicle's motion model as a circular planar motion (2DOF). Then, we adopt a 1-point method to improve the Ransac algorithm and the relative motion estimation. In the Ransac algorithm, we use a 1-point method to generate the hypothesis and then adopt the Levenberg-Marquardt method to minimize the geometric error function and verify inliers. In motion estimation, we combine the 1-point method with a simple least-square minimization solution to handle cases in which only a few feature points are present. The 1-point method is the key to speed up our visual odometry application to real-time systems. Finally, a Bundle Adjustment algorithm is adopted to refine the pose estimation. The results on real datasets in urban dynamic environments demonstrate the effectiveness of our proposed algorithm.

Towards lighting-invariant visual navigation: An appearance-based approach using scanning laser-rangefinders

In an effort to facilitate lighting-invariant exploration, this paper presents an appearance-based approach using 3D scanning laser-rangefinders for two core visual navigation techniques: visual odometry (VO) and visual teach and repeat (VT&R). The key to our method is to convert raw laser intensity data into greyscale camera-like images, in order to apply sparse, appearance-based techniques traditionally used with camera imagery. The novel concept of an image stack is introduced, which is an array of azimuth, elevation, range, and intensity images that are used to generate keypoint measurements and measurement uncertainties. Using this technique, we present the following four experiments. In the first experiment, we explore the stability of a representative keypoint detection/description algorithm on camera and laser intensity images collected over a 24 h period outside. In the second and third experiments, we validate our VO algorithm using real data collected outdoors with two different 3D scanning laser-rangefinders. Lastly, our fourth experiment presents promising preliminary VT&R localization results, where the teaching phase was done during the day and the repeating phase was done at night. These experiments show that it possible to overcome lighting sensitivity encountered with cameras, yet continue to exploit the heritage of the appearance-based visual odometry pipeline.

Semi-parametric learning for visual odometry

This paper addresses the visual odometry problem from a machine learning perspective. Optical flow information from a single camera is used as input for a multiple-output Gaussian process (MOGP) framework, that estimates linear and angular camera velocities. This approach has several benefits. (1) It substitutes the need for conventional camera calibration, by introducing a semi-parametric model that is able to capture nuances that a strictly parametric geometric model struggles with. (2) It is able to recover absolute scale if a range sensor (e.g. a laser scanner) is used for ground-truth, provided that training and testing data share a certain similarity. (3) It is naturally able to provide measurement uncertainties. We extend the standard MOGP framework to include the ability to infer joint estimates (full covariance matrices) for both translation and rotation, taking advantage of the fact that all estimates are correlated since they are derived from the same vehicle. We also modify the common zero mean assumption of a Gaussian process to accommodate a standard geometric model of the camera, thus providing an initial estimate that is then further refined by the non-parametric model. Both Gaussian process hyperparameters and camera parameters are trained simultaneously, so there is still no need for traditional camera calibration, although if these values are known they can be used to speed up training. This approach has been tested in a wide variety of situations, both 2D in urban and off-road environments (two degrees of freedom) and 3D with unmanned aerial vehicles (six degrees of freedom), with results that are comparable to standard state-of-the-art visual odometry algorithms and even more traditional methods, such as wheel encoders and laser-based Iterative Closest Point. We also test its limits to generalize over environment changes by varying training and testing conditions independently, and also by changing cameras between training and testing.

A Kinect-based natural interface for quadrotor control

This paper presents a new and challenging approach to the control of mobile platforms. Natural user interfaces (NUIs) and visual computing techniques are used to control the navigation of a quadrotor in GPS-denied indoor environments. A visual odometry algorithm allows the platform to autonomously navigate the environment, whereas the user can control complex manoeuvres by gestures and body postures. This approach makes the human–computer interaction (HCI) more intuitive, usable, and receptive to the user’s needs: in other words, more user-friendly and, why not, fun. The NUI presented in this paper is based on the Microsoft Kinect and users can customize the association among gestures/postures and platform commands, thus choosing the more intuitive and effective interface.

Driving Control of Planetary Rover Based on Slip Ratio Estimation

Slip-sinkage is one of the main factors influencing the driving safety of planetary rover in soft terrain. In the environment of scarce features, based on the stereo visual sensors, the visual odometry algorithm is firstly developed to calculate the position changes of planetary rover with six degrees of freedom, which combine with the wheel speed sensor achieve the estimation of wheel slip ratios. Then, on the basis of mechanical analysis of soft terrain, choosing the slip ratio as the state variable, the coordinated driving control algorithm is designed. Simulation and experiment results demonstrate the effectiveness of the proposed control algorithm.

Visual odometry for road vehicles—feasibility analysis

Estimating the global position of a road vehicle without using GPS is a challenge that many scientists look forward to solving in the near future. Normally, inertial and odometry sensors are used to complement GPS measures in an attempt to provide a means for maintaining vehicle odometry during GPS outage. Nonetheless, recent experiments have demonstrated that computer vision can also be used as a valuable source to provide what can be denoted as visual odometry. For this purpose, vehicle motion can be estimated using a non-linear, photogrametric approach based on RAndom SAmple Consensus (RANSAC). The results prove that the detection and selection of relevant feature points is a crucial factor in the global performance of the visual odometry algorithm. The key issues for further improvement are discussed in this letter.

Two years of Visual Odometry on the Mars Exploration Rovers

AbstractNASA's two Mars Exploration Rovers (MER) have successfully demonstrated a robotic Visual Odometry capability on another world for the first time. This provides each rover with accurate knowledge of its position, allowing it to autonomously detect and compensate for any unforeseen slip encountered during a drive. It has enabled the rovers to drive safely and more effectively in highly sloped and sandy terrains and has resulted in increased mission science return by reducing the number of days required to drive into interesting areas. The MER Visual Odometry system comprises onboard software for comparing stereo pairs taken by the pointable mast‐mounted 45 deg FOV Navigation cameras (NAVCAMs). The system computes an update to the 6 degree of freedom rover pose (x, y, z, roll, pitch, yaw) by tracking the motion of autonomously selected terrain features between two pairs of 256×256 stereo images. It has demonstrated good performance with high rates of successful convergence (97% on Spirit, 95% on Opportunity), successfully detected slip ratios as high as 125%, and measured changes as small as 2 mm, even while driving on slopes as high as 31 deg. Visual Odometry was used over 14% of the first 10.7 km driven by both rovers. During the first 2 years of operations, Visual Odometry evolved from an “extra credit” capability into a critical vehicle safety system. In this paper we describe our Visual Odometry algorithm, discuss several driving strategies that rely on it (including Slip Checks, Keep‐out Zones, and Wheel Dragging), and summarize its results from the first 2 years of operations on Mars. © 2006 Wiley Periodicals, Inc.