Robust RGB-D odometry under depth uncertainty for structured environments

Abstract

Visual odometry, the process of tracking the trajectory of a moving camera based on its captured video is a fundamental problem behind autonomous mobile robotics and augmented reality applications. Yet, despite almost 40 years of extensive research on the problem, state-of-the-art systems are still vulnerable to several pitfalls that arise in challenging environments due to specific sensor limitations and restrictive assumptions. This thesis, in particular, investigates the use of RGB-D cameras for robust visual odometry in man-made environments, such as industrial plants. These spaces, contrary to natural environments, follow mainly a rectilinear structure made of simple geometric entities. Thus, this work exploits this structure by taking a feature-based approach, where lines, planes and cylinder segments are explicitly extracted as visual cues for egomotion estimation. While the depth captured by RGB-D cameras helps to resolve the ambiguity inherent of passive cameras especially on uniform and low textured surfaces, these active cameras suffer from several limitations, which may deteriorate the performance of RGB-D Odometry, such as, limited operating range, near-infrared light interference and systematic errors, leading to incomplete and noisy depth maps. To address these issues, we have first developed a visual odometry framework that leverages both depth measurements from active sensing and depth estimates from temporal stereo obtained via probabilistic filtering. Our experiments demonstrate that this framework is able to operate on large indoor and outdoor spaces, where the absence and inaccuracy of depth measurements is too high to rely just on RGB-D Odometry. Secondly, this thesis considers the depth sensor error by proposing a depth fusion framework based on Mixture of Gaussians to denoise the depth measurements and model their uncertainties through spatio-temporal observations. Extensive results on RGB-D sequences show that applying this depth model to RGB-D odometry improves significantly its performance and supports our hypothesis that the uncertainty of fused depth needs to be exposed. To fully exploit this probabilistic depth model, the depth uncertainty needs to be propagated throughout the visual odometry pipeline. Therefore, we reformulated the visual odometry system as a probabilistic process by (i) deriving plane and 3D line fitting solutions that model the uncertainties of the feature parameters and (ii) estimating the camera pose by combining different feature-type matches weighted by their respective uncertainties. Lastly, this thesis addresses man-made environments made also of smooth curved surfaces by proposing a curve-aware plane and cylinder extraction algorithm which is shown empirically to be more efficient and accurate than an alternative state-of-the-art plane extraction approach, leading ultimately to better visual odometry performance in scenes made of cylindrical surfaces. To incorporate this feature extractor in visual odometry, the system described above is extended to handle cylinder primitives.