Monocular Visual Odometry System Research Articles

Cost-effective visual odometry system for vehicle motion control in agricultural environments

In precision agriculture, innovative cost-effective technologies and new improved solutions, aimed at making operations and processes more reliable, robust and economically viable, are still needed. In this context, robotics and automation play a crucial role, with particular reference to unmanned vehicles for crop monitoring and site-specific operations. However, unstructured and irregular working environments, such as agricultural scenarios, require specific solutions regarding positioning and motion control of autonomous vehicles.In this paper, a reliable and cost-effective monocular visual odometry system, properly calibrated for the localisation and navigation of tracked vehicles on agricultural terrains, is presented. The main contribution of this work is the design and implementation of an enhanced image processing algorithm, based on the cross-correlation approach. It was specifically developed to use a simplified hardware and a low complexity mechanical system, without compromising performance. By providing sub-pixel results, the presented algorithm allows to exploit low-resolution images, thus obtaining high accuracy in motion estimation with short computing time. The results, in terms of odometry accuracy and processing time, achieved during the in-field experimentation campaign on several terrains proved the effectiveness of the proposed method and its fitness for automatic control solutions in precision agriculture applications.

Using Unsupervised Deep Learning Technique for Monocular Visual Odometry

Deep learning technique-based visual odometry systems have recently shown promising results compared to feature matching-based methods. However, deep learning-based systems still require the ground truth poses for training and the additional knowledge to obtain absolute scale from monocular images for reconstruction. To address these issues, this paper presents a novel visual odometry system based on a recurrent convolutional neural network. The system employs an unsupervised end-to-end training approach. The depth information of scenes is used alongside monocular images to train the network in order to inject scale. Poses are inferred only from monocular images, thus making the proposed visual odometry system a monocular one. The experiments are conducted and the results show that the proposed method performs better than other monocular visual odometry systems. This paper has made two main contributions: 1) the creation of the unsupervised training framework in which the camera ground truth poses are only deployed for system performance evaluation rather than for training and 2) the absolute scale could be recovered without the post-processing of poses.

A Novel Resection–Intersection Algorithm With Fast Triangulation Applied to Monocular Visual Odometry

This paper proposes a new technique for two-view camera pose estimation applied to vehicular monocular visual odometry. The proposed method is based on the resection–intersection concept using Gauss–Newton non-linear regression to minimize projection residuals. Different from other solutions, the proposed method does not suppose fixed positions of points during resection but rather takes into account the adjustment in the 3-D position of points when calculating the new camera pose increment. A simplified fast two-view triangulation method is used to mitigate the cost of intense 3-D point position recalculation. The method is integrated into a monocular visual odometry solution and tested on the KITTI public dataset. The final system does not use loop closing or key frame selection, and RANSAC is replaced by sequential backward selection. The resection–intersection method presents robustness to errors in initial camera pose and rapid convergence rate, reaching the final camera pose in up to three iterations in 65% of the cases. A single thread Python/OpenCV implementation of the monocular visual odometry system on an Intel Core i7 spent around 75 ms per frame, of which one third was used in the resection–intersection step. The achieved results presented better accuracy than all other published monocular odometry works in the KITTI benchmark, in a simple solution using only two consecutive frames for camera pose estimation.

Single-camera visual odometry to track a surgical X-ray C-arm base.

This study provides a framework for a single-camera odometry system for localizing a surgical C-arm base. An application-specific monocular visual odometry system (a downward-looking consumer-grade camera rigidly attached to the C-arm base) is proposed in this research. The cumulative dead-reckoning estimation of the base is extracted based on frame-to-frame homography estimation. Optical-flow results are utilized to feed the odometry. Online positional and orientation parameters are then reported. Positional accuracy of better than 2% (of the total traveled distance) for most of the cases and 4% for all the cases studied and angular accuracy of better than 2% (of absolute cumulative changes in orientation) were achieved with this method. This study provides a robust and accurate tracking framework that not only can be integrated with the current C-arm joint-tracking system (i.e. TC-arm) but also is capable of being employed for similar applications in other fields (e.g. robotics).

Estimation of image scale variations in monocular visual odometry systems

Reliable estimation of the vehicle position is the main prerequisite in all autonomous mobile robotic applications. Image‐scale uncertainty in correlation‐based monocular visual odometry systems negatively affects the accuracy of the vehicle motion estimation. This paper presents the development of a new technique and algorithm to estimate image‐scale variations due to camera height fluctuations when the vehicle is driven on uneven terrains or when the height of vehicle from ground changes as a result of changes in load or number of passengers in the vehicle. This technique depends on marking the image frames by two red laser points, as independent reference points, which have a certain distance between them. The image‐scale variations can be estimated by monitoring the variations in the distance between these two reference points. The proposed technique eliminates the need for camera recalibration and the use of sensors to measure the variations of camera height from ground, such as laser range finders and acceleration sensors. The developed system uses a single downward‐facing monocular camera supported by a lighting module and installed underneath the test vehicle to avoid the negative effect of directional sunlight and shadows, which can disturb the correlation. Indoor and outdoor experiments have proven the efficiency of the suggested technique in resolving image‐scale uncertainty and ensuring an image‐scale‐invariant correlation‐based matching, with only less than 5% additional computational time. © 2016 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Adaptive‐search template matching technique based on vehicle acceleration for monocular visual odometry system

One of the important tasks of an autonomous mobile vehicle is the reliable and fast estimation of its position over time. This paper presents the development of an adaptive technique to hasten and improve the quality of correlation‐based template matching for monocular visual odometry systems that estimate the relative motion of ground vehicles in low‐textured environments. Moreover, the factors that can affect the maximum permissible vehicle driving speed were determined and the related equations were derived. The developed system uses a single downward‐facing monocular camera installed at an optimum location to avoid the negative effect of directional sunlight and shadows which can disturb the correlation. In addition, the normalized cross‐correlation method is implemented to calculate the pixel displacement between image frames. Although this method is highly effective for template matching because of its invariance to linear brightness and contrast variations, it incurs high computational cost. Thus, the optimal sizes of image template and matching search area are selected and their locations are dynamically changed according to vehicle acceleration, in order to achieve a compromise between the performance and the computational cost of correlation. The proposed technique increases the allowable vehicle driving speed and reduces the probability of template false‐matching. Moreover, compared to traditional full search matching techniques, the adaptive technique demonstrates high efficiency and accuracy and improves the quality and speed of the correlation with more than 87% of reduction in computational cost. © 2016 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

High Accuracy Monocular SFM and Scale Correction for Autonomous Driving.

We present a real-time monocular visual odometry system that achieves high accuracy in real-world autonomous driving applications. First, we demonstrate robust monocular SFM that exploits multithreading to handle driving scenes with large motions and rapidly changing imagery. To correct for scale drift, we use known height of the camera from the ground plane. Our second contribution is a novel data-driven mechanism for cue combination that allows highly accurate ground plane estimation by adapting observation covariances of multiple cues, such as sparse feature matching and dense inter-frame stereo, based on their relative confidences inferred from visual data on a per-frame basis. Finally, we demonstrate extensive benchmark performance and comparisons on the challenging KITTI dataset, achieving accuracy comparable to stereo and exceeding prior monocular systems. Our SFM system is optimized to output pose within 50 ms in the worst case, while average case operation is over 30 fps. Our framework also significantly boosts the accuracy of applications like object localization that rely on the ground plane.

Automating visual inspection of pipes used for natural gas production

Abstract Qatar is a worldwide leader in liquid natural gas (LNG) production and is poised to lead the world in gas-to-liquids (GTL) production with the commissioning of the Pearl GTL facility. Unfortunately, Qatar's gas fields contain non-negligible quantities of corrosive and toxic hydrogen sulfide (H2S), resulting in the ongoing need for expensive and labor intensive pipe inspection to detect and monitor areas of corrosion. Such inspection is critical to plant integrity, worker safety, and to ensure the economic productivity of the facility. Current industry practice relies on manual sensors operated by a worker located externally to the pipe. The complex pipe geometries and sheer number of pipes, result in a sparse inspection process that forces inspectors to extrapolate measurements to large areas of the pipe network that are unseen. To overcome these limitations, we are pursuing a radically different approach that uses an articulated robot to navigate inside the pipe, combined with a vision-based perception system that can build a detailed, registered, high resolution 3D appearance map of the inside pipe surface. By using an articulated robot, we can significantly increase the direct measurement coverage of the pipe network. By using a vision-based perception system, we can build models for visualization of the inside pipe surface that can be directly evaluated for corrosion damage. Moreover, our approach lays the foundation for automating corrosion detection by enabling changes in co-registered multi-sensor fusion (e.g. using magnetic flux leakage) to be evaluated over time. Our work to date has focused on developing monocular and stereo visual odometry systems, which are the core component to building high resolution 3D appearance maps of the pipe surface from a robot crawler located inside the pipe. We have developed algorithms that take imagery collected from a robot moving inside the pipe, and are able to estimate the motion of the vehicle and the resulting structure and appearance of the pipe surface. We have evaluated our algorithms on pipe segments and have generated accurate, high resolution stitched images of the internal pipe surface. We will describe the details of our algorithms, current results, and next steps in our work.