Monocular Obstacle Detection Research Articles

Image-Based Obstacle Detection Methods for the Safe Navigation of Unmanned Vehicles: A Review

Mobile robots lack a driver or a pilot and, thus, should be able to detect obstacles autonomously. This paper reviews various image-based obstacle detection techniques employed by unmanned vehicles such as Unmanned Surface Vehicles (USVs), Unmanned Aerial Vehicles (UAVs), and Micro Aerial Vehicles (MAVs). More than 110 papers from 23 high-impact computer science journals, which were published over the past 20 years, were reviewed. The techniques were divided into monocular and stereo. The former uses a single camera, while the latter makes use of images taken by two synchronised cameras. Monocular obstacle detection methods are discussed in appearance-based, motion-based, depth-based, and expansion-based categories. Monocular obstacle detection approaches have simple, fast, and straightforward computations. Thus, they are more suited for robots like MAVs and compact UAVs, which usually are small and have limited processing power. On the other hand, stereo-based methods use pair(s) of synchronised cameras to generate a real-time 3D map from the surrounding objects to locate the obstacles. Stereo-based approaches have been classified into Inverse Perspective Mapping (IPM)-based and disparity histogram-based methods. Whether aerial or terrestrial, disparity histogram-based methods suffer from common problems: computational complexity, sensitivity to illumination changes, and the need for accurate camera calibration, especially when implemented on small robots. In addition, until recently, both monocular and stereo methods relied on conventional image processing techniques and, thus, did not meet the requirements of real-time applications. Therefore, deep learning networks have been the centre of focus in recent years to develop fast and reliable obstacle detection solutions. However, we observed that despite significant progress, deep learning techniques also face difficulties in complex and unknown environments where objects of varying types and shapes are present. The review suggests that detecting narrow and small, moving obstacles and fast obstacle detection are the most challenging problem to focus on in future studies.

Testing the reliability of monocular obstacle detection methods in a simulated 3D factory environment

Automated driving in public traffic still faces many technical and legal challenges. However, automating vehicles at low speeds in controlled industrial environments is already achievable today. A reliable obstacle detection is mandatory to prevent accidents. Recent advances in convolutional neural network-based algorithms hypothetically allow the replacement of distance measuring laser scanners with common monocameras. In this paper, we present a photorealistic 3D simulated factory environment for testing vision-based obstacle detecting algorithms preceding field tests on the safety–critical system. We further test two obstacle detection methods employing state-of-the-art semantic segmentation and depth estimation in a range of challenging test scenarios. Both models performed well under common factory settings. Some edge cases, however, lead to vehicle crashes.

Efficient obstacle detection based on prior estimation network and spatially constrained mixture model for unmanned surface vehicles

AbstractRecently, spatially constrained mixture model has become the mainstream method for the task of vision‐based obstacle detection in unmanned surface vehicles (USVs), and has shown its potential of modeling the semantic structure of the marine environment. However, the expectation maximization (EM) optimization of this model is quite sensitive to initial values and easily falls into a local optimal solution in the presence of significant rolling and pitching in rough seas. In addition, existing methods based on spatially constrained mixture model are susceptible to false positives in the presence of sun glitter. In this paper, a prior estimation network (PEN) is proposed to improve the mixture model, which together enable reliable monocular obstacle detection for USVs. We develop a weakly supervised E‐step to train the PEN for learning the semantic structure of marine images and estimating initial class priors in obstacle detection. To mitigate the influence of poor initial parameters on the convergence of EM optimization, we use the priors estimated by the PEN to calculate the initial parameters of the mixture model and automatically adjust the hyper priors on the semantic components in the mixture model. The output of the PEN is also applied to set the probability values of the outlier component in the mixture model, aiming to reduce false positives caused by sun glitter. Experimental results show that our approach outperforms the current state‐of‐the‐art monocular method by 15% improvement in sea edge estimation and a 3.3% increase in F‐score on the marine obstacle detection data set, as well as 69.5% improvement in sea edge estimation and a 39.2% increase in F‐score on our data set, while running over 40 fps.

Monocular obstacle detection using reciprocal-polar rectification

Our obstacle detection method is applicable to deliberative translation motion of a mobile robot and, in such motion, the epipole of each image of an image pair is coincident and termed the focus of expansion (FOE). We present an accurate method for computing the FOE and then we use this to apply a novel rectification to each image, called a reciprocal-polar (RP) rectification. When robot translation is parallel to the ground, as with a mobile robot, ground plane image motion in RP-space is a pure shift along an RP image scan line and hence can be recovered by a process of 1D correlation, even over large image displacements and without the need for corner matches. Furthermore, we show that the magnitude of these shifts follows a sinusoidal form along the second (orientation) dimension of the RP image. This gives the main result that ground plane motion over RP image space forms a 3D sinusoidal manifold. Simultaneous ground plane pixel grouping and recovery of the ground plane motion thus amounts to finding the FOE and then robustly fitting a 3D sinusoid to shifts of maximum correlation in RP space. The phase of the recovered sinusoid corresponds to the orientation of the vanishing line of the ground plane and the amplitude is related to the magnitude of the robot/camera translation. Recovered FOE, vanishing line and sinusoid amplitude fully define the ground plane motion (homography) across a pair of images and thus obstacles and ground plane can be segmented without any explicit knowledge of either camera parameters or camera motion.