Obstacle Detection by Fusing Point Clouds and Monocular Image

Abstract

Obstacle detection is a significant and fundamental issue in autonomous driving and robotics. This paper proposes a novel method to locate obstacles in the scene by comprehensively utilizing the sparse point clouds captured by a Lidar and the natural image taken from a camera. We do this because Lidar is able to capture the data accurately, while the object details can be perfectly preserved by an image. To establish the depth map, the proposed method firstly uses cross-calibration to align the point clouds with reference image. Then we introduce the common U-disparity map in the stereo vision to deal with this depth map and extract all the points belonging to obstacles. By employing 3D point coordinates and the pairwise image pixels as input, the features for density based clustering technique are learned from a specific sparse regression model. After adopting the clustering technique, the corresponding obstacles can be localized by a subset of the relevant points. Quantitative and qualitative experimental results on KITTI object detection benchmark reveal that our proposed method achieves very encouraging performances in various practical environments.