Stereo priori RCNN based car detection on point level for autonomous driving

Abstract

Binocular vision target detection algorithms generally require selection of a large number of keypoints, which result in a heavy computational effort for online calculation and the lack of ability to utilize spatial semantic information to full advantage. This paper proposed a stereo priori RCNN based car detection method on point level for autonomous driving. The algorithm combines traditional Region Proposal Network (RPN) with a Mask-branch mechanism, which improves the accuracy of 3D target detection through minimizing luminosity errors, and employs RGB images to provide semantic information for spatial point-clouds. Firstly, the proposed algorithm obtains bounding boxes of left and right images through RPN and classification networks. Then, the prior information of vehicles and branched convolutional neural networks are used to extract wheel features from a feature map. Therefore, the coordinates and orientation of vehicles on bird eye map can be fitted. Afterward, by predicting the keypoints, a 3D bounding box of each vehicle is roughly restored. Then Region of Interest (ROI) is applied on both sides of left and right cameras to minimize the photometric error, so that the precise position and the size of the detection frame are obtained. In the meantime, a Mask-branch mechanism is adopted to achieve a precise semantic segmentation of each ROI. Finally, a 3D bounding box of vehicles is used for point-cloud segmentation, and the semantic information provided by the Mask-branch is exploited to improve the segmentation accuracy. Extensive numerical experiments are conducted on the Kitti dataset and nuScenes dataset to demonstrate the efficiency and effectiveness of the proposed algorithm.