Dual-view 3D object recognition and detection via Lidar point cloud and camera image

Abstract

When it comes to the accuracy of autonomous motion, it is necessary to consider object detection and recognition, especially for the robot application of the complex environment. This paper investigates novel dual-view 3D object detection networks combined with the Lidar point cloud and RGB image in engineering scenarios. The developed system is applied for autonomous vehicles that the detected objects are cars, cyclists, and pedestrians. Firstly, a feature extraction network based on the residual module is presented, and the specific features are from the RGB image. The point cloud is transformed into Bird’s Eye View (BEV), and the BEV feature extraction network is built based on sparse convolution. Besides, the feature maps are input into the region proposal network (RPN) to obtain the optimal proposal so that the object classification and the bounding box regression are obtained. Finally, to evaluate the flexibility of the developed framework, extensive data sets are generated through the CARLA simulator and verified on the KITTI data set and unmanned motion platform (BIT-NAZA robot), indicating that the proposed networks can achieve satisfactory performance in the real-world scenario.