Visual-LiDAR Based 3D Object Detection and Tracking for Embedded Systems

Abstract

In recent years, persistent news updates on autonomous vehicles and the claims of companies entering the space, brace the notion that vehicular autonomy of level 5 is just around the corner. However, the main hindrance in asserting the full autonomy still boils down to environmental perception that affects the autonomous decisions. An efficient perceptual system requires redundancy in sensor modalities capable of performing in varying environmental conditions, and providing a reliable information using limited computational resources. This work addresses the task of 3D object detection and tracking in the vehicles’ environment, using camera and 3D LiDAR as primary sensors. The proposed framework is designed to operate in an embedded system that visually classifies the objects using a lightweight neural network, while tracking is performed in 3D space using LiDAR information. The main contribution of this work is 3D LiDAR point cloud classification using visual object detector, and an IMM-UKF-JPDAF based object tracker that jointly performs 3D object detection and tracking. The performance evaluation is carried out using MOT16 metrics and ground truths provided by KITTI Datasets. Furthermore, the proposed tracker is evaluated and compared with state-of-the-art approaches. The experiments suggest that the proposed framework offers a suitable solution for embedded systems to solve 3D object detection and tracking problem, with added benefits.