Object detection and classication in outdoor environments for autonomous passenger vehicle navigation based on Data Fusion of Articial Vision System and LiDAR sensor

Abstract

This research project took part in the SENA project (Autonomous Embedded Navigation System), which was developed at the Mobile Robotics Lab of the Mechatronics Group at the Engineering School of Sao Carlos, University of Sao Paulo (EESC - USP) in collaboration with the Sao Carlos Institute of Physics. Aiming for an autonomous behavior in the prototype vehicle this dissertation focused on deploying some machine learning algorithms to support its perception. These algorithms enabled the vehicle to execute articial-intelligence tasks, such as prediction and memory retrieval for object classication. Even though in autonomous navigation there are several perception, cognition and actuation tasks, this dissertation focused only on perception, which provides the vehicle control system with information about the environment around it. The most basic information to be provided is the existence of objects (obstacles) around the vehicle. In formation about the sort of object it is also provided, i.e., its classication among cars, pedestrians, stakes, the road, as well as the scale of such an object and its position in front of the vehicle. The environmental data was acquired by using a camera and a Velodyne LiDAR. A ceiling analysis of the object detection pipeline was used to simulate the proposed methodology. As a result, this analysis estimated that processing specic regions in the PDF Compressor Pro xii image (i.e., Regions of Interest, or RoIs), where it is more likely to nd an object, would be the best way of improving our recognition system, a process called image normalization. Consequently, experimental results in a data-fusion approach using laser data and images, in which RoIs were found using the LiDAR data, showed that the fusion approach can provide better object detection and classication compared with the use of either camera or LiDAR alone. Deploying a data-fusion classication using RoI method can be executed at 6 Hz and with 100% precision in pedestrians and 92.3% in cars. The fusion also enabled road estimation even when there were shadows and colored road markers in the image. Vision-based classier supported by LiDAR data provided a good solution for multi-scale object detection and even for the non-uniform illumination problem.