Abstract
Abstract. This paper presents an approach which combines LiDAR sensors and cameras of a mobile multi-sensor system to obtain information about pedestrians in the vicinity of the sensor platform. Such information can be used, for example, in the context of driver assistance systems. In the first step, our approach starts by using LiDAR sensor data to detect and track pedestrians, benefiting from LiDAR’s capability to directly provide accurate 3D data. After LiDAR-based detection, the approach leverages the typically higher data density provided by 2D cameras to determine the body pose of the detected pedestrians. The approach combines several state-of-the-art machine learning techniques: it uses a neural network and a subsequent voting process to detect pedestrians in LiDAR sensor data. Based on the known geometric constellation of the different sensors and the knowledge of the intrinsic parameters of the cameras, image sections are generated with the respective regions of interest showing only the detected pedestrians. These image sections are then processed with a method for image-based human pose estimation to determine keypoints for different body parts. These keypoints are finally projected from 2D image coordinates to 3D world coordinates using the assignment of the original LiDAR points to a particular pedestrian.
Highlights
Pedestrians belong to the group of vulnerable road users, especially in an urban environment where pedestrians and vehicles have to share the traffic space
For our experiments we used data recorded with a multi-sensor vehicle which we have presented in our earlier work (Borgmann et al, 2018)
The LiDAR sensor was operating with 10 rotations per second, a single 360◦ scan took place in a time period of 0.1 s
Summary
Pedestrians belong to the group of vulnerable road users, especially in an urban environment where pedestrians and vehicles have to share the traffic space It is helpful for the development of driver assistance systems, for autonomous driving, and for traffic or urban planning to have automatic capabilities for immediate or long-term information acquisition about pedestrians in the vicinity of vehicles and in urban areas. Among the sensor equipment discussed for future vehicles, a number of established sensor technologies can be found These include cameras for visible or infrared light, LiDAR sensors, or RADAR. LiDAR sensors constitute their own light source and are able to directly acquire three-dimensional geometric information They typically provide a much lower data density than cameras and are not capable of capturing color-based features. Multi-sensor systems, i.e., vehicles equipped with different types of sensors, ideally allow the advantages of the different sensor types to be combined to provide as much information as possible about pedestrian activity in the vehicle’s vicinity
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
