MONITORING OF DYNAMIC OBJECTS ON A 2D OCCUPANCY MAP USING NEURAL NETWORKS AND MULTIMODAL DATA

Abstract

Abstract. The paper deals with the construction of dynamic occupancy maps, where the grid cell can contain not only information about the presence or absence of an obstacle, but also information about its velocity. We propose a multimodal approach to constructing 2D dynamic occupancy maps from LiDAR point clouds and camera images. The approach involves building a static occupancy map from LiDAR data and then adding information about cell velocities based on neural network instance segmentation and object tracking in monocular onboard camera images. Pedestrians and vehicles were considered as dynamic objects. One of the important stages is the projection of the object masks found in the images onto a 2D occupancy map. We compared the proposed approach with the classical method of constructing dynamic occupancy maps from LiDAR data based on the Monte Carlo method. An experimental quality evaluation of the approaches was carried out using the popular Mapillary Vistas and Waymo Open Datasets containing street scenes and a large number of dynamic objects. We demonstrate that the considered approaches can work in real time, which indicates the possibility of their application as part of the on-board control systems of autonomous cars or ground mobile robots. The software implementation of the proposed dynamic occupancy reconstruction approach is publicly available at the link: https://github.com/andrey1908/nn-dynamic-occupancy.