Knowledge Graphs for Automated Driving

Abstract

Automated Driving (AD) datasets, when used in combination with deep learning techniques, have enabled significant progress on difficult AD tasks such as perception, trajectory prediction and motion planning. These datasets represent the content of driving scenes as captured by various sensors, including cameras, RADAR, and LiDAR, along with 2D/3D annotations of traffic participants. Such datasets, however, often fail to capture and to represent the spatial, temporal, functional, and semantic relations between entities in a scene. This lack of knowledge leads to a shallow understanding of the true complexity and dynamics inherent in a driving scene. In this paper, we argue that a knowledge graph based representation of driving scenes, that provides a richer structure and semantics, will lead to further improvements in automated driving. Towards this goal, we developed a layered architecture and ontologies for specific automated driving datasets and a fundamental ontology of shared concepts. We also built knowledge graphs (KG) for three different AD datasets. We perform an analysis w.r.t. information contained in the AD KGs and outline how the additional semantic information contained in the KGs could improve the performance of different AD tasks. Moreover, example queries are provided to retrieve relevant information that can be exploited for augmenting the AD pipelines. All artefacts needed for reproducability purposes are provided via a Dropbox folder <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> shorturl.at/iwyCV - we will go through an internal approval process for making all artefacts publicly available. We removed our internal namespaces of reused ontologies, because of confidentiality and to provide self-contained ontologies. As the original datasets are under specific licences we can not publish the KGs themselves, but we provided the scripts to generate them.