Test Your Self-Driving Algorithm: An Overview of Publicly Available Driving Datasets and Virtual Testing Environments

Abstract

Many companies aim for delivering systems for autonomous driving reaching out for SAE Level 5. As these systems run much more complex software than typical premium cars of today, a thorough testing strategy is needed. Early prototyping of such systems can be supported using recorded data from onboard and surrounding sensors as long as open-loop testing is applicable; later, though, closed-loop testing is necessary-either by testing on the real vehicle or by using a virtual testing environment. This paper is a substantial extension of our work presented at the 2017 IEEE International Conference on Intelligent Transportation Systems (ITSC) that was surveying the area of publicly available driving datasets. Our previous results are extended by additional datasets and complemented with a summary of publicly available virtual testing environments to support closed-loop testing. As such, a steadily growing number of 37 datasets for open-loop testing and 22 virtual testing environments for closed-loop testing have been surveyed in detailed. Thus, conducting research toward autonomous driving is significantly supported from complementary community efforts: A growing number of publicly accessible datasets allow for experiments with perception approaches or training and testing machine-learning-based algorithms, while virtual testing environments enable end-to-end simulations.