CornerSim: A Virtualization Framework to Generate Realistic Corner-Case Scenarios for Autonomous Driving Perception Testing

Abstract

Autonomous driving development requires rigorous testing in real-world scenarios, including adverse weather, unpredictable events, object variations, and sensor limitations. However, these challenging “corner cases” are elusive in conventional datasets due to their unpredictability, high costs, and inherent risks. Recognizing the critical role of ground truth data in autonomous driving, the demand for synthetic data becomes evident. Contemporary machine learning-based algorithms essential to autonomous vehicles heavily depend on labeled data for training and validation. Simulation of scenarios not only mitigates the scarcity of real-world data but also facilitates controlled experimentation in situations that are challenging to replicate physically. The challenge extends beyond data scarcity, encompassing the impediment posed by the inability to systematically control and manipulate specific scenarios, hindering progress.To overcome these challenges, we present CornerSim, a dynamic virtualization framework simplifying the creation and modification of diverse driving scenarios. Leveraging simulation, CornerSim generates synthetic environments for comprehensive testing, providing essential outputs like raw sensor data (cameras, LiDAR, etc.) and labeled data (object detection bounding boxes, classes, semantic segmentation). The unpredictable nature of real-world corner cases complicates obtaining a sufficiently large and diverse annotated dataset. CornerSim addresses this challenge by not only generating synthetic data but also supplying necessary ground truth for training and evaluating machine learning models.This paper emphasizes the introduction of CornerSim and its ability to challenges related to testing autonomous vehicles in realistic scenarios. It focuses on the framework’s capabilities, design principles, and integration, with the goal of enhancing thorough testing and validation of autonomous driving systems in a simulated environment, improving their robustness and safety. Our approach involves running simulations to generate datasets, which are statistically studied and compared with real data. Furthermore, we apply state-of-the-art detection algorithms to assess if data generated by CornerSim is suitable for both training and validation stages.