Semantic-guided fuzzing for virtual testing of autonomous driving systems

Abstract

Autonomous driving systems (ADS) have achieved spectacular development and have been utilized in numerous safety-critical tasks. Nonetheless, in spite of their considerable advancement, ADS perception components with high complexity and low interpretability often demonstrate unexpected corner-case behaviors. Several real-world accidents involving self-driving cars even lead to fatalities. Before rolling the autonomous vehicles out to the end-users, it is vital to test and guarantee the safety of ADS. As one of the most critical autonomous driving testing techniques, the prevailing virtual testing depends on the tester using tool-specific languages to code traffic simulation programs correctly. However, this process often requires plenty of effort, and it may fail to capture various rare events from complex driving situations that require sophisticated awareness of the surroundings.In this paper, we design and implement a semantic-guided scene fuzzing framework for autonomous driving systems, namely FuzzScene, based on the metamorphic testing theory. It employs driving scenario description language for scenario representation and equips a tree-based mutation strategy to generate tests with proper oracle information. To improve the testing efficiency and detect misbehaviors under different settings, we propose a unique sampling strategy and construct a testing guidance criterion to optimize FuzzScene. We experiment FuzzScene with multiple steering controllers to evaluate its performance on different tasks. The experiment results show that the semantic transformed driving scenarios generated by FuzzScene efficiently detect hundreds of inconsistent behaviors of ADS. Also, the results confirm that FuzzScene can improve steering precision by retraining with the generated scenes.