Runtime unknown unsafe scenarios identification for SOTIF of autonomous vehicles

Abstract

Safety is a critical concern for autonomous vehicles (AVs). Current testing approaches face challenges in simultaneously meeting the requirements of being valid, safe, and fast. To address these challenges, the silent testing approach that tests functions or systems in the background without interfering with driving is motivated. Building upon our previous research, this study first extends the method to specifically address the validation of AV perception, utilizing a lane marking detection algorithm (LMDA) as a case study. Second, field experiments were conducted to investigate the method’s effectiveness in validating AV systems. For both studies, an architecture for describing the working principle is presented. The efficacy of the method in evaluating the LMDA is demonstrated through the use of adversarial images generated from a dataset. Furthermore, various scenarios involving pedestrians crossing a road under different levels of criticality were constructed to gain practical insights into the method’s applicability for AV system validation. The results show that corner cases of the LMDA are successfully identified by the given evaluation metrics. Furthermore, the experiments highlight the benefits of employing multiple virtual instances with different initial states, enabling the expansion of the test space and the discovery of unknown unsafe scenarios, particularly those prone to false-positive objects. The practical implementation and systematic discussion of the method offer a significant contribution to AV safety validation.