Systems Approach to Creating Test Scenarios for Automated Driving Systems

Siddartha Khastgir,Simon Brewerton,John Thomas,Paul Jennings

doi:10.1016/j.ress.2021.107610

Abstract

Increased safety has been advocated as one of the major benefits of the introduction of Automated Driving Systems (ADSs). Incorporation of ADSs in vehicles means that associated software has safety critical application, thus requiring exhaustive testing. To prove ADSs are safer than human drivers, some work has suggested that they will need to be driven for over 11 billion miles. The number of test miles driven is not, by itself, a meaningful metric for judging the safety of ADSs. Rather, the types of scenarios encountered by the ADSs during testing are critically important.With a Hazard Based Testing approach, this paper proposes that the extent to which testing miles are ‘smart miles’ that reflect hazard-based scenarios relevant to the way in which an ADS fails or handles hazards is a fundamental, if not pivotal, consideration for safety-assurance of ADSs. Using Systems Theoretic Process Analysis (STPA) method as a foundation, an extension to the STPA method has been developed to identify test scenarios. The approach has been applied to a real-world case study of a SAE Level 4 Low-Speed Automated Driving system (a.k.a. a shuttle). This paper, discusses the STPA analysis and a newly-developed test scenarios creation method derived from STPA.

Highlights

The last few decades have seen an increase in the amount of auto mation in safety critical systems in various industries e.g. aviation, manufacturing, automotive etc
Hazard Based Testing (HBT) and the tradition of sociotechnical systems instead suggest that the number of miles driven, by itself is not a meaningful metric for judging confidence in Advanced Driver Assistance Systems (ADASs) or Automated Driving Systems (ADSs)
Systems Theoretic Process Analysis (STPA) was used as it has been demonstrated to identify haz ards that might be missed by other hazard identification methods like FMEA, FTA, HAZOP, ETA etc. especially for complex systems involving human-automation interaction

Summary

Introduction

The last few decades have seen an increase in the amount of auto mation in safety critical systems in various industries e.g. aviation, manufacturing, automotive etc. The intro duction of automation is driven by its many potential benefits like increased safety [9,19,52] among others. While introduction of auto mation has a potential to increase safety in various domains including automotive, they add complexity especially in cyber-physical sys tems, requiring new risk assessment and safety verification methods for such systems [4,28,63]. The aviation industry approaches safety assessment by placing high safety integrity targets throughout the product development and use cycle [13]. The introduction of Advanced Driver Assistance Systems (ADASs) and Automated Driving Systems (ADSs) is further increasing the complexity manyfold [10]

Objectives

Findings

Discussion

Conclusion