Achieving weight coverage for an autonomous driving system with search-based test generation (HOP track at GECCO 2021)

Abstract

Autonomous Driving Systems (ADSs) are complex critical systems that must be thoroughly tested. Still, assessing the strength of tests for ADSs is an open and complex problem. Weight Coverage is a test criterion targeting ADSs which are based on a weighted cost function. It measures how much each weight, related to different aspects of the ADS's decision process, is involved in the decisions taken in a test scenario. All weights/aspects should be involved for a strong test suite. Although weight coverage can measure the quality of a test suite, it does not provide clear guidance for test generation. This work proposes weight coverage-driven search-based test generation for ADSs. It describes and compares three designs of the search process: a single-objective search aiming at generating a test covering a single weight; a multi-objective search where each objective targets a different weight; and a single-objective search where the fitness function is an aggregate function representing the coverage over multiple weights. Experiments using an ADS system provided by our industry partner show the validity of the method and provide insights into the benefits of each search design. This Hot-off-the-Press paper summarises the paper [2]: T. Laurent, P. Arcaini, F. Ishikawa and A. Ventresque, Achieving Weight Coverage for an Autonomous Driving System with Search-based Test Generation, 25th International Conference on Engineering of Complex Computer Systems (ICECCS 2020).