Metamorphic Testing for Autonomous Driving Systems in Fog based on Quantitative Measurement

Abstract

Fog is one of the major disaster weather affecting traffic safety. For autonomous driving system in foggy, the image is captured by the camera that will have some problems, which will make the autonomous driving system easily affected. Therefore, it is worth to do further research on how and to what extent foggy weather affects autonomous driving systems. This paper proposes a test method, that can quantify the influence of fog for the autonomous driving system whose output is steering angle. The method relies on the metamorphic testing and proposes the metamorphic relations from the density and direction of fog to determine whether the autonomous driving system predicts inconsistent with driving behavior. In this paper, the density of the fog is divided into 5 levels according to GB/T 27964–2011, and the direction of the fog is divided into 9 directions according to the regional information, and the fog scene images with different densities and directions are simulated using OpenCV and CycleGAN. The results of relevant experiments carried out for three different autonomous driving systems show that the method can quantitatively and effectively detect a large number of failures in the autonomous driving system. Specifically, the autonomous driving system is rarely affected by the direction of fog, but the number of failures in the autonomous driving system is positively correlated with the density of fog, and it is more likely to fail in the case of dense fog.