Object Removal for Testing Object Detection in Autonomous Vehicle Systems

Abstract

An object detection system is a critical part of autonomous vehicle systems. To ensure the safety and efficiency of autonomous vehicles, object detection is required to satisfy high sensitivity and accuracy. However, the state-of-the-art object detection systems fully rely on the construction of Deep Neural Networks (DNNs), which are complex and difficult to understand. It is difficult to employ white-box testing on DNNs since the output of a single neuron is inexplicable to developers. In this paper, we propose a black-box testing method based on metamorphic testing to test object detection systems. This method can reveal errors in object detection and generate high-quality test input data, i.e., a large amount of mutated images. To this end, we set up a metamorphic relation for evaluation on the testing results of prediction and design a novel strategy via object removal to generate mutated images. Instead of existing methods of adding noises to images, our method constructs mutated images by removing an object from the image background. This work can generate new images for testing from input images and detect errors in object detection in autonomous vehicle systems.