Abstract
The safety of the intended functionality (SOTIF) has become one of the hottest topics in the field of autonomous driving. However, no testing and evaluating system for SOTIF performance has been proposed yet. Therefore, this paper proposes a framework based on the advanced You Only Look Once (YOLO) algorithm and the mean Average Precision (mAP) method to evaluate the object detection performance of the camera under SOTIF-related scenarios. First, a dataset is established, which contains road images with extreme weather and adverse lighting conditions. Second, the Monte Carlo dropout (MCD) method is used to analyze the uncertainty of the algorithm and draw the uncertainty region of the predicted bounding box. Then, the confidence of the algorithm is calibrated based on uncertainty results so that the average confidence after calibration can better reflect the real accuracy. The uncertainty results and the calibrated confidence are proposed to be used for online risk identification. Finally, the confusion matrix is extended according to the several possible mistakes that the object detection algorithm may make, and then the mAP is calculated as an index for offline evaluation and comparison. This paper offers suggestions to apply the MCD method to complex object detection algorithms and to find the relationship between the uncertainty and the confidence of the algorithm. The experimental results verified by specific SOTIF scenarios proof the feasibility and effectiveness of the proposed uncertainty acquisition approach for object detection algorithm, which provides potential practical implementation chance to address perceptual related SOTIF risk for autonomous vehicles.
Highlights
According to the data provided by World Health Organization, about 1.25 million people are killed and even more people are injured in traffic accidents each year
The uncertainty analysis module receives the safety of the intended functionality (SOTIF) dataset as the input, calculates the prediction uncertainty using the Monte Carlo dropout (MCD) method, and outputs them to the calibration module
Comparator generates a cluster for every object in the first sampled detection result, and a subsequent bounding box will be assigned to a cluster only if the intersection over union (IoU) of them exceeds the given IoU threshold
Summary
According to the data provided by World Health Organization, about 1.25 million people are killed and even more people are injured in traffic accidents each year. A significant amount of work has been carried out along with technological advancements to ensure the safety of automobiles. The advanced driving assistance system (ADAS) plays an auxiliary role for the general public to drive, and the intelligent transportation system (ITS) brings convenience to road safety management. Developing a more powerful automatic driving system and evaluating its safety are still great challenges that need further studies. The safety concerns of automobiles mainly include functional safety, automotive cybersecurity, and safety of the intended functionality (SOTIF), which focuses on the risks caused by potential hazards such as functional insufficiencies and reasonably predictable personnel misuses [1]. The SOTIF performance of the perception algorithms is mainly considered in this research, to evaluate which a relevant database is needed as support.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
