Reliability Assessment of Safety-Critical Sensor Information: Does One Need a Reference Truth?

Abstract

The automation of safety-relevant technical systems leads to the challenge of demonstrating the reliability of safety-critical sensor information that enables automation. An example of the same is the environment perception in automated driving vehicles, provided by lidar, radar, and, camera sensors. One way of assessing the reliability of these sensors is to conduct field tests with a reference sensing system. However, in this method, the required test effort is enormous and generating a reference truth is time consuming. In this paper, this motivates to investigate if in principle, it is possible to learn sensor information reliabilities without a reference truth, by solely comparing the output of redundant sensors. We develop such a testing framework, which enables learning of the sensors’ reliabilities and sensor error dependencies without a reference truth. We show with synthetic datasets that the framework correctly determines the sensor information reliability if an adequate statistical model for sensor errors and dependencies among sensors is employed. Therefore, sensor information reliabilities can potentially be learned from driver-controlled cars, equipped solely with standard sensors without reference systems, which is an opportunity for large-scale testing. The main challenge to avoid wrong inference is to check the appropriateness of the selected statistical model without reference truth.