Validation Scores to Evaluate the Detection Capability of Sensor Systems Used for Autonomous Machines in Outdoor Environments

Abstract

The characterization of the detection capability assumes significance when the reliable monitoring of the region of interest by a non-contact sensor is a safety-relevant function. This paper introduces new validation scores that evaluate the detection capability of non-contact sensors intended to be applied to outdoor machines. The scores quantify, in terms of safety, the suitability of the sensor for the intended implementation in an environmental perception system of (highly) automated machines. This was achieved by developing an extension to the new Real Environment Detection Area (REDA) method and linking the methodology with the sensor standard IEC/TS 62998-1. The extension includes point-by-point and statistic-based error evaluation which leads to the Usability-Score, Availability-Score, and Reliability-Score. By applying the principle in the agricultural sector using ISO 18497 and linking this with data from a real outdoor test stand, it was possible to show that the validation scores offer a generic approach to quantify the detection capability and express this in a machine manufacturer-oriented manner. The findings of this study have significant implications for the advancement of safety-related sensor systems integrated into machines operating in complex environments. In order to achieve full implementation, it is necessary to define in the standards which score is required for each Performance Level (PL).