Quadsight® Vision System in Adverse Weather Maximizing the benefits of visible and thermal cameras

Abstract

Autonomous vehicles are currently one of the most popular research topics in computer vision. United Nations Economic Commission for Europe recently proposed a regulation for SAE level 3 automated driving systems. The current Operational Design Domains (ODD) are highway, slow speed (i.e. traffic jam), and clear weather conditions. Research is steadily creeping towards a focus on harsh weather conditions. There are now two major issues to investigate: (1) knowing how to characterize ODD and (2) extending ODD to include ‘new’ conditions. This investigation is being carried out within the framework of the AWARD project at Cerema's PAVIN platform. Foresight Automotive's QuadSight® vision system was tested under a range of artificially reproduced weather conditions. The novelty of this work is to present results of a 3D object detection ODD characterization: (a) on a commercially ready system, (b) using visible and thermal wavelengths, and (c) in controlled fog and rain conditions. The use of dual visible and long-wave infrared thermal sensors in stereo is essential to the all-weather detection of pedestrians and vehicles. The thermal sensor is essential in challenging conditions such as nighttime or adverse weather conditions. Rain and low lighting conditions pose no problem for the QuadSight system. The system also performs well in foggy conditions, with the only exception of compromised performance in very dense fog.