Monocular thermal imaging for pedestrian detection and ranging

Abstract

To react to the presence of pedestrians, an automated braking system must first find the pedestrian(s) including locations relative to the automobile both in angular position and in distance. In the daytime, cameras and radar can provide the necessary information, but this combination, which requires ambient or active illumination, fails at night. Passive thermal sensors are now being enlisted to dramatically improve imaging at night, whereas substantial effort is underway to assure proper fusing of object information from the thermal sensor with other sensors on the automobile. To simplify the acquisition of information needed to make valid automated braking decisions at night, a camera system with a thermal image sensor was developed that identifies pedestrians and labels each identified pedestrian with location and distance data. The camera utilizes a single uncooled custom microbolometer sensor and a software suite implementing artificial intelligence and machine learning capabilities, running on a combination of convolutional neural networks and fusion processing to provide the data that an automobile host computer needs to implement fast, safe, and accurate automatic braking. We present details of the system construction and operation as well as initial test results showing the potential this technology has to dramatically reduce pedestrian fatalities at night as well as augment safety across all conditions, whether day, night, fog, rain, snow, dust, sun glare, or headlight glare.