AI-Based Pedestrian Detection and Avoidance at Night Using Multiple Sensors

Abstract

In this paper, we present a pedestrian detection and avoidance scheme utilizing multi-sensor data collection and machine learning for intelligent transportation systems (ITSs). The system integrates a video camera, an infrared (IR) camera, and a micro-Doppler radar for data acquisition and training. A deep convolutional neural network (DCNN) is employed to process RGB and IR images. The RGB dataset comprises 1200 images (600 with pedestrians and 600 without), while the IR dataset includes 1000 images (500 with pedestrians and 500 without), 85% of which were captured at night. Two distinct DCNNs were trained using these datasets, achieving a validation accuracy of 99.6% with the RGB camera and 97.3% with the IR camera. The radar sensor determines the pedestrian’s range and direction of travel. Experimental evaluations conducted in a vehicle demonstrated that the multi-sensor detection scheme effectively triggers a warning signal to a vibrating motor on the steering wheel and displays a warning message on the passenger’s touchscreen computer when a pedestrian is detected in potential danger. This system operates efficiently both during the day and at night.