A Novel Day-to-Night Obstacle Detection Method for Excavators Based on Image Enhancement and Multisensor Fusion

Abstract

Traditional excavator driving relies only on manual observation, resulting in increased hazards in unstructured environments. When the excavator works in a relatively dark environment, there will be potential risks for both the driver and the surrounding pedestrians. In order to address this issue, this study takes the advantages of three different sensors, including infrared cameras, RGB cameras, and lidar sensors, and proposes a novel day-to-night obstacle detection approach by fusing data from multiple sensors. For the dark environment at night, the infrared camera is adopted for the detection task. However, compared with RGB cameras, the infrared camera usually have lower resolutions, making it difficult to be directly applied for the obstacle detection. Therefore, an image enhancement processing method for low-resolution infrared images is developed based on the Difference of Gaussian (DoG). Then, an image recognition method based on YOLO-v5 is proposed to detect images after image enhancement. Finally, a multi-sensor fusion method is suggested to identify the semantic information and 3D coordinates of objects. Experimental studies are carried out to assess image quality and the effectiveness of various object recognition tasks. The results of the experiments demonstrate that our method is capable of not only accurately extracting pedestrian position information from a complicated background environment and realizing timely pedestrian alarms, but also maintaining detection performance in an excavator working environment at night.