URTSegNet: A real-time segmentation network of unstructured road at night based on thermal infrared images for autonomous robot system

Abstract

When it comes to poor lighting conditions such as night, backlight, haze, etc., the detection method of unstructured roads based on visible light cameras often fails. Therefore, this paper aims to improve the detection performance of unstructured roads based on thermal infrared images for autonomous robot systems. For this purpose, we propose a real-time network for unstructured road segmentation at night based on thermal infrared images (URTSegNet) in this paper. Like BiSeNet V2, we adopt a two-branch structure, including Detail Branch and Semantic Branch, to extract spatial detail and contextual features. The Detail Branch maintains a small receptive field and high capacity and adjusts the computational cost by employing depth-wise separable convolutions. In the Semantic Branch, we modify Stem Block to obtain more information because of the characteristics of infrared images. Dilated convolution expands the receptive field while keeping the number of parameters unchanged. Finally, we design a more concise and efficient Aggregation Block using a gate mechanism. An infrared unstructured road dataset, including total darkness at night, low light, backlight, haze, and other scenes, is collected to get the real-world dataset and effectively evaluate performance. We got 97.5% Intersection over Union (IoU) and reached an inference speed of 129 FPS on a single NVIDIA GTX 1660S card on our dataset. In addition, extensive evaluation experiments are conducted on two public datasets to verify the generalization ability of the model and the role of each module. This research addresses the problem of road detection for unmanned platforms in unstructured environments at night or under poor lighting conditions. The algorithm has high accuracy and a fast inference speed so that it can be implemented on various embedded devices.