Low-latency perception in off-road dynamical low visibility environments

Abstract

Robust systems are required for autonomous driving on non-uniform terrain commonly found in open-pit mines and developing countries. To help narrow the gap in this kind of application, this work proposes a perception system for autonomous vehicles and advanced driver assistance specialized on unpaved roads and off-road environments capable of navigating through rough terrain without a predefined trail. As part of this system, the Configurable Modular Segmentation Network (CMSNet) framework is proposed facilitating the creation of different architectures arrangements. Some CMSNet configurations were ported and trained to segment obstacles and trafficable ground on a new collection of images from unpaved roads and off-road scenarios containing adverse conditions such as night, rain, and dust. It was also performed: an investigation regarding the feasibility of applying deep learning to detect regions where the vehicle can pass through when there is no clear track boundary; a study of how our proposed segmentation algorithms behave in different severity levels of visibility impairment; and an evaluation of field tests carried out with semantic segmentation architectures conditioned for real-time inference. The new dataset (named Kamino) has almost 12,000 new images collected from an operated vehicle with various sensors, including eight cameras capturing synchronized sequences from different points of view. The Kamino dataset has a high number of labeled pixels compared to similar publicly available collections. It includes images collected from an off-road proving ground exclusively assembled for testing the system that emulates an open-pit mine scenario under different adverse conditions of visibility. To achieve embedded real-time inference and allows field tests, many layers of the CMSNet CNN networks were methodically removed and fused using TensorRT, C++, and CUDA. Empirical experiments on two datasets validated the effectiveness of the proposed system. The proposed solution achieves a mIoU of ∼87% (50.40% higher than the PSPNet and 56.21% than the DeepLabV3) on the Kamino dataset, and ∼81 % on the DepScene dataset, reaching up to 29 FPS on the RTX2060 GPU and almost 100 FPS with the optimized configurations, and low standard deviation values with the DrivePX2 computer platform.