Outdoor Semantic Segmentation for UGVs Based on CNN and Fully Connected CRFs

Abstract

This paper studies semantic segmentation in outdoor scene based on multi-sensor fusion data by unmanned ground vehicle (UGV). Laser, camera, and inertial navigation are fused into RGB-DI (RGB, depth and intensity) point cloud. Because of the speed change of the UGV in outdoor scene, laser scanning points in 3D space are distributed irregularly and unbalanced. It is difficult to extract features in point cloud to describe objects accurately. Therefore, this paper proposes a projection algorithm to generate a 2D RGB-DI image from the 3D RGB-DI point cloud so that the semantic segmentation in RGB-DI cloud points is transformed to the semantic segmentation in RGB-DI images. To adequately describe multiple objects in the RGB-DI images, a convolutional neural network (CNN) model is designed to extract abstract features. Since the fully connected CRF model takes into account the context of each object location in an RGB-DI image, the fully connected CRF model is used as a classifier to complete the semantic segmentation in the RGB-DI image. According to the corresponding relation between each point in the 3D point cloud and each pixel in the RGB-DI image, segmentation results in the RGB-DI image are mapped back to the original point clouds. Different datasets are used to evaluate our algorithms. Moreover, real-world experiments were applied to our UGV platform to show the practicability and validity of the proposed approach.