Semantic Segmentation of Terrestrial Laser Scanning Point Clouds Using Locally Enhanced Image-Based Geometric Representations

Abstract

Point cloud data acquired using terrestrial laser scanning (TLS) often need to be semantically segmented to support many applications. To this end, various point-, voxel-, and image-based methods have been developed. For large-scale point cloud data, the former two types of methods often require extensive computational effort. In contrast, image-based methods are favorable from the perspective of computational efficiency. However, existing image-based methods are highly dependent on RGB information and do not provide an effective means of representing and utilizing the local geometric characteristics of point cloud data in images. This not only limits the overall segmentation accuracy but also prohibits their application to situations where the RGB information is absent. To overcome such issues, this research proposes a novel image enhancement method to reveal the local geometric characteristics in images derived by the projection of the point cloud coordinates. Based on this method, various feature channel combinations were investigated experimentally. It was found that the new combination <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$IZ_{e}D_{e}$ </tex-math></inline-formula> (i.e., intensity, enhanced <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Z$ </tex-math></inline-formula> -coordinate, and enhanced range images) outperformed the conventional <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$I$ </tex-math></inline-formula> RGB and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$I$ </tex-math></inline-formula> RGB <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$D$ </tex-math></inline-formula> channel combinations. As such, the approach can be used to replace the RGB channels for semantic segmentation. Using this new combination and the pretrained HR-EHNet considered, a mean intersection over union (mIoU) of 74.2% and an overall accuracy (OA) of 92.1% were achieved on the Semantic3D benchmark, which sets a new state of the art (SOTA) for the semantic segmentation accuracy of image-based methods.