Extraction of local structure information of point clouds through space-filling curve for semantic segmentation

Abstract

• The relative angle feature between the center point in the neighborhood ball and their neighbors can improve the semantic segmentation results. • The space filling curve has a good ability to maintain the local geometric structure of point cloud. • The result of semantic segmentation does not always improve with the increase of the number of neighborhood points. • With the increase of feature vector dimension, the result of semantic segmentation will tend to be stable. The point cloud semantic segmentation network based on point-wise multi-layer perceptron (MLP) has gained extensive applications because of its end-to-end advantages. However, there exist two major limitations for this type of network: (1) In a neighborhood ball, the relative features between the central point and its neighboring point are not adequately exploited. (2) Only the relative features in the neighborhood ball are extracted, but the overall morphological information of the neighborhood ball is ignored. To overcome the limitations, this paper proposes two-fold improvements on local structure information extraction: (1) The relative angular feature is added and combined with other initial features as the input of the model to exploit the relative geometric features between points to the greatest extent. (2) The unordered points in the neighborhood ball are reordered based on the space-filling curve (SFC) and then fed into the MLP to extract the overall structure information of the neighborhood ball. And a semantic segmentation model is developed based on the two proposed feature extraction modules and U-Net, which is evaluated on two public datasets. The experimental result has demonstrated that the two proposed feature extraction modules can effectively extract geometric information in the point cloud and the proposed semantic segmentation model has strong semantic recognition capability for objects with complex morphologies. The mean intersection over union (mIoU) of the proposed model reached 70.6% and 47.8% for the Semantic 3D and Semantic KITTI datasets, respectively. Besides, the proposed model achieves real-time segmentation with only four encoder layers.