A TWO-STAGE APPROACH FOR RARE CLASS SEGMENTATION IN LARGE-SCALE URBAN POINT CLOUDS

Abstract

Abstract. Although deep learning has greatly improved the semantic segmentation accuracy of point clouds, the segmentation of rare classes in large-scale urban scenes has not been targeted in available methods. This paper proposes a two-stage segmentation framework with automated workflows for imbalanced rare classes based on general semantic segmentation. The proposed approach includes two stages: general semantic segmentation and object-based refined semantic segmentation. Firstly, general segmentation networks are utilized to segment general large objects. Secondly, refined semantic segmentation is conducted by an automated workflow: 3D clustering and bounding box (BBox) generation are applied to the point cloud of rare fine-grained objects during the training, followed by object detection to extract fine-grained objects. Afterwards, as the constraints, the extracted BBoxes further refine the segmentation results. Our approach is evaluated on the Hessigheim High-Resolution 3D Point Cloud (H3D) Benchmark and obtains state-of-the-art 89.35% overall accuracy and outstanding 75.70% mean F1-Score. Furthermore, rare classes Vehicle and Chimney achieve breakthroughs from zero to 63.63% and 52.00% in F1-score, respectively.