Learning Hybrid Semantic Affinity for Point Cloud Segmentation

Abstract

In this paper, we present a hybrid semantic affinity learning method (HSA) to capture and leverage the dependencies of categories for 3D semantic segmentation. Unlike existing methods that only use the cross-entropy loss to perform one-to-one supervision and ignore the semantic relations between points, our approach aims to learn the label dependencies between 3D points from a hybrid perspective. From a global view, we introduce the structural correlations among different classes to provide global priors for point features. Specifically, we fuse word embeddings of labels and scene-level features as category nodes, which are processed via a graph convolutional network (GCN) to produce the sample-adapted global priors. These priors are then combined with point features to enhance the rationality of semantic predictions. From a local view, we propose the concept of local affinity to effectively model the intra-class and inter-class semantic similarities for adjacent neighborhoods, making the predictions more discriminative. Experimental results show that our method consistently improves the performance of state-of-the-art models across indoor (S3DIS, ScanNet), outdoor (SemanticKITTI), and synthetic (ShapeNet) datasets.