HCFS3D: Hierarchical coupled feature selection network for 3D semantic and instance segmentation

Abstract

Semantic segmentation and instance segmentation based on 3D point clouds involve significant challenges, specifically in the task of joint semantic and instance segmentation. The efficient and effective mutual assistance between semantic and instance segmentation is rarely considered and still remains an unaddressed research problem. To address this, herein, a novel and robust 3D point cloud segmentation framework employing hierarchical coupled feature selection, named HCFS3D, is proposed; this framework can jointly and reciprocally perform semantic and instance segmentation. The framework is designed to promote these two tasks to exploit beneficial information from each other, on a shallow as well as a deep level. Moreover, to prevent the network from overfitting and to improve performance, we designed a loss function called the Adaptive Smooth Loss, which can adaptively assign different weights to samples that are difficult to segment. Furthermore, joint semantic and instance conditional random fields are included in the proposed framework to further improve its performance. Extensive experiments based on different datasets and various backbone networks demonstrate that HCFS3D outperforms other state-of-the-art methods.