DualGroup for 3D instance and panoptic segmentation

Abstract

Existing 3D instance segmentation methods usually learn the offsets (also known as center-shifted vectors) from points to their instance center for clustering and generating segmentation results. However, due to the instances with different scales, direct regression offsets will make the model pay more attention to the larger instances and ignore the smaller instances. Besides, the clustering also may fail because a single bandwidth for point grouping is insufficient for instances with different scales. To address these two problems, we propose a new framework (DualGroup) for 3D instance segmentation. For the first issue, different from directly learning the offsets, we propose an encoded center-shifted vector learning (ECSVL), which effectively compresses the range of the regression center-shifted vectors for more conducive learning of smaller instances. Second, to handle the instances with different scales in clustering, we propose a dual hierarchical grouping (DHG) to better group all points into different instances. The cooperation of these two components leads to the success of indoor instance segmentation. Moreover, the DualGroup is extended to the 3D panoptic segmentation by fusing the semantic predictions and instance results. Experimental results on the ScanNet v2 and S3DIS datasets demonstrate the effectiveness and superiority of the DualGroup.