Densely connected graph convolutional network for joint semantic and instance segmentation of indoor point clouds

Abstract

In this paper, a densely connected graph convolutional network is proposed to jointly realize the semantic and instance segmentation of indoor point clouds. We combine a Graph Convolutional Network (GCN) and Multilayer Perceptron (MLP) into a new model (namely GCN-MLP) and design a more efficient attention pooling operation to establish a new and efficient module for extracting point cloud features. Also, we add a point cloud channel aggregation module to aggregate multi-level deep features to better express the discriminative characteristics of indoor point clouds. Next, a framework for joint semantic and instance segmentation is designed on the basis of the above modules. In the framework, the semantic branch and instance branch promote each other and obtain better semantic and instance segmentation effects simultaneously. Besides, a dense connection way among different levels of feature maps is designed, to fully extract the features of indoor scene. We have conducted the experiments on the public datasets (S3DIS and ShapeNet), and the results show that our framework is superior to other methods and also achieve better results in part segmentation. Compared with JSNet (Zhao and Tao, 2020), our method improves the indicators of mCov, mWCov, mRec, and mPrec by 3.3, 3.5, 4.6, and 1.8, respectively, in the instance segmentation of S3DIS (Area 5). Besides, we obtain the best results in 10 of the 13 classes in terms of IoU. For part segmentation of indoor objects, we also outperform ASIS (Wang et al., 2019b) by 0.4 on mIoU.