Dual fusion network for semantic segmentation of point clouds

Abstract

The current networks for point cloud segmentation fail to fully consider the connections between points in each neighborhood, which makes the final segmentation capability underperform. To obtain richer local relationships between points, this paper proposes a network called GDFNet (Graph Convolution-based Dual Fusion Network), which dynamically mines the correlations between points in each layer of the network. GDFNet is divided into two parts: a density-sensitive assignment module and a spatial dynamic convolution module. The density-sensitive assignment module treats the traditional convolution kernel as a nonlinear function consisting of a weight and a density function applicable to the local coordinates of 3D points. For a given point, the weight function is learned by a multilayer perceptron network, and the density function is learned by kernel density estimation. In this way, the density features of each point are fitted, thus improving the inherent defect of the original point cloud with non-uniform density. The spatial dynamic convolution module groups the original feature channels and then uses gated processing units to enrich the feature representation of the input points. At the same time, dynamic convolution is embedded in the Euclidean space to obtain the structural information of each point and further supplement the local detail features. Extensive experiments demonstrate the effectiveness and superiority of our method on S3DIS and ScanNetv2 datasets.