TreeNet: Structure preserving multi-class 3D point cloud completion

Abstract

Generating the missing data of 3D object point clouds from partial observations is a challenging task. Existing state-of-the-art learning-based 3D point cloud completion methods tend to use a limited number of categories/classes of training data and regenerate the entire point cloud based on the training datasets. As a result, output 3D point clouds generated by such methods may lose details (i.e. sharp edges and topology changes) due to the lack of multi-class training. These methods also lose the structural and spatial details of partial inputs due to the models do not separate the reconstructed partial input from missing points in the output.In this paper, we propose a novel deep learning network - TreeNet for 3D point cloud completion. TreeNet has two networks in hierarchical tree-based structures: TreeNet-multiclass focuses on multi-class training with a specific class of the completion task on each sub-tree to improve the quality of point cloud output; TreeNet-binary focuses on generating points in missing areas and fully preserving the original partial input. TreeNet-multiclass and TreeNet-binary are both network decoders and can be trained independently. TreeNet decoder is the combination of TreeNet-multiclass and TreeNet-binary and is trained with an encoder from existing methods (i.e. PointNet encoder). We compare the proposed TreeNet with five state-of-the-art learning-based methods on fifty classes of the public Shapenet dataset and unknown classes, which shows that TreeNet provides a significant improvement in the overall quality and exhibits strong generalization to unknown classes that are not trained.