Structure-preserving shape completion of 3D point clouds with generative adversarial network

Abstract

Due to the difficulty in maintaining the fine structures of 3D point cloud in shape completion, this study, with the help of the generative adversarial network framework, proposes a novel neural network for automatically repairing and completing the 3D shape of point clouds. This network consists of a generator and a discriminator. The generator of the proposed neural network adopts an encoder-decoder structure and takes the missing 3D point cloud shape data as the input. Firstly, it aligns the sampling point position and feature information of the input point cloud data by the input transform and feature transform. Then the weighted shared multi-layer perceptron extracts the local shape features for each sampling point and also extracts its feature codewords using the maximum pool layer and multi-layer perceptron coding. Secondly, it adds the feature codewords of sampling points with the grid coordinate data, and the decoder converts the grid data into the missing data of the underlying point cloud using two successive three-layer perceptron folding operations. Finally, it merges the missing completion data and the input data to get the complete 3D point cloud shape. Meanwhile, the proposed neural network discriminator receives the real and the completed point cloud data generated by the generator. The same encoder structure as the generator is also adopted to distinguish the true or false of the point cloud data, while the classification results are a feedback for optimizing the generator. Also, the generator will generate the “real” point cloud shape data. Experimental results illustrate that, for both the dense and sparse incomplete point cloud data, the proposed method effectively maintains the fine structures of the input point clouds while repairing the missing part of the underlying shapes.