Cross-domain point cloud completion for multi-class indoor incomplete objects via class-conditional GAN inversion

Abstract

Indoor point clouds from real-world scans are often incomplete and sparse due to limited observation views and severe occlusion between objects. Point cloud completion can restore missing parts to ensure data integrity, so that subsequent tasks like point cloud classification, segmentation, object detection and three-dimensional (3D) reconstruction can obtain more accurate and complete geometric and semantic information. Existing point cloud completion methods focus more on synthetic data or require separate modeling on each class of objects, which struggle to meet the inherent requirements of the completion task for real-world indoor scenes containing multi-class objects with significant pose variations. In this research, we present an efficient class-conditional generative adversarial network (GAN) inversion framework for the task of recovering incomplete objects in real-world indoor scenes, in which we first train a class-conditional GAN to learn rich shape priors for multi-class objects from synthetic data domain and then find an optimal latent code in GAN’s latent space that gives a complete shape that best recovers the given incomplete input from real-world data domain. The performance of our proposed method is evaluated on both real-world and synthetic datasets. In comparison to ShapeInversion (Zhang et al., 2021), our method demonstrates an improvement in F-score by 3.7 and 2.0 percentage points while reducing the chamfer distance (CD) by 4.15 and 1.1 on ScanNet dataset and CRN dataset, respectively. The developed code will be publicly available at: https://github.com/CNU-DLandCV-lab/cgan-inversion.