A Review on the Deep Learning-based Surface Reconstruction from the Point Clouds

Abstract

Background: Point cloud has become one of the most important data formats for 3D presentation because of the increased availability of acquisition devices and its wide applications. Deep learning has the most powerful ability to capture features from data and has successfully solved various problems in the field of image, such as classification, segmentation, and generation. Deep learning is commonly used to process data with a structured grid, while point cloud is irregular and unstructured. The irregularity of point clouds makes it difficult to use deep learning to solve the problems represented by point clouds. Recently, numerous approaches have been proposed to process point clouds with deep learning to solve various problems. Objective: The objective of this study is to serve as a guide to new scholars in the field of deep learning on 3D surface reconstruction from point clouds as it presents the recent progress in deep learning-based surface reconstruction for point clouds. It helps scholars to grasp the current research situation better and further explore the search direction. Method: This study reviews the recent progress in deep learning-based methods used for surface reconstruction from point clouds and large-scale 3D point cloud benchmark datasets commonly used. Results: Several relevant articles on deep learning used for surface reconstruction from point clouds and some recent patents on deep learning applications are collected and reviewed in this paper. The difficulty of irregularity of point clouds can be overcome by deep learning methods, thus achieving remarkable progress in surface reconstruction. Conclusion: Deep learning for 3D surface reconstruction from point clouds is becoming a research hotspot due to its performance in terms of anti-interference and generalization. Although the advance is remarkable, there are still some challenges that need to be further studied.