Multi-resolution Dense Network for Point Cloud Completion

CarvingNet: Point cloud completion by stepwise refining multi-resolution features

Point clouds obtained from laser scanners or other devices often exhibit incompleteness, which poses a challenge for subsequent point cloud processing. Therefore, accurately predicting the complete shape from partial observations has paramount significance. In this paper, we introduce PCCDiff, a probabilistic model inspired by Denoising Diffusion Probabilistic Models (DDPMs), designed for point cloud completion tasks. Our model aims to predict missing parts in incomplete 3D shapes by learning the reverse diffusion process, transforming a 3D Gaussian noise distribution into the desired shape distribution without any structural assumption (e.g., geometric symmetry). Firstly, we design a conditional point cloud completion network that integrates Missing-Transformer and TreeGCN, facilitating the prediction of complete point cloud features. Subsequently, at each step of the diffusion process, the obtained point cloud features serve as condition inputs for the symmetric Diffusion ResUNet. By incorporating these condition features and incomplete point clouds into the diffusion process, PCCDiff demonstrates superior generation performance compared to other methods. Finally, extensive experiments are conducted to demonstrate the effectiveness of our proposed generative model for completing point clouds.

With the continuous advancement of 3D perception technology, point cloud data has found increasingly widespread application. However, the presence of holes in point cloud data caused by device limitations and environmental interference severely restricts algorithmic performance, making point cloud completion a research topic of high interest. This study observes that most existing mainstream point cloud completion methods primarily focus on capturing global features, while often underrepresenting local structural details. Moreover, the generation process of complete point clouds lacks effective control over fine-grained features, leading to insufficient detail in the completed outputs and reduced data integrity. To address these issues, we propose a Set Combination Multi-Layer Perceptron (SCMP) module that enables the simultaneous extraction of both local and global features, thereby reducing the loss of local detail information. In addition, we introduce the Squeeze Excitation Pooling Network (SEP-Net) module, an informative channel attention mechanism capable of adaptively identifying and enhancing critical channel features, thus improving the overall feature representation capability. Based on these modules, we further design a novel Feature Fusion Point Fractal Network (FFPF-Net), which fuses multi-dimensional point cloud features to enhance representation capacity and progressively refines the missing regions to generate a more complete point cloud. Extensive experiments conducted on the ShapeNet-Part and MVP datasets compared to L-GAN and PCN showed average prediction error improvements of 1.3 and 1.4, respectively. The average completion errors on the ShapeNet-Part and MVP datasets are 0.783 and 0.824, highlighting the improved fine-detail reconstruction capability of our network. These results indicate that the proposed method effectively enhances point cloud completion performance and can further promote the practical application of point cloud data in various real-world scenarios.

LiDAR is an active remote sensing technology widely used in forestry applications, such as forest resource surveys, tree information collection, and ecosystem monitoring. However, due to the resolution limitations of 3D-laser scanners and the canopy occlusion in forest environments, the tree point clouds obtained often have missing data. This can reduce the accuracy of individual tree segmentation, which subsequently affects the tree species classification. To address the issue, this study used point cloud data with RGB information collected by the UAV platform to improve tree species classification by completing the missing point clouds. Furthermore, the study also explored the effects of point cloud completion, feature selection, and classification methods on the results. Specifically, both a traditional geometric method and a deep learning-based method were used for point cloud completion, and their performance was compared. For the classification of tree species, five machine learning algorithms—Random Forest (RF), Support Vector Machine (SVM), Back Propagation Neural Network (BPNN), Quadratic Discriminant Analysis (QDA), and K-Nearest Neighbors (KNN)—were utilized. This study also ranked the importance of features to assess the impact of different algorithms and features on classification accuracy. The results showed that the deep learning-based completion method provided the best performance (avgCD = 6.14; avgF1 = 0.85), generating more complete point clouds than the traditional method. On the other hand, compared with SVM and BPNN, RF showed better performance in dealing with multi-classification tasks with limited training samples (OA-87.41%, Kappa-0.85). Among the six dominant tree species, Pinus koraiensis had the highest classification accuracy (93.75%), while that of Juglans mandshurica was the lowest (82.05%). In addition, the vegetation index and the tree structure parameter accounted for 50% and 30%, respectively, in the top 10 features in terms of feature importance. The point cloud intensity also had a high contribution to the classification results, indicating that the lidar point cloud data can also be used as an important basis for tree species classification.

Acquiring semantics directly from a point cloud is an important requirement for handling point cloud tasks. However, point clouds captured with laser scanner equipment are often incomplete due to the limitations posed by target occlusion and light reflection. Consequently, recovering the complete point clouds from partial and sparse ones is essential for further studies. In this paper, we model a novel projected generative adversarial network (PGAN) for point cloud completion. First, we present a multi-scale generator module (MSGM) to fully capture the local structures and global shape in the raw incompletion point cloud and generate the multi-scale complete point cloud. In contrast to existing point cloud feature extractors, our MSGM promotes a correlation between different regions of an incomplete point cloud and integrates the contextual information of the point cloud. Second, we observe that the existing point discriminator is inadequate to enhance the discrimination of the prediction point cloud. To address this problem, we project the completed point cloud to 2D maps and apply adversarial training to discriminate the geometrical shape from a specific viewpoint. Comprehensive experiments on the ShapeNet and ModelNet40 datasets show that the proposed method performs well against existing point cloud completion tasks. We also present an ablation study to demonstrate the advantages of the projected generative adversarial network.

Point cloud completion aims at predicting dense complete 3D shapes from sparse incomplete point clouds captured from 3D sensors or scanners. It plays an essential role in various applications such as autonomous driving, 3D reconstruction, augmented reality, and robot navigation. Existing point cloud completion methods follow the encoder-decoder paradigm, in which the complete point clouds are recovered in a coarse-to-fine strategy. However, only using the global feature is difficult and will lead to blurring of the global structure and distortion of local details. To address this problem, we propose a novel Partial-to-Partial Point Generation Network ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{P}^{2}$</tex-math></inline-formula> GNet), a learning-based approach for point cloud completion. In <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{P}^{2}$</tex-math></inline-formula> GNet, we use a feature disentangle encoder to obtain the global feature, and missing code and novel view partial point cloud are generated conditioned on the view-related missing code. To better aggregate partial point clouds, an attentive sampling module is proposed to sample multiple partial point clouds into the final complete result. Extensive experiments on several public benchmarks demonstrate that our <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{P}^{2}$</tex-math></inline-formula> GNet outperforms state-of-the-art point cloud completion methods.

Point cloud completion aims to reconstruct complete point clouds from partial point clouds, which is widely used in various fields such as autonomous driving and robotics. Most existing methods are sparse point cloud completion, where the number of point clouds after completion is relatively small and the details are insufficient. This article proposes a novel end-to-end generative adversarial network-based dense point cloud completion architecture (DPCG-Net). We design two generative adversarial network (GAN)-based modules that translate point cloud completion into mapping between global feature distributions obtained by encoding partial point clouds and ground truth, respectively. The first designed generator module proposes skip connections to fully connected layer-based network for regenerating global feature and changing the global feature distribution derived from the encoder module to approximate the ground truth global feature distribution. The second proposed discriminator module divides high-dimensional global feature vectors into several smaller batches for judgment to guarantee the similarity between the regenerated global feature and the ground truth. We perform quantitative and qualitative experiments on the ShapeNet and KITTI datasets. Experiments on ShapeNet demonstrate that our model outperforms other models in cases where the lack of a large proportion of point clouds results in a large loss of spatial structure, especially when 80% of point clouds are missing. Moreover, KITTI experiments reveal that it is also valid for realistic situations. In addition, application in classification shows that the classification accuracy of point clouds completed with DPCG-Net is as high as 86.5% under the condition of 80% missing point clouds.

GMP-Net: Graph based Missing Part Patching Network for Point Cloud Completion

Modern autonomous vehicles perceive their surroundings with the help of 3D computer vision technologies (e.g., object detection, vehicle recognition). The 3D point cloud obtained by vehicle-mounted mobile LiDAR is the primary data for 3D visual tasks in the driving system. Due to partial observations, a complete point cloud of the surrounding vehicle cannot be obtained. In this paper, we proposed Point Voxel Completion Network (PVCNet), an end-to-end learning-based model for 3D vehicle completion. Unlike existing point cloud completion methods, PVCNet only predicts the missing parts of the input and preserves the original spatial structure of the point cloud. PVCNet consists of two branches, the voxel branch and the point cloud branch. The voxel branch extracts the local features of the input and predicts the position of the missing point cloud in voxel form. The point cloud branch extracts the global features of the input point cloud. Combined with features from two branches, PVCNet can generate the missing points. The proposed PVCNet is validated on our hybrid dataset and KITTI dataset and performed competitive results in the task of urban vehicle point cloud completion.

Point cloud completion aims to acquire complete and high-fidelity point clouds from partial and low-quality point clouds, which are used in remote sensing applications. Existing methods tend to solve this problem solely from the point cloud modality, limiting the completion process to only 3D structure while overlooking the information from other modalities. Nevertheless, additional modalities possess valuable information that can greatly enhance the effectiveness of point cloud completion. The edge information in depth images can serve as a supervisory signal for ensuring accurate outlines and overall shape. To this end, we propose a brand-new point cloud completion network, dubbed MMDR, which utilizes point-based Differentiable Rendering (DR) to obtain the depth images to ensure that the model preserves the point cloud structures from the depth image domain. Moreover, the Attentional Feature Extractor (AFE) module is devised to exploit the global features inherent in the partial input, and the extracted global features together with the coordinates and features of the patch center are fed into the Point Roots Predictor (PRP) module to obtain a set of point roots for the upsampling module with Point Upsampling Transformer (PU-Transformer). Furthermore, the Multi-modality Consistency loss between the depth images from predicted point clouds and corresponding ground truth enables the PU-transformer to generate a high-fidelity point cloud with predicted point agents. Extensive experiments conducted on various existing datasets give evidence that MMDR surpasses the off-the-shelf methods for point cloud completion after qualitative and quantitative analysis.

Point cloud data in the real world is often affected by occlusion and light reflection, leading to incompleteness of the data. Large‐region missing point clouds will cause great deviations in downstream tasks. A dual feature fusion network (DFF‐Net) is proposed to improve the accuracy of the completion of a large missing region of the point cloud. First, a dual feature encoder is designed to extract and fuse the global and local features of the input point cloud. Subsequently, a decoder is used to directly generate a point cloud of missing region that retains local details. In order to make the generated point cloud more detailed, a loss function with multiple terms is employed to emphasise the distribution density and visual quality of the generated point cloud. A large number of experiments show that the authors’ DFF‐Net is better than the previous state‐of‐the‐art methods in the aspect of point cloud completion.

Point cloud completion aims to complete partial point clouds captured from the real world, which is a crucial step in the pipeline of many point cloud tasks. Among the existing methods for solving this problem, SnowflakeNet is the most outstanding. However, SnowflakeNet cannot recover the detailed structure of point clouds in latent code because it uses many max-pooling operations in the encoding stage. Therefore, we propose an improved architecture to effectively acquire and preserve more detail information from input point clouds, thereby enhancing the quality of point cloud completion. Specifically, the improved lightweight DGCNN is added to the encoder to extract local features. The geometric perception block of PoinTr is introduced to extract the global features of the point cloud, which can fully model the structural information and inter-point relationships of known point clouds. The new optimizer Adan is also used in the training process to complete the partial point clouds. Comparative experiments on Completion3D and PCN datasets show that our method is better than most current point cloud completion methods. Our method has the ability to produce the entire shape with details, including but not only smooth surfaces, well-defined edges, and distinct corners.

A point contextual transformer network for point cloud completion

Point cloud completion aims to infer the complete geometries for missing regions of 3D objects from incomplete ones. Previous methods usually predict the complete point cloud based on the global shape representation extracted from the incomplete input. However, the global representation often suffers from the information loss of structure details on local regions of incomplete point cloud. To address this problem, we propose Skip-Attention Network (SA-Net) for 3D point cloud completion. Our main contributions lie in the following two-folds. First, we propose a skip-attention mechanism to effectively exploit the local structure details of incomplete point clouds during the inference of missing parts. The skip-attention mechanism selectively conveys geometric information from the local regions of incomplete point clouds for the generation of complete ones at different resolutions, where the skip-attention reveals the completion process in an interpretable way. Second, in order to fully utilize the selected geometric information encoded by skip-attention mechanism at different resolutions, we propose a novel structure-preserving decoder with hierarchical folding for complete shape generation. The hierarchical folding preserves the structure of complete point cloud generated in upper layer by progressively detailing the local regions, using the skip-attentioned geometry at the same resolution. We conduct comprehensive experiments on ShapeNet and KITTI datasets, which demonstrate that the proposed SA-Net outperforms the state-of-the-art point cloud completion methods.

Recently, unstructured 3D point clouds have been widely used in remote sensing application. However, inevitable is the appearance of an incomplete point cloud, primarily due to the angle of view and blocking limitations. Therefore, point cloud completion is an urgent problem in point cloud data applications. Most existing deep learning methods first generate rough frameworks through the global characteristics of incomplete point clouds, and then generate complete point clouds by refining the framework. However, such point clouds are undesirably biased toward average existing objects, meaning that the completion results lack local details. Thus, we propose a multi-view-based shape-preserving point completion network with an encoder–decoder architecture, termed a point projection network (PP-Net). PP-Net completes and optimizes the defective point cloud in a projection-to-shape manner in two stages. First, a new feature point extraction method is applied to the projection of a point cloud, to extract feature points in multiple directions. Second, more realistic complete point clouds with finer profiles are yielded by encoding and decoding the feature points from the first stage. Meanwhile, the projection loss in multiple directions and adversarial loss are combined to optimize the model parameters. Qualitative and quantitative experiments on the ShapeNet dataset indicate that our method achieves good results in learning-based point cloud shape completion methods in terms of chamfer distance (CD) error. Furthermore, PP-Net is robust to the deletion of multiple parts and different levels of incomplete data.