PCCDiff: Point Cloud Completion with Conditional Denoising Diffusion Probabilistic Models

Acquiring semantics directly from a point cloud is an important requirement for handling point cloud tasks. However, point clouds captured with laser scanner equipment are often incomplete due to the limitations posed by target occlusion and light reflection. Consequently, recovering the complete point clouds from partial and sparse ones is essential for further studies. In this paper, we model a novel projected generative adversarial network (PGAN) for point cloud completion. First, we present a multi-scale generator module (MSGM) to fully capture the local structures and global shape in the raw incompletion point cloud and generate the multi-scale complete point cloud. In contrast to existing point cloud feature extractors, our MSGM promotes a correlation between different regions of an incomplete point cloud and integrates the contextual information of the point cloud. Second, we observe that the existing point discriminator is inadequate to enhance the discrimination of the prediction point cloud. To address this problem, we project the completed point cloud to 2D maps and apply adversarial training to discriminate the geometrical shape from a specific viewpoint. Comprehensive experiments on the ShapeNet and ModelNet40 datasets show that the proposed method performs well against existing point cloud completion tasks. We also present an ablation study to demonstrate the advantages of the projected generative adversarial network.

The task of 3D point cloud completion is to predict a complete point cloud from the incomplete partial point cloud. Generally, the encoder is used to extract the global shape features of the input incomplete point cloud, and then the decoder infers the complete point cloud. At present, some methods have been improved by multi-resolution encoders and multi-layer decoders, and achieved obvious results. However, these methods still cannot fully express the shape features. In order to solve this problem, we propose a feature fusion mechanism based on skip connection. The features extracted from each resolution point cloud are connected with the input of corresponding decoder. Then they are weighted and fused to obtain denser features, which can be decoded into a finer point cloud. In addition, the current loss function is still not a good measure of the similarity between two point clouds, so we also proposed a multi-stage local average Hausdroff Loss to form a joint reconstruction loss function to guide the generation of missing point clouds. Experimental results prove the effectiveness of our method in point cloud completion tasks, and show that it products better performance than existing methods.

CarvingNet: Point cloud completion by stepwise refining multi-resolution features

GMP-Net: Graph based Missing Part Patching Network for Point Cloud Completion

DPG-Net: Densely progressive-growing network for point cloud completion

LiDAR is an active remote sensing technology widely used in forestry applications, such as forest resource surveys, tree information collection, and ecosystem monitoring. However, due to the resolution limitations of 3D-laser scanners and the canopy occlusion in forest environments, the tree point clouds obtained often have missing data. This can reduce the accuracy of individual tree segmentation, which subsequently affects the tree species classification. To address the issue, this study used point cloud data with RGB information collected by the UAV platform to improve tree species classification by completing the missing point clouds. Furthermore, the study also explored the effects of point cloud completion, feature selection, and classification methods on the results. Specifically, both a traditional geometric method and a deep learning-based method were used for point cloud completion, and their performance was compared. For the classification of tree species, five machine learning algorithms—Random Forest (RF), Support Vector Machine (SVM), Back Propagation Neural Network (BPNN), Quadratic Discriminant Analysis (QDA), and K-Nearest Neighbors (KNN)—were utilized. This study also ranked the importance of features to assess the impact of different algorithms and features on classification accuracy. The results showed that the deep learning-based completion method provided the best performance (avgCD = 6.14; avgF1 = 0.85), generating more complete point clouds than the traditional method. On the other hand, compared with SVM and BPNN, RF showed better performance in dealing with multi-classification tasks with limited training samples (OA-87.41%, Kappa-0.85). Among the six dominant tree species, Pinus koraiensis had the highest classification accuracy (93.75%), while that of Juglans mandshurica was the lowest (82.05%). In addition, the vegetation index and the tree structure parameter accounted for 50% and 30%, respectively, in the top 10 features in terms of feature importance. The point cloud intensity also had a high contribution to the classification results, indicating that the lidar point cloud data can also be used as an important basis for tree species classification.

With the continuous advancement of 3D perception technology, point cloud data has found increasingly widespread application. However, the presence of holes in point cloud data caused by device limitations and environmental interference severely restricts algorithmic performance, making point cloud completion a research topic of high interest. This study observes that most existing mainstream point cloud completion methods primarily focus on capturing global features, while often underrepresenting local structural details. Moreover, the generation process of complete point clouds lacks effective control over fine-grained features, leading to insufficient detail in the completed outputs and reduced data integrity. To address these issues, we propose a Set Combination Multi-Layer Perceptron (SCMP) module that enables the simultaneous extraction of both local and global features, thereby reducing the loss of local detail information. In addition, we introduce the Squeeze Excitation Pooling Network (SEP-Net) module, an informative channel attention mechanism capable of adaptively identifying and enhancing critical channel features, thus improving the overall feature representation capability. Based on these modules, we further design a novel Feature Fusion Point Fractal Network (FFPF-Net), which fuses multi-dimensional point cloud features to enhance representation capacity and progressively refines the missing regions to generate a more complete point cloud. Extensive experiments conducted on the ShapeNet-Part and MVP datasets compared to L-GAN and PCN showed average prediction error improvements of 1.3 and 1.4, respectively. The average completion errors on the ShapeNet-Part and MVP datasets are 0.783 and 0.824, highlighting the improved fine-detail reconstruction capability of our network. These results indicate that the proposed method effectively enhances point cloud completion performance and can further promote the practical application of point cloud data in various real-world scenarios.

Point cloud completion aims to infer the complete geometries for missing regions of 3D objects from incomplete ones. Previous methods usually predict the complete point cloud based on the global shape representation extracted from the incomplete input. However, the global representation often suffers from the information loss of structure details on local regions of incomplete point cloud. To address this problem, we propose Skip-Attention Network (SA-Net) for 3D point cloud completion. Our main contributions lie in the following two-folds. First, we propose a skip-attention mechanism to effectively exploit the local structure details of incomplete point clouds during the inference of missing parts. The skip-attention mechanism selectively conveys geometric information from the local regions of incomplete point clouds for the generation of complete ones at different resolutions, where the skip-attention reveals the completion process in an interpretable way. Second, in order to fully utilize the selected geometric information encoded by skip-attention mechanism at different resolutions, we propose a novel structure-preserving decoder with hierarchical folding for complete shape generation. The hierarchical folding preserves the structure of complete point cloud generated in upper layer by progressively detailing the local regions, using the skip-attentioned geometry at the same resolution. We conduct comprehensive experiments on ShapeNet and KITTI datasets, which demonstrate that the proposed SA-Net outperforms the state-of-the-art point cloud completion methods.

Point cloud completion aims at predicting dense complete 3D shapes from sparse incomplete point clouds captured from 3D sensors or scanners. It plays an essential role in various applications such as autonomous driving, 3D reconstruction, augmented reality, and robot navigation. Existing point cloud completion methods follow the encoder-decoder paradigm, in which the complete point clouds are recovered in a coarse-to-fine strategy. However, only using the global feature is difficult and will lead to blurring of the global structure and distortion of local details. To address this problem, we propose a novel Partial-to-Partial Point Generation Network ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{P}^{2}$</tex-math></inline-formula> GNet), a learning-based approach for point cloud completion. In <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{P}^{2}$</tex-math></inline-formula> GNet, we use a feature disentangle encoder to obtain the global feature, and missing code and novel view partial point cloud are generated conditioned on the view-related missing code. To better aggregate partial point clouds, an attentive sampling module is proposed to sample multiple partial point clouds into the final complete result. Extensive experiments on several public benchmarks demonstrate that our <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{P}^{2}$</tex-math></inline-formula> GNet outperforms state-of-the-art point cloud completion methods.

A point contextual transformer network for point cloud completion

Point cloud completion aims to predict a complete shape in high accuracy from its partial observation. However, previous methods usually suffered from discrete nature of point cloud and unstructured prediction of points in local regions, which makes it hard to reveal fine local geometric details on the complete shape. To resolve this issue, we propose SnowflakeNet with Snowflake Point Deconvolution (SPD) to generate the complete point clouds. The SnowflakeNet models the generation of complete point clouds as the snowflake-like growth of points in 3D space, where the child points are progressively generated by splitting their parent points after each SPD. Our insight of revealing detailed geometry is to introduce skip-transformer in SPD to learn point splitting patterns which can fit local regions the best. Skip-transformer leverages attention mechanism to summarize the splitting patterns used in the previous SPD layer to produce the splitting in the current SPD layer. The locally compact and structured point cloud generated by SPD is able to precisely capture the structure characteristic of 3D shape in local patches, which enables the network to predict highly detailed geometries, such as smooth regions, sharp edges and corners. Our experimental results outperform the state-of-the-art point cloud completion methods under widely used benchmarks. Code will be available at https://github.com/AllenXiangX/SnowflakeNet.

Point cloud completion reconstructs incomplete, sparse inputs into complete 3D shapes. However, in the current 3D completion task, it is difficult to effectively extract the local details of an incomplete one, resulting in poor restoration of local details and low accuracy of the completed point clouds. To address this problem, this paper proposes a multi-scale point cloud completion method based on local neighborhood dynamic fusion (LNDF: adaptive aggregation of multi-scale local features through dynamic range and weight adjustment). Firstly, the farthest point sampling (FPS) strategy is applied to the original incomplete and defective point clouds for down-sampling to obtain three types of point clouds at different scales. When extracting features from point clouds of different scales, the local neighborhood aggregation of key points is dynamically adjusted, and the Transformer architecture is integrated to further enhance the correlation of local feature extraction information. Secondly, by combining the method of generating point clouds layer by layer in a pyramid-like manner, the local details of the point clouds are gradually enriched from coarse to fine to achieve point cloud completion. Finally, when designing the decoder, inspired by the concept of generative adversarial networks (GANs), an attention discriminator designed in series with a feature extraction layer and an attention layer is added to further optimize the completion performance of the network. Experimental results show that LNDM-Net reduces the average Chamfer Distance (CD) by 5.78% on PCN and 4.54% on ShapeNet compared to SOTA. The visualization of completion results demonstrates the superior performance of our method in both point cloud completion accuracy and local detail preservation. When handling diverse samples and incomplete point clouds in real-world 3D scenarios from the KITTI dataset, the approach exhibits enhanced generalization capability and completion fidelity.

During the point clouds collection process, the errors can be easily introduced into dataset, leading to biases in subsequent analyses, so the completion of point clouds is a critical step of point cloud analysis. In this paper, we introduce a conditional diffusion model architecture (PCDM) to solve point cloud completion problem. The diffusion model has shown remarkable success in image generation and has recently started to be applied in other domains, showing amazing results. By harnessing the potent generative abilities of diffusion models, we progressively get the complete point clouds derived from pure noise data. In the denosing process, we employ a Local-Global Net (LoGNet) to fuse global and local features, guiding the generation of point clouds by the model. Furthermore, to strike a balance between the completeness of the generated point cloud data and local details, we introduce an offset-attention mechanism to extract features from incomplete point clouds. Experiments conducted on multiple public datasets demonstrate that the point cloud completion method proposed in this paper outperforms previous approaches.

Tree structural information is essential for studying forest ecosystem functions, driving mechanisms, and global change response mechanisms. Although current terrestrial laser scanning (TLS) can acquire high-precision 3D structural information of forests, mutual occlusion between trees, the scanner’s field of view, and terrain changes make the point clouds captured by laser scanning sensors incomplete, further hindering downstream tasks. This study proposes a skeleton-embedded tree point cloud completion method, termed SK-TreePCN, which recovers complete individual tree point clouds from incomplete scanning data in the field. SK-TreePCN employs a transformer trained on simulated point clouds generated by a 3D radiative transfer model. Unlike existing point cloud completion algorithms designed for regular shapes and simple structures, the SK-TreePCN method addresses structurally heterogeneous trees. The 3D radiative transfer model LESS, which can simulate various TLS data over highly heterogeneous scenes, is employed to generate massive point clouds with training labels. Among the various point cloud completion methods evaluated, SK-TreePCN exhibits outstanding performance regarding the Chamfer distance (CD) and F1 Score. The generated point clouds display a more natural appearance and clearer branches. The accuracy of tree height and diameter at breast height extracted from the recovered point cloud achieved R2 values of 0.929 and 0.904, respectively. SK-TreePCN demonstrates applicability and robustness in recovering individual tree point clouds. It demonstrated great potential for TLS-based field measurements of trees, refining point cloud 3D reconstruction and tree information extraction and reducing field data collection labor while retaining satisfactory data quality.

ABSTRACTWith the development of the 3D point cloud field in recent years, point cloud completion of 3D objects has increasingly attracted researchers' attention. Point cloud data can accurately express the shape information of 3D objects at different resolutions, but the original point clouds collected directly by various 3D scanning equipment are often incomplete and have uneven density. Tactile is one distinctive way to perceive the 3D shape of an object. Tactile point clouds can provide local shape information for unknown areas during completion, which is a valuable complement to the point cloud data acquired with visual devices. In order to effectively improve the effect of point cloud completion using tactile information, the authors propose an innovative tactile‐assisted point cloud completion network, TAPCNet. This network is the first neural network customised for the input of tactile point clouds and incomplete point clouds, which can fuse two types of point cloud information in the feature domain. Besides, a new dataset named 3DVT was rebuilt, to fit the proposed network model. Based on the tactile fusion strategy and related modules, multiple comparative experiments were conducted by controlling the quantity of tactile point clouds on the 3DVT dataset. The experimental data illustrates that TAPCNet can outperform the state‐of‐the‐art methods in the benchmark.