Point Projection Network: A Multi-View-Based Point Completion Network with Encoder-Decoder Architecture

A point contextual transformer network for point cloud completion

LiDAR is an active remote sensing technology widely used in forestry applications, such as forest resource surveys, tree information collection, and ecosystem monitoring. However, due to the resolution limitations of 3D-laser scanners and the canopy occlusion in forest environments, the tree point clouds obtained often have missing data. This can reduce the accuracy of individual tree segmentation, which subsequently affects the tree species classification. To address the issue, this study used point cloud data with RGB information collected by the UAV platform to improve tree species classification by completing the missing point clouds. Furthermore, the study also explored the effects of point cloud completion, feature selection, and classification methods on the results. Specifically, both a traditional geometric method and a deep learning-based method were used for point cloud completion, and their performance was compared. For the classification of tree species, five machine learning algorithms—Random Forest (RF), Support Vector Machine (SVM), Back Propagation Neural Network (BPNN), Quadratic Discriminant Analysis (QDA), and K-Nearest Neighbors (KNN)—were utilized. This study also ranked the importance of features to assess the impact of different algorithms and features on classification accuracy. The results showed that the deep learning-based completion method provided the best performance (avgCD = 6.14; avgF1 = 0.85), generating more complete point clouds than the traditional method. On the other hand, compared with SVM and BPNN, RF showed better performance in dealing with multi-classification tasks with limited training samples (OA-87.41%, Kappa-0.85). Among the six dominant tree species, Pinus koraiensis had the highest classification accuracy (93.75%), while that of Juglans mandshurica was the lowest (82.05%). In addition, the vegetation index and the tree structure parameter accounted for 50% and 30%, respectively, in the top 10 features in terms of feature importance. The point cloud intensity also had a high contribution to the classification results, indicating that the lidar point cloud data can also be used as an important basis for tree species classification.

Tree structural information is essential for studying forest ecosystem functions, driving mechanisms, and global change response mechanisms. Although current terrestrial laser scanning (TLS) can acquire high-precision 3D structural information of forests, mutual occlusion between trees, the scanner’s field of view, and terrain changes make the point clouds captured by laser scanning sensors incomplete, further hindering downstream tasks. This study proposes a skeleton-embedded tree point cloud completion method, termed SK-TreePCN, which recovers complete individual tree point clouds from incomplete scanning data in the field. SK-TreePCN employs a transformer trained on simulated point clouds generated by a 3D radiative transfer model. Unlike existing point cloud completion algorithms designed for regular shapes and simple structures, the SK-TreePCN method addresses structurally heterogeneous trees. The 3D radiative transfer model LESS, which can simulate various TLS data over highly heterogeneous scenes, is employed to generate massive point clouds with training labels. Among the various point cloud completion methods evaluated, SK-TreePCN exhibits outstanding performance regarding the Chamfer distance (CD) and F1 Score. The generated point clouds display a more natural appearance and clearer branches. The accuracy of tree height and diameter at breast height extracted from the recovered point cloud achieved R2 values of 0.929 and 0.904, respectively. SK-TreePCN demonstrates applicability and robustness in recovering individual tree point clouds. It demonstrated great potential for TLS-based field measurements of trees, refining point cloud 3D reconstruction and tree information extraction and reducing field data collection labor while retaining satisfactory data quality.

The task of 3D point cloud completion is to predict a complete point cloud from the incomplete partial point cloud. Generally, the encoder is used to extract the global shape features of the input incomplete point cloud, and then the decoder infers the complete point cloud. At present, some methods have been improved by multi-resolution encoders and multi-layer decoders, and achieved obvious results. However, these methods still cannot fully express the shape features. In order to solve this problem, we propose a feature fusion mechanism based on skip connection. The features extracted from each resolution point cloud are connected with the input of corresponding decoder. Then they are weighted and fused to obtain denser features, which can be decoded into a finer point cloud. In addition, the current loss function is still not a good measure of the similarity between two point clouds, so we also proposed a multi-stage local average Hausdroff Loss to form a joint reconstruction loss function to guide the generation of missing point clouds. Experimental results prove the effectiveness of our method in point cloud completion tasks, and show that it products better performance than existing methods.

DPG-Net: Densely progressive-growing network for point cloud completion

Acquiring semantics directly from a point cloud is an important requirement for handling point cloud tasks. However, point clouds captured with laser scanner equipment are often incomplete due to the limitations posed by target occlusion and light reflection. Consequently, recovering the complete point clouds from partial and sparse ones is essential for further studies. In this paper, we model a novel projected generative adversarial network (PGAN) for point cloud completion. First, we present a multi-scale generator module (MSGM) to fully capture the local structures and global shape in the raw incompletion point cloud and generate the multi-scale complete point cloud. In contrast to existing point cloud feature extractors, our MSGM promotes a correlation between different regions of an incomplete point cloud and integrates the contextual information of the point cloud. Second, we observe that the existing point discriminator is inadequate to enhance the discrimination of the prediction point cloud. To address this problem, we project the completed point cloud to 2D maps and apply adversarial training to discriminate the geometrical shape from a specific viewpoint. Comprehensive experiments on the ShapeNet and ModelNet40 datasets show that the proposed method performs well against existing point cloud completion tasks. We also present an ablation study to demonstrate the advantages of the projected generative adversarial network.

With the continuous advancement of 3D perception technology, point cloud data has found increasingly widespread application. However, the presence of holes in point cloud data caused by device limitations and environmental interference severely restricts algorithmic performance, making point cloud completion a research topic of high interest. This study observes that most existing mainstream point cloud completion methods primarily focus on capturing global features, while often underrepresenting local structural details. Moreover, the generation process of complete point clouds lacks effective control over fine-grained features, leading to insufficient detail in the completed outputs and reduced data integrity. To address these issues, we propose a Set Combination Multi-Layer Perceptron (SCMP) module that enables the simultaneous extraction of both local and global features, thereby reducing the loss of local detail information. In addition, we introduce the Squeeze Excitation Pooling Network (SEP-Net) module, an informative channel attention mechanism capable of adaptively identifying and enhancing critical channel features, thus improving the overall feature representation capability. Based on these modules, we further design a novel Feature Fusion Point Fractal Network (FFPF-Net), which fuses multi-dimensional point cloud features to enhance representation capacity and progressively refines the missing regions to generate a more complete point cloud. Extensive experiments conducted on the ShapeNet-Part and MVP datasets compared to L-GAN and PCN showed average prediction error improvements of 1.3 and 1.4, respectively. The average completion errors on the ShapeNet-Part and MVP datasets are 0.783 and 0.824, highlighting the improved fine-detail reconstruction capability of our network. These results indicate that the proposed method effectively enhances point cloud completion performance and can further promote the practical application of point cloud data in various real-world scenarios.

CarvingNet: Point cloud completion by stepwise refining multi-resolution features

To address the issue that point cloud data is often incomplete and difficult to obtain, we propose a point cloud completion method to improve the PoinTr method based on feature enhancement. In dataset preprocessing, the farthest point of the original point cloud is sampled to obtain the central point coordinates. Our method constructs an MLP network, where the local information of these central points is obtained and the location embedding is performed. Combining network and SENet network, the local features of the point cloud are extracted and enhanced, and the location embedding and local features are added to obtain the point proxies of the original point cloud. Afterward, our method predicts the missing part of the point cloud by using an Encoder to model the relationship between the point cloud structure information and points, and then using a Decoder to learn the relationship between the missing and existing parts of the point cloud and reconstruct the missing point cloud. Our method also modifies the attention mechanism to make the features more global and enhance the network expression. Finally, the point cloud is refined, and is realized by predicting multiple points around each point of the coarse point cloud through the FoldingNet network, and the final output is the complete point cloud. Experimental results show that the proposed method can not only reduce the performance overhead, but also improve the effects of point cloud completion.

GMP-Net: Graph based Missing Part Patching Network for Point Cloud Completion

Recent advances in depth measurement and its utilization have made point cloud processing more critical. Additionally, the human head is essential for communication, and its three-dimensional data are expected to be utilized in this regard. However, a single RGB-Depth (RGBD) camera is prone to occlusion and depth measurement failure for dark hair colors such as black hair. Recently, point cloud completion, where an entire point cloud is estimated and generated from a partial point cloud, has been studied, but only the shape is learned, rather than the completion of colored point clouds. Thus, this paper proposes a machine learning-based completion method for colored point clouds with XYZ location information and the International Commission on Illumination (CIE) LAB () color information. The proposed method uses the color difference between point clouds based on the Chamfer Distance (CD) or Earth Mover’s Distance (EMD) of point cloud shape evaluation as a color loss. In addition, an adversarial loss to images rendered from the output point cloud can improve the visual quality. The experiments examined networks trained using a colored point cloud dataset created by combining two 3D datasets: hairstyles and faces. Experimental results show that using the adversarial loss with the colored point cloud renderer in the proposed method improves the image domain’s evaluation.

SPC: Self-supervised point cloud completion.

Point cloud completion aims to complete partial point clouds captured from the real world, which is a crucial step in the pipeline of many point cloud tasks. Among the existing methods for solving this problem, SnowflakeNet is the most outstanding. However, SnowflakeNet cannot recover the detailed structure of point clouds in latent code because it uses many max-pooling operations in the encoding stage. Therefore, we propose an improved architecture to effectively acquire and preserve more detail information from input point clouds, thereby enhancing the quality of point cloud completion. Specifically, the improved lightweight DGCNN is added to the encoder to extract local features. The geometric perception block of PoinTr is introduced to extract the global features of the point cloud, which can fully model the structural information and inter-point relationships of known point clouds. The new optimizer Adan is also used in the training process to complete the partial point clouds. Comparative experiments on Completion3D and PCN datasets show that our method is better than most current point cloud completion methods. Our method has the ability to produce the entire shape with details, including but not only smooth surfaces, well-defined edges, and distinct corners.

Point clouds obtained from laser scanners or other devices often exhibit incompleteness, which poses a challenge for subsequent point cloud processing. Therefore, accurately predicting the complete shape from partial observations has paramount significance. In this paper, we introduce PCCDiff, a probabilistic model inspired by Denoising Diffusion Probabilistic Models (DDPMs), designed for point cloud completion tasks. Our model aims to predict missing parts in incomplete 3D shapes by learning the reverse diffusion process, transforming a 3D Gaussian noise distribution into the desired shape distribution without any structural assumption (e.g., geometric symmetry). Firstly, we design a conditional point cloud completion network that integrates Missing-Transformer and TreeGCN, facilitating the prediction of complete point cloud features. Subsequently, at each step of the diffusion process, the obtained point cloud features serve as condition inputs for the symmetric Diffusion ResUNet. By incorporating these condition features and incomplete point clouds into the diffusion process, PCCDiff demonstrates superior generation performance compared to other methods. Finally, extensive experiments are conducted to demonstrate the effectiveness of our proposed generative model for completing point clouds.

Point cloud completion aims at predicting dense complete 3D shapes from sparse incomplete point clouds captured from 3D sensors or scanners. It plays an essential role in various applications such as autonomous driving, 3D reconstruction, augmented reality, and robot navigation. Existing point cloud completion methods follow the encoder-decoder paradigm, in which the complete point clouds are recovered in a coarse-to-fine strategy. However, only using the global feature is difficult and will lead to blurring of the global structure and distortion of local details. To address this problem, we propose a novel Partial-to-Partial Point Generation Network ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{P}^{2}$</tex-math></inline-formula> GNet), a learning-based approach for point cloud completion. In <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{P}^{2}$</tex-math></inline-formula> GNet, we use a feature disentangle encoder to obtain the global feature, and missing code and novel view partial point cloud are generated conditioned on the view-related missing code. To better aggregate partial point clouds, an attentive sampling module is proposed to sample multiple partial point clouds into the final complete result. Extensive experiments on several public benchmarks demonstrate that our <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{P}^{2}$</tex-math></inline-formula> GNet outperforms state-of-the-art point cloud completion methods.