Large-scale Point Clouds Research Articles

Semantic segmentation of large-scale point cloud scenes via dual neighborhood feature and global spatial-aware

As a core task in 3D scene information extraction, point cloud semantic segmentation is crucial for understanding 3D scenes and environmental perception. While extracting local geometric structural features from point clouds, existing research often overlooks the long-range dependencies present in the scene, making it challenging to fully uncover the long-range contextual features hidden within point clouds. On this basis, we propose a segmentation algorithm (DG-Net) that integrates dual neighborhood features with global spatial-aware. Initially, the local structure information encoding module is designed to learn about local geometric shapes by encoding spatial position and directional features, thus supplementing structural information. Subsequently, a dual neighborhood features complementary module is introduced to merge the geometric structural and semantic features within local neighborhoods, learning local dependencies and capturing distinguishable local contextual features. Finally, these features are relayed to a global spatial-aware module equipped with a gated unit, which dynamically adjusts the weights of features at different stages, effectively modeling long-range dependencies between local structures and finely extracting long-range contextual features. We conducted experiments on benchmark datasets of point cloud scenes, and both quantitative and qualitative results demonstrate that our algorithm can accurately identify small-scale objects with complex geometric structures within scenes, surpassing other mainstream networks in segmentation performance. The mIoU on the S3DIS, Toronto3D, and SensatUrban datasets are 71.9 %, 82.1 %, and 59.8 %, respectively.

A large-scale point cloud semantic segmentation neural network based on long-range contextual dependencies enhancement

ABSTRACT This letter presents a long-range contextual dependency enhanced network (LCDE-Net) for semantic segmentation of large-scale point cloud, which employs a U-shaped framework. Firstly, point clouds are subsampled with grid sampling and fed into convolutional layers to learn more representative local features of points. Then global and local encoders (GLE) are designed to exploit long-range contextual dependencies and local features simultaneously. The core of GLE consists of two parts: global feature enhancement (GFE) module and feature channel modulation (FCM) module. Secondly, through decoder layers, the encoded features are upsampled through the nearest-neighbour interpolation and aggregated with the intermediate encoded features by skip connections to capture multi-scale discriminative features for semantic segmentation of point cloud. Finally, via Fully Connection layer and Softmax classifier, each point’s label is assigned. Two different benchmark datasets are conducted to evaluate the performance of the proposed method, Experimental results report that the proposed LCDE-Net achieves 78.6% in terms of mean intersection over union (mIoU) on Semantic3D, and 68.2% on S3DIS, which is the highest among the comparison methods. The code of LCDE-Net is available at https://github.com/xrzmyz/LCDE-Net.

Semantic segmentation of large-scale point clouds by integrating attention mechanisms and transformer models

In the current field of point cloud processing, semantic segmentation of large-scale point clouds remains a challenging problem. Traditional methods often underperform when faced with the complexity and density variations present in large-scale point cloud data. This study introduces an innovative model designed for semantic segmentation of large-scale point clouds. The model integrates a CNN-Transformer-based Context Aggregation Module with a Slot Attention mechanism to enhance the understanding of entity relationships and improve semantic segmentation performance. Furthermore, by flexibly adjusting the weight parameters within the loss function, we have optimized the training process, increasing its flexibility. In our experiments, we compared the proposed model with 55 classical and contemporary point cloud semantic segmentation models, evaluating semantic segmentation performance, computational resource consumption, inference time, and the number of model parameters. The results show a significant improvement in performance on the S3DIS and Semantic3D datasets, specifically achieving an mIoU of 71.53% and an OA of 93.92%. Ablation studies were conducted to further ascertain the contribution of each module to the overall performance of the model. This study presents an innovative approach to overcoming the challenges of large-scale point cloud semantic segmentation, offering significant contributions to the advancement of point cloud processing.

FLNA: Flexibly Accelerating Feature Learning Networks for Large-Scale Point Clouds With Efficient Dataflow Decoupling

FLNA: Flexibly Accelerating Feature Learning Networks for Large-Scale Point Clouds With Efficient Dataflow Decoupling

Efficient low-overlapping point cloud registration based on two-stage network learning

ABSTRACT This letter introduces an efficient registration method for large-scale point clouds with low overlap, which adopts a two-stage transformation learning process. In the first stage, we integrate a point-voxel encoding structure with an attention mechanism to predict rotation and point-wise importance weight. In the second stage, we employ a PointNet-based structure to estimate translation using optimized data. By adopting this two-stage procedure, our model can effectively learn rotation and translation separately, thereby increasing flexibility. We conducted extensive experiments on two large-scale indoor datasets (3Dmatch and S3DIS) and an outdoor dataset (KITTI odometry). The results demonstrate that our method outperforms traditional and deep learning-based approaches, achieving state-of-the-art performance in registration. Furthermore, our method exhibits superior generalization capabilities, highlighting its effectiveness in various registration scenarios.

Fast Inter-frame Motion Prediction for Compressed Dynamic Point Cloud Attribute Enhancement

Recent years have witnessed the success of deep learning methods in quality enhancement of compressed point cloud. However, existing methods focus on geometry and attribute enhancement of single-frame point cloud. This paper proposes a novel compressed quality enhancement method for dynamic point cloud (DAE-MP). Specifically, we propose a fast inter-frame motion prediction module (IFMP) to explicitly estimate motion displacement and achieve inter-frame feature alignment. To maintain motion continuity between consecutive frames, we propose a motion consistency loss for supervised learning. Furthermore, a frequency component separation and fusion module is designed to extract rich frequency features adaptively. To the best of our knowledge, the proposed method is the first deep learning-based work to enhance the quality for compressed dynamic point cloud. Experimental results show that the proposed method can greatly improve the quality of compressed dynamic point cloud and provide a fast and efficient motion prediction plug-in for large-scale point cloud. For dynamic point cloud attribute with severely compressed artifact, our proposed DAE-MP method achieves up to 0.52dB (PSNR) performance gain. Moreover, the proposed IFMP module has a certain real-time processing ability for calculating the motion offset between dynamic point cloud frame.

Large-scale Point Cloud Segmentation based on Multi-feature Local Enhanced Fusion

This paper introduces a framework for large-scale 3D point cloud semantic segmentation - the MLEF-Net model. The model aims to improve the segmentation accuracy of large-scale point clouds by innovatively combining Manhattan distance-based KNN neighborhood search with feature aggregation techniques. This approach uniquely handles spatial, color, and normal vector attributes, thereby improving the segmentation results. The superiority of the model is validated through comprehensive testing on the SemanticKITTI and nuScenes datasets, demonstrating its potential to enhance point cloud segmentation through advanced feature fusion strategies.

Recognizing Predictive Substructures With Subgraph Information Bottleneck.

The emergence of Graph Convolutional Network (GCN) has greatly boosted the progress of graph learning. However, two disturbing factors, noise and redundancy in graph data, and lack of interpretation for prediction results, impede further development of GCN. One solution is to recognize a predictive yet compressed subgraph to get rid of the noise and redundancy and obtain the interpretable part of the graph. This setting of subgraph is similar to the information bottleneck (IB) principle, which is less studied on graph-structured data and GCN. Inspired by the IB principle, we propose a novel subgraph information bottleneck (SIB) framework to recognize such subgraphs, named IB-subgraph. However, the intractability of mutual information and the discrete nature of graph data makes the objective of SIB notoriously hard to optimize. To this end, we introduce a bilevel optimization scheme coupled with a mutual information estimator for irregular graphs. Moreover, we propose a continuous relaxation for subgraph selection with a connectivity loss for stabilization. We further theoretically prove the error bound of our estimation scheme for mutual information and the noise-invariant nature of IB-subgraph. Extensive experiments on graph learning and large-scale point cloud tasks demonstrate the superior property of IB-subgraph.

Research on Key Technologies for Container Ship Loading Test Based on 3D Laser Scanning

_ Traditional accuracy check methods for cargo hold in container ships rely solely on manual and visual operations, which are time-consuming and resource-intensive. Addressing the challenge of extracting and analyzing key data, such as cell guides and container pedestals, from large-scale point clouds obtained through three-dimensional (3D) laser scanning in container ship trial runs, this paper proposes an algorithmic framework based on 3D laser scanning. Building upon this framework, an improved coordinate-axis filtering RANSAC algorithm is employed to optimize the extraction of cell guide planes. Additionally, an algorithmic process based on Bhattacharyya distance is utilized to automatically extract container pedestal point clouds. Furthermore, a combination of the genetic algorithm and the ICP algorithm is proposed to achieve the fitting of the container pedestal edge contour through point cloud registration. Experimental results demonstrate the consistency between the extracted cell guide and container pedestal data and the actual results, indicating the high practical value of the proposed methodology. Keywords container ship loading test; cargo hold accuracy check; 3D point cloud processing

Accurate and complete line segment extraction for large-scale point clouds

Line segment extraction from point clouds is critical for analyzing and understanding large-scale scenes. The main challenge is to generate line segments accurately as well as completely. However, state-of-the-art approaches continue to struggle with this issue. In this paper, we propose a novel method for effectively generating line segments from large-scale point clouds. To this end, we design a weighted centroid displacement scheme for identifying comprehensive feature points. Then, we employ an L1-median optimization to refine the identified feature points to perceive geometric edges on the underlying surface accurately. Following that, we generate complete and concise line segments from the refined feature points by designing three geometric operators: clustering, exclusion, and assimilation. The clustering operator generates the initial line segments based on optimized feature points, and the exclusion operator and the assimilation operator ensure the completeness and continuity of these line segments. We evaluate our approach on various scene point clouds, such as TLS, MLS, and ALS data. Extensive experimental results show that our method can outperform the competing approaches in terms of both accuracy and efficiency.

A Deep-Learning-Based Method for Extracting an Arbitrary Number of Individual Power Lines from UAV-Mounted Laser Scanning Point Clouds

In recent years, laser scanners integrated with Unmanned Aerial Vehicles (UAVs) have exhibited great potential in conducting power line inspections in harsh environments. The point clouds collected for power line inspections have numerous advantages over remote image data. However, point cloud-based individual power line extraction, which is a crucial technology required for power line inspections, still poses several challenges such as massive 3D points, imbalanced category points, etc. Moreover, in various power line scenarios, previous studies often require manual setup and careful adjustment of different thresholds to separate different power lines, which is inefficient for practical applications. To handle these challenges, in this paper, we propose a multi-branch network to automatically extract an arbitrary number of individual power lines from point clouds collected by UAV-based laser scanners. Specifically, to handle the massive 3D point clouds in complex outdoor scenarios, we propose to leverage deep neural network for efficient and rapid feature extraction in large-scale point clouds. To mitigate imbalanced data quantities across different categories, we propose to design a weighted cross-entropy loss function to measure the varying importance of each category. To achieve the effective extraction of an arbitrary number of power lines, we propose leveraging a loss function to learn the discriminative features that can differentiate the points belonging to different power lines. Once the discriminative features are learned, the Mean Shift method can distinguish the individual power lines by clustering without supervision. The evaluations are executed on two datasets, which are acquired at different locations with UAV-mounted laser scanners. The proposed method has been thoroughly tested and evaluated, and the results and discussions confirm its outstanding ability to extract an arbitrary number of individual power lines in point clouds.

A Large-Scale Network Construction and Lightweighting Method for Point Cloud Semantic Segmentation.

To significantly enhance the performance of point cloud semantic segmentation, this manuscript presents a novel method for constructing large-scale networks and offers an effective lightweighting technique. First, a latent point feature processing (LPFP) module is utilized to interconnect base networks such as PointNet++ and Point Transformer. This intermediate module serves both as a feature information transfer and a ground truth supervision function. Furthermore, in order to alleviate the increase in computational costs brought by constructing large-scale networks and better adapt to the demand for terminal deployment, a novel point cloud lightweighting method for semantic segmentation network (PCLN) is proposed to compress the network by transferring multidimensional feature information of large-scale networks. Specifically, at different stages of the large-scale network, the structure and attention information of the point features are selectively transferred to guide the compressed network to train in the direction of the large-scale network. This paper also solves the problem of representing global structure information of large-scale point clouds through feature sampling and aggregation. Extensive experiments on public datasets and real-world data demonstrate that the proposed method can significantly improve the performance of different base networks and outperform the state-of-the-art.

Dense Dual-Branch Cross Attention Network for Semantic Segmentation of Large-Scale Point Clouds

Dense Dual-Branch Cross Attention Network for Semantic Segmentation of Large-Scale Point Clouds

Enhanced Local Feature Learning With Simple Offset Attention for Semantic Segmentation of Large-Scale Point Clouds

Enhanced Local Feature Learning With Simple Offset Attention for Semantic Segmentation of Large-Scale Point Clouds

PointNAT: Large-Scale Point Cloud Semantic Segmentation via Neighbor Aggregation With Transformer

PointNAT: Large-Scale Point Cloud Semantic Segmentation via Neighbor Aggregation With Transformer

Semantic segmentation of large-scale point clouds with neighborhood uncertainty

Semantic segmentation of large-scale point clouds with neighborhood uncertainty

Prediction of Transonic Flow over Cascades via Graph Embedding Methods on Large-Scale Point Clouds

In this research, we introduce a deep-learning-based framework designed for the prediction of transonic flow through a linear cascade utilizing large-scale point-cloud data. In our experimental cases, the predictions demonstrate a nearly four-fold speed improvement compared to traditional CFD calculations while maintaining a commendable level of accuracy. Taking advantage of a multilayer graph structure, the framework can extract both global and local information from the cascade flow field simultaneously and present prediction over unstructured data. In line with the results obtained from the test datasets, we conducted an in-depth analysis of the geometric attributes of the cascades reconstructed using our framework, considering adjustments made to the geometric information of the point cloud. We fine-tuned the input using 1603 data points and quantified the contribution of each point. The outcomes reveal that variations in the suction side of the cascade have a significantly more substantial influence on the field results compared to the pressure side and explain the way graph neural networks work for cascade flow-field prediction, enhancing the comprehension of graph-based flow-field prediction among developers and proves the potential of graph neural networks in flow-field prediction on large-scale point clouds and design.

SSRNet: Scalable 3D Surface Reconstruction Network.

Learning-based surface reconstruction methods have received considerable attention in recent years due to their excellent expressiveness. However, existing learning-based methods lack scalability in processing large-scale point clouds. This paper proposes a novel scalable learning-based 3D surface reconstruction method based on octree, called SSRNet. SSRNet works in a scalable reconstruction pipeline, which divides oriented point clouds into different local parts and then processes them in parallel. Accommodating this scalable design pattern, SSRNet constructs local geometric features for octree vertices. Such features comprise the relation between the vertices and the implicit surface, ensuring geometric perception. Focusing on local geometric information also enables the network to avoid the overfitting problem and generalize well on different datasets. Finally, as a learning-based method, SSRNet can process large-scale point clouds in a short time. And to further solve the efficiency problem, we provide a lightweight and efficient version that is about five times faster while maintaining reconstruction performance. Experiments show that our methods achieve state-of-the-art performance with outstanding efficiency.

Towards robustness and generalization of point cloud representation: A geometry coding method and a large-scale object-level dataset

Robustness and generalization are two challenging problems for learning point cloud representation. To tackle these problems, we first design a novel geometry coding model, which can effectively use an invariant eigengraph to group points with similar geometric information, even when such points are far from each other. We also introduce a large-scale point cloud dataset, PCNet184. It consists of 184 categories and 51,915 synthetic objects, which brings new challenges for point cloud classification, and provides a new benchmark to assess point cloud cross-domain generalization. Finally, we perform extensive experiments on point cloud classification, using ModelNet40, ScanObjectNN, and our PCNet184, and segmentation, using ShapeNetPart and S3DIS. Our method achieves comparable performance to state-of-the-art methods on these datasets, for both supervised and unsupervised learning. Code and our dataset are available at https://github.com/MingyeXu/PCNet184.

BEMF-Net: Semantic Segmentation of Large-Scale Point Clouds via Bilateral Neighbor Enhancement and Multi-Scale Fusion

The semantic segmentation of point clouds is a crucial undertaking in 3D reconstruction and holds great importance. However, achieving precise semantic segmentation represents a significant hurdle. In this paper, we present BEMF-Net, an efficient method for large-scale environments. It starts with an effective feature extraction method. Unlike images, 3D data comprise not only geometric relations but also texture information. To accurately depict the scene, it is crucial to take into account the impacts of texture and geometry on the task, and incorporate modifications to improve feature description. Additionally, we present a multi-scale feature fusion technique that effectively promotes the interaction between features at different resolutions. The approach mitigates the problem of the smoothing of detailed information caused by downsampling mechanisms, while ensuring the integrity of features across different layers, allowing a more comprehensive representation of the point cloud. We confirmed the effectiveness of this method by testing it on benchmark datasets such as S3DIS, SensatUrban, and Toronto3D.