Large-scale Point Clouds Research Articles

A large-scale point cloud semantic segmentation neural network based on long-range contextual dependencies enhancement

ABSTRACT This letter presents a long-range contextual dependency enhanced network (LCDE-Net) for semantic segmentation of large-scale point cloud, which employs a U-shaped framework. Firstly, point clouds are subsampled with grid sampling and fed into convolutional layers to learn more representative local features of points. Then global and local encoders (GLE) are designed to exploit long-range contextual dependencies and local features simultaneously. The core of GLE consists of two parts: global feature enhancement (GFE) module and feature channel modulation (FCM) module. Secondly, through decoder layers, the encoded features are upsampled through the nearest-neighbour interpolation and aggregated with the intermediate encoded features by skip connections to capture multi-scale discriminative features for semantic segmentation of point cloud. Finally, via Fully Connection layer and Softmax classifier, each point’s label is assigned. Two different benchmark datasets are conducted to evaluate the performance of the proposed method, Experimental results report that the proposed LCDE-Net achieves 78.6% in terms of mean intersection over union (mIoU) on Semantic3D, and 68.2% on S3DIS, which is the highest among the comparison methods. The code of LCDE-Net is available at https://github.com/xrzmyz/LCDE-Net.

Semantic segmentation of large-scale point clouds by integrating attention mechanisms and transformer models

In the current field of point cloud processing, semantic segmentation of large-scale point clouds remains a challenging problem. Traditional methods often underperform when faced with the complexity and density variations present in large-scale point cloud data. This study introduces an innovative model designed for semantic segmentation of large-scale point clouds. The model integrates a CNN-Transformer-based Context Aggregation Module with a Slot Attention mechanism to enhance the understanding of entity relationships and improve semantic segmentation performance. Furthermore, by flexibly adjusting the weight parameters within the loss function, we have optimized the training process, increasing its flexibility. In our experiments, we compared the proposed model with 55 classical and contemporary point cloud semantic segmentation models, evaluating semantic segmentation performance, computational resource consumption, inference time, and the number of model parameters. The results show a significant improvement in performance on the S3DIS and Semantic3D datasets, specifically achieving an mIoU of 71.53% and an OA of 93.92%. Ablation studies were conducted to further ascertain the contribution of each module to the overall performance of the model. This study presents an innovative approach to overcoming the challenges of large-scale point cloud semantic segmentation, offering significant contributions to the advancement of point cloud processing.

FLNA: Flexibly Accelerating Feature Learning Networks for Large-Scale Point Clouds With Efficient Dataflow Decoupling

FLNA: Flexibly Accelerating Feature Learning Networks for Large-Scale Point Clouds With Efficient Dataflow Decoupling

Efficient low-overlapping point cloud registration based on two-stage network learning

ABSTRACT This letter introduces an efficient registration method for large-scale point clouds with low overlap, which adopts a two-stage transformation learning process. In the first stage, we integrate a point-voxel encoding structure with an attention mechanism to predict rotation and point-wise importance weight. In the second stage, we employ a PointNet-based structure to estimate translation using optimized data. By adopting this two-stage procedure, our model can effectively learn rotation and translation separately, thereby increasing flexibility. We conducted extensive experiments on two large-scale indoor datasets (3Dmatch and S3DIS) and an outdoor dataset (KITTI odometry). The results demonstrate that our method outperforms traditional and deep learning-based approaches, achieving state-of-the-art performance in registration. Furthermore, our method exhibits superior generalization capabilities, highlighting its effectiveness in various registration scenarios.

Fast Inter-frame Motion Prediction for Compressed Dynamic Point Cloud Attribute Enhancement

Recent years have witnessed the success of deep learning methods in quality enhancement of compressed point cloud. However, existing methods focus on geometry and attribute enhancement of single-frame point cloud. This paper proposes a novel compressed quality enhancement method for dynamic point cloud (DAE-MP). Specifically, we propose a fast inter-frame motion prediction module (IFMP) to explicitly estimate motion displacement and achieve inter-frame feature alignment. To maintain motion continuity between consecutive frames, we propose a motion consistency loss for supervised learning. Furthermore, a frequency component separation and fusion module is designed to extract rich frequency features adaptively. To the best of our knowledge, the proposed method is the first deep learning-based work to enhance the quality for compressed dynamic point cloud. Experimental results show that the proposed method can greatly improve the quality of compressed dynamic point cloud and provide a fast and efficient motion prediction plug-in for large-scale point cloud. For dynamic point cloud attribute with severely compressed artifact, our proposed DAE-MP method achieves up to 0.52dB (PSNR) performance gain. Moreover, the proposed IFMP module has a certain real-time processing ability for calculating the motion offset between dynamic point cloud frame.

Large-scale Point Cloud Segmentation based on Multi-feature Local Enhanced Fusion

This paper introduces a framework for large-scale 3D point cloud semantic segmentation - the MLEF-Net model. The model aims to improve the segmentation accuracy of large-scale point clouds by innovatively combining Manhattan distance-based KNN neighborhood search with feature aggregation techniques. This approach uniquely handles spatial, color, and normal vector attributes, thereby improving the segmentation results. The superiority of the model is validated through comprehensive testing on the SemanticKITTI and nuScenes datasets, demonstrating its potential to enhance point cloud segmentation through advanced feature fusion strategies.

Recognizing Predictive Substructures With Subgraph Information Bottleneck.

The emergence of Graph Convolutional Network (GCN) has greatly boosted the progress of graph learning. However, two disturbing factors, noise and redundancy in graph data, and lack of interpretation for prediction results, impede further development of GCN. One solution is to recognize a predictive yet compressed subgraph to get rid of the noise and redundancy and obtain the interpretable part of the graph. This setting of subgraph is similar to the information bottleneck (IB) principle, which is less studied on graph-structured data and GCN. Inspired by the IB principle, we propose a novel subgraph information bottleneck (SIB) framework to recognize such subgraphs, named IB-subgraph. However, the intractability of mutual information and the discrete nature of graph data makes the objective of SIB notoriously hard to optimize. To this end, we introduce a bilevel optimization scheme coupled with a mutual information estimator for irregular graphs. Moreover, we propose a continuous relaxation for subgraph selection with a connectivity loss for stabilization. We further theoretically prove the error bound of our estimation scheme for mutual information and the noise-invariant nature of IB-subgraph. Extensive experiments on graph learning and large-scale point cloud tasks demonstrate the superior property of IB-subgraph.

Research on Key Technologies for Container Ship Loading Test Based on 3D Laser Scanning

_ Traditional accuracy check methods for cargo hold in container ships rely solely on manual and visual operations, which are time-consuming and resource-intensive. Addressing the challenge of extracting and analyzing key data, such as cell guides and container pedestals, from large-scale point clouds obtained through three-dimensional (3D) laser scanning in container ship trial runs, this paper proposes an algorithmic framework based on 3D laser scanning. Building upon this framework, an improved coordinate-axis filtering RANSAC algorithm is employed to optimize the extraction of cell guide planes. Additionally, an algorithmic process based on Bhattacharyya distance is utilized to automatically extract container pedestal point clouds. Furthermore, a combination of the genetic algorithm and the ICP algorithm is proposed to achieve the fitting of the container pedestal edge contour through point cloud registration. Experimental results demonstrate the consistency between the extracted cell guide and container pedestal data and the actual results, indicating the high practical value of the proposed methodology. Keywords container ship loading test; cargo hold accuracy check; 3D point cloud processing

Accurate and complete line segment extraction for large-scale point clouds

Line segment extraction from point clouds is critical for analyzing and understanding large-scale scenes. The main challenge is to generate line segments accurately as well as completely. However, state-of-the-art approaches continue to struggle with this issue. In this paper, we propose a novel method for effectively generating line segments from large-scale point clouds. To this end, we design a weighted centroid displacement scheme for identifying comprehensive feature points. Then, we employ an L1-median optimization to refine the identified feature points to perceive geometric edges on the underlying surface accurately. Following that, we generate complete and concise line segments from the refined feature points by designing three geometric operators: clustering, exclusion, and assimilation. The clustering operator generates the initial line segments based on optimized feature points, and the exclusion operator and the assimilation operator ensure the completeness and continuity of these line segments. We evaluate our approach on various scene point clouds, such as TLS, MLS, and ALS data. Extensive experimental results show that our method can outperform the competing approaches in terms of both accuracy and efficiency.

A Deep-Learning-Based Method for Extracting an Arbitrary Number of Individual Power Lines from UAV-Mounted Laser Scanning Point Clouds

In recent years, laser scanners integrated with Unmanned Aerial Vehicles (UAVs) have exhibited great potential in conducting power line inspections in harsh environments. The point clouds collected for power line inspections have numerous advantages over remote image data. However, point cloud-based individual power line extraction, which is a crucial technology required for power line inspections, still poses several challenges such as massive 3D points, imbalanced category points, etc. Moreover, in various power line scenarios, previous studies often require manual setup and careful adjustment of different thresholds to separate different power lines, which is inefficient for practical applications. To handle these challenges, in this paper, we propose a multi-branch network to automatically extract an arbitrary number of individual power lines from point clouds collected by UAV-based laser scanners. Specifically, to handle the massive 3D point clouds in complex outdoor scenarios, we propose to leverage deep neural network for efficient and rapid feature extraction in large-scale point clouds. To mitigate imbalanced data quantities across different categories, we propose to design a weighted cross-entropy loss function to measure the varying importance of each category. To achieve the effective extraction of an arbitrary number of power lines, we propose leveraging a loss function to learn the discriminative features that can differentiate the points belonging to different power lines. Once the discriminative features are learned, the Mean Shift method can distinguish the individual power lines by clustering without supervision. The evaluations are executed on two datasets, which are acquired at different locations with UAV-mounted laser scanners. The proposed method has been thoroughly tested and evaluated, and the results and discussions confirm its outstanding ability to extract an arbitrary number of individual power lines in point clouds.

A Large-Scale Network Construction and Lightweighting Method for Point Cloud Semantic Segmentation.

To significantly enhance the performance of point cloud semantic segmentation, this manuscript presents a novel method for constructing large-scale networks and offers an effective lightweighting technique. First, a latent point feature processing (LPFP) module is utilized to interconnect base networks such as PointNet++ and Point Transformer. This intermediate module serves both as a feature information transfer and a ground truth supervision function. Furthermore, in order to alleviate the increase in computational costs brought by constructing large-scale networks and better adapt to the demand for terminal deployment, a novel point cloud lightweighting method for semantic segmentation network (PCLN) is proposed to compress the network by transferring multidimensional feature information of large-scale networks. Specifically, at different stages of the large-scale network, the structure and attention information of the point features are selectively transferred to guide the compressed network to train in the direction of the large-scale network. This paper also solves the problem of representing global structure information of large-scale point clouds through feature sampling and aggregation. Extensive experiments on public datasets and real-world data demonstrate that the proposed method can significantly improve the performance of different base networks and outperform the state-of-the-art.

Dense Dual-Branch Cross Attention Network for Semantic Segmentation of Large-Scale Point Clouds

Dense Dual-Branch Cross Attention Network for Semantic Segmentation of Large-Scale Point Clouds

Enhanced Local Feature Learning With Simple Offset Attention for Semantic Segmentation of Large-Scale Point Clouds

Enhanced Local Feature Learning With Simple Offset Attention for Semantic Segmentation of Large-Scale Point Clouds

PointNAT: Large-Scale Point Cloud Semantic Segmentation via Neighbor Aggregation With Transformer

PointNAT: Large-Scale Point Cloud Semantic Segmentation via Neighbor Aggregation With Transformer

Semantic segmentation of large-scale point clouds with neighborhood uncertainty

Semantic segmentation of large-scale point clouds with neighborhood uncertainty

Prediction of Transonic Flow over Cascades via Graph Embedding Methods on Large-Scale Point Clouds

In this research, we introduce a deep-learning-based framework designed for the prediction of transonic flow through a linear cascade utilizing large-scale point-cloud data. In our experimental cases, the predictions demonstrate a nearly four-fold speed improvement compared to traditional CFD calculations while maintaining a commendable level of accuracy. Taking advantage of a multilayer graph structure, the framework can extract both global and local information from the cascade flow field simultaneously and present prediction over unstructured data. In line with the results obtained from the test datasets, we conducted an in-depth analysis of the geometric attributes of the cascades reconstructed using our framework, considering adjustments made to the geometric information of the point cloud. We fine-tuned the input using 1603 data points and quantified the contribution of each point. The outcomes reveal that variations in the suction side of the cascade have a significantly more substantial influence on the field results compared to the pressure side and explain the way graph neural networks work for cascade flow-field prediction, enhancing the comprehension of graph-based flow-field prediction among developers and proves the potential of graph neural networks in flow-field prediction on large-scale point clouds and design.

SSRNet: Scalable 3D Surface Reconstruction Network.

Learning-based surface reconstruction methods have received considerable attention in recent years due to their excellent expressiveness. However, existing learning-based methods lack scalability in processing large-scale point clouds. This paper proposes a novel scalable learning-based 3D surface reconstruction method based on octree, called SSRNet. SSRNet works in a scalable reconstruction pipeline, which divides oriented point clouds into different local parts and then processes them in parallel. Accommodating this scalable design pattern, SSRNet constructs local geometric features for octree vertices. Such features comprise the relation between the vertices and the implicit surface, ensuring geometric perception. Focusing on local geometric information also enables the network to avoid the overfitting problem and generalize well on different datasets. Finally, as a learning-based method, SSRNet can process large-scale point clouds in a short time. And to further solve the efficiency problem, we provide a lightweight and efficient version that is about five times faster while maintaining reconstruction performance. Experiments show that our methods achieve state-of-the-art performance with outstanding efficiency.

Towards robustness and generalization of point cloud representation: A geometry coding method and a large-scale object-level dataset

Robustness and generalization are two challenging problems for learning point cloud representation. To tackle these problems, we first design a novel geometry coding model, which can effectively use an invariant eigengraph to group points with similar geometric information, even when such points are far from each other. We also introduce a large-scale point cloud dataset, PCNet184. It consists of 184 categories and 51,915 synthetic objects, which brings new challenges for point cloud classification, and provides a new benchmark to assess point cloud cross-domain generalization. Finally, we perform extensive experiments on point cloud classification, using ModelNet40, ScanObjectNN, and our PCNet184, and segmentation, using ShapeNetPart and S3DIS. Our method achieves comparable performance to state-of-the-art methods on these datasets, for both supervised and unsupervised learning. Code and our dataset are available at https://github.com/MingyeXu/PCNet184.

BEMF-Net: Semantic Segmentation of Large-Scale Point Clouds via Bilateral Neighbor Enhancement and Multi-Scale Fusion

The semantic segmentation of point clouds is a crucial undertaking in 3D reconstruction and holds great importance. However, achieving precise semantic segmentation represents a significant hurdle. In this paper, we present BEMF-Net, an efficient method for large-scale environments. It starts with an effective feature extraction method. Unlike images, 3D data comprise not only geometric relations but also texture information. To accurately depict the scene, it is crucial to take into account the impacts of texture and geometry on the task, and incorporate modifications to improve feature description. Additionally, we present a multi-scale feature fusion technique that effectively promotes the interaction between features at different resolutions. The approach mitigates the problem of the smoothing of detailed information caused by downsampling mechanisms, while ensuring the integrity of features across different layers, allowing a more comprehensive representation of the point cloud. We confirmed the effectiveness of this method by testing it on benchmark datasets such as S3DIS, SensatUrban, and Toronto3D.

QuickFPS: Architecture and Algorithm Co-Design for Farthest Point Sampling in Large-Scale Point Clouds

Point clouds have been employed extensively in machine perception applications. Farthest point sampling (FPS) is a critical kernel for point cloud processing. With the rapid growth of point cloud scale, FPS introduces a large number of memory accesses, which become the bottleneck of the large-scale point cloud processing. In this paper, we present QuickFPS, an architecture and algorithm co-design of farthest point sampling in large-scale point clouds. First, We systemically analyze the characteristics of FPS and put forward a bucket-based farthest point sampling algorithm. The algorithm introduces a two-level tree data structure to organize the large-scale point cloud into multiple buckets. By using two mechanisms named merged computation and implicit computation for the buckets, the external memory accesses and compute cost are significantly reduced. Then, we design an efficient domain-specific accelerator for farthest point sampling in large-scale point clouds. The accelerator takes advantage of different forms of parallelism and further improves the accelerator’s efficiency. Finally, we evaluate QuickFPS with several widely used point cloud datasets, which include small-scale and large-scale point clouds (up to 120,000 points). Overall, QuickFPS achieves performance speedups of 43.4x and 12.2x compared to GTX 1080Ti GPU and state-of-the-art point cloud accelerator PointAcc, respectively.