Large-scale Point Clouds Research Articles

Deep learning for large-scale point cloud segmentation in tunnels considering causal inference

This work designs an integrated deep learning approach to accomplish a multi-label segmentation of various objects including seepage from 3D point clouds of tunnels. 3D laser scanning is first utilized to collect raw point clouds from the operating tunnels. An integration approach based on projection strategy and point-based method is developed. A causal inference-based data augmentation approach is proposed to enhance the segmentation performance between the seepage and the segment class. A Dempster–Shafer (D-S) evidence-based feature fusion model is established to combine the local feature with global features from different CNN models. A real cross-river tunnel section in China is applied as a case study for illustration. Results verify that (1) the integration framework shows an outperforming performance of seepage segmentation, and the Intersection over Union (IOU), Precision, Recall, and F1 score for seepage segmentation achieve 0.70, 0.896, 0.762, and 0.824, respectively; (2) the segmentation model achieves competitive results in multiple classes, and the segmentation F1 score of the cable, segment, pipe, powertrack and the track classes obtain 0.913, 0.911, 0.993, 0.898, and 0.963, respectively; (3) the segmentation model is superior to other SOTA point cloud segmentation algorithms in terms of evaluation metrics.

Improving the robustness of analog deep neural networks through a Bayes-optimized noise injection approach

Analog deep neural networks (DNNs) provide a promising solution, especially for deployment on resource-limited platforms, for example in mobile settings. However, the practicability of analog DNNs has been limited by their instability due to multi-factor reasons from manufacturing, thermal noise, etc. Here, we present a theoretically guaranteed noise injection approach to improve the robustness of analog DNNs without any hardware modifications or sacrifice of accuracy, which proves that within a certain range of parameter perturbations, the prediction results would not change. Experimental results demonstrate that our algorithmic framework can outperform state-of-the-art methods on tasks including image classification, object detection, and large-scale point cloud object detection in autonomous driving by a factor of 10 to 100. Together, our results may serve as a way to ensure the robustness of analog deep neural network systems, especially for safety-critical applications.

NeiEA-NET: Semantic segmentation of large-scale point cloud scene via neighbor enhancement and aggregation

NeiEA-NET: Semantic segmentation of large-scale point cloud scene via neighbor enhancement and aggregation

Efficient Point Cloud Object Classifications with GhostMLP

Efficient models capable of handling large numbers of data points in point cloud research are in high demand in computer vision. Despite recent advancements in 3D classification and segmentation tasks in point cloud processing, the deep learning PointNeXt and PointMLP models are plagued with heavy computation requirements with limited efficiencies. In this paper, a novel GhostMLP model for point clouds is thus introduced. It takes the advantages of the GhostNet design modules and uses them to replace the MLP layers in the existing PointMLP model. The resulting GhostMLP architecture achieves superior classification performance with lower computation requirements. Compared to the PointMLP, GhostMLP maintains sustainable performance with fewer parameters and lower FLOPs computations. Indeed, it outperforms PointMLP on the ScanObjectNN dataset, achieving 88.7% overall accuracy and 87.6% mean accuracy with only 6 million parameters and 7.2 GFLOPs—about half the resources required by PointMLP. At the same time, GhostMLP-S is introduced as a lightweight version which also outperforms PointMLP in performance. GhostMLP completes faster training and inference with GPU and is the best-performing method that does not require any extra training data in the ScanObjectNN benchmark. Efficient point cloud analysis is essential in computer vision, and we believe that GhostMLP has the potential to become a powerful tool for large-scale point cloud analysis.

MFTR-Net: A Multi-Level Features Network with Targeted Regularization for Large-Scale Point Cloud Classification.

There are some irregular and disordered noise points in large-scale point clouds, and the accuracy of existing large-scale point cloud classification methods still needs further improvement. This paper proposes a network named MFTR-Net, which considers the local point cloud's eigenvalue calculation. The eigenvalues of 3D point cloud data and the 2D eigenvalues of projected point clouds on different planes are calculated to express the local feature relationship between adjacent point clouds. A regular point cloud feature image is constructed and inputs into the designed convolutional neural network. The network adds TargetDrop to be more robust. The experimental result shows that our methods can learn more high-dimensional feature information, further improving point cloud classification, and our approach can achieve 98.0% accuracy with the Oakland 3D dataset.

Toward Efficient and Complete Line Segment Extraction for Large-Scale Point Clouds via Plane Segmentation and Projection

An efficient method based on plane segmentation and projection is proposed to extract as complete boundary (straight) line segments as possible. To efficiently detect boundary points, 3-D planar point clouds (patches) generated by plane segmentation are converted into 2-D images by graphical projection to avoid point-based nearest neighbors’ searching. To preserve boundary details, the projection resolution for each plane is determined through accurate statistics of average local intervals between points in voxels composing corresponding planar point clouds. To alleviate erroneous line segmentation extraction caused by uneven densities of planar point clouds, improved mathematical morphological operation, whose structural element order is self-adaptive to local point density, is proposed for crack inpainting of projected images. As for extraction of 3-D boundary line segments, the back-projection results of 2-D boundary line segments detected from images are utilized as references for clustering of boundary points, and a 3-D straight line fitting algorithm based on recursive weighted least squares is employed to reduce the effect of boundary noise. A series of refinement strategies are further presented to control the quality of extracted 3-D line segments. Experimental results demonstrate that the proposed line segment extraction method can provide efficient and fine processing results, as well as automatic processing ability, which satisfies the efficient processing demand for large-scale point clouds.

FA-ResNet: Feature affine residual network for large-scale point cloud segmentation

Semantic segmentation for large-scale point clouds in 3D computer vision remains challenging. Most existing studies focus on creating complex local geometry extractors without considering the sparsity of point clouds and the multi-scale problem of objects in large-scale, resulting in networks that fail to efficiently extract local features and affect segmentation accuracy. In this study, we propose a novel Feature Affine Residual (FA-Res) learnable module for this problem to learn robust point cloud semantic information from the point cloud domain. First, we create a Local Relation-shape Learning module to learn local shape relationships, thereby supplementing the structural information. Second, we propose a lightweight Feature Affine module that can conduct adaptive modifications on local point features to reduce differences between local point clouds with varying densities and the K-nearest neighbor (KNN) algorithm’s domain determination. Finally, we design a Residual MLP Pooling module that can learn deep aggregation features to explore more sophisticated semantic data and provide better guidance for semantic segmentation. We compare our network with state-of-the-art networks on two separate datasets to show its effectiveness. Specifically, our method achieves 68.1% mIoU on S3DIS tested on Area 5, which is an improvement of 2.7% compared with the latest representative network.

Real-Time Semantic Segmentation of Point Clouds Based on an Attention Mechanism and a Sparse Tensor

A 3D point cloud is one of the main data sources for robot environmental cognition and understanding. Due to the limited computation and memory capacities of the robotic platform, existing semantic segmentation models of 3D point clouds cannot meet the requirements of real-time applications. To solve this problem, a lightweight, fully convolutional network based on an attention mechanism and a sparse tensor is proposed to better balance the accuracy and real-time performance of point cloud semantic segmentation. On the basis of the 3D-Unet structure, a global feature-learning module and a multi-scale feature fusion module are designed. The former improves the ability of features to describe important areas by learning the importance of spatial neighborhoods. The latter realizes the fusion of multi-scale semantic information and suppresses useless information through the task correlation learning of multi-scale features. Additionally, to efficiently process the large-scale point clouds acquired in real time, a sparse tensor-based implementation method is introduced. It is able to reduce unnecessary computation according to the sparsity of the 3D point cloud. As demonstrated by the results of experiments conducted with the SemanticKITTI and NuScenes datasets, our model improves the mIoU metric by 6.4% and 5%, respectively, over existing models that can be applied in real time. Our model is a lightweight model that can meet the requirements of real-time applications.

Large-scale 3D object segmentation employing conditional angular clustering technique

This paper aims to develop an automatic 3D object segmentation method for the large-scale point clouds. Given a range image, the preprocessing is first applied to get the optimal 3D point cloud. A [Formula: see text]-nearest neighbor is built, and a segmentation algorithm based on the conditional angular clustering technique is used to segment the objects from the point cloud. The algorithm is tested on the real point cloud datasets. The experiment results demonstrated that the developed segmentation method can be used to localize the object with the relative uncertainty of 0.27%.

Class-aware tiny object recognition over large-scale 3D point clouds

Although tremendous strides have been made in object recognition over large-scale 3D point clouds, one of the remaining open challenges is detecting tiny objects. The signal and appearance information of tiny objects are generally sparse and insufficient due to their tiny size, which makes the tiny object difficult to be detected with existing studies both in tiny 2D object recognition and general 3D object recognition since 3D topology relations are lost due to feature projection in tiny 2D object recognition and massive background may dominate the feature learner in general 3D object recognition. To the best of our knowledge, we explore tiny object recognition for the first time over large-scale 3D point clouds by designing deep neural network. To cope with this problem, a novel two-stage approach is proposed in this paper, which can effectively learn an informative and discriminative feature representation and efficiently process large-scale point clouds. Specifically, a class-aware filtering strategy is designed to dispense with the redundant background information. After that, we introduce a novel aggregating and sampling scheme in a supervised manner to progressively increase the receptive field for each 3D point, which can adaptively capture useful signals from tiny objects and learn complex geometric structures. Last, we propose a biased loss function that is inversely correlated to the number of tiny object points, thereby effectively handling the sparsity of tiny objects. Extensive experimental results over two real-world data sets validate the effectiveness and efficiency of the proposed approach.

A large-scale point cloud semantic segmentation network via local dual features and global correlations

For large-scale point cloud semantic segmentation, the relationships between long-range neighbourhoods are as important as short-range features. The current methods focus on aggregating the fine-grained geometric structures of local regions, the long-range features are usually ignored. To this end, we propose a novel framework named LG-Net that can efficiently learn local dual features and global correlations from point clouds. We adopt random sampling to decrease point density, thereby processing large-scale point cloud efficiently. Specifically, we propose a dual feature complementary (DFC) module that makes geometric and semantic features complementary to preserve local information. We utilize positional relations to capture geometric details, and aggregate semantic features according to the similarity of neighbours. The global correlation mining (GCM) module simply shares an attention map for all 3D points to learn long-range contextual information. We conducted extensive experiments on public datasets, including indoor scenes (S3DIS, ScanNet) and outdoor scenes (SemanticKITTI, SensatUrban). Experimental results demonstrate that our method outperforms the state-of-the-art approaches.

Design of constrained dynamic path planning algorithms in large-scale 3D point cloud maps for UAVs

The technology of unmanned aerial vehicles (UAVs) has been growing rapidly in recent years. To achieve highly autonomous flight capability, collision-free path generation becomes a key research topic. However, for a large-scale terrain map, an efficient path searching algorithm is always a challenging problem. Based on the Goal-bias Rapidly-exploring Random Tree Star (GB-RRT*) together with the aid of a bidirectional search technique, in this paper, a new constrained path planning algorithm, called Parent Revisited Goal and Parent Bias RRT* (PRGPB-RRT*), is proposed. Three major objectives are considered as the development benchmark: 1). shorter total distance, 2). smoother flight path, and 3). high computational efficiency. To meet different task requirements, a dynamic path planning method is investigated to diminish the high costs due to large-scale environments and keep track of the changing destination by finding local target points. In addition, unknown object avoidance is an essential part of flight path planning. As a consequence, the Fitting-Octree is presented to partition the point clouds into multiple cubes to represent the obstacles precisely. Moreover, the method of grouping point clouds into clusters effectively increases the efficiency of the implementation under a large number of obstacles. Finally, the feasibility of the proposed method is verified by a real-world 3D large-scale point cloud map. Experiments show that the proposed method successfully enhances the path generation efficiency as well as the trajectory smoothness. Moreover, the presented PRGPB-RRT* can be applied to different intelligent UAVs subject to flight kinematics constrain.

Cascaded Contextual Reasoning for Large-Scale Point Cloud Semantic Segmentation

Large-scale point cloud semantic segmentation aims to efficiently process millions of points and classify each point into the correct class. Most traditional methods are designed for indoor scenes and are inpractical to process large-scale point clouds directly. To improve efficiency, appropriate point reduction strategies such as random sampling is necessary to reduce points significantly. However, it brings some problems. First, such reduction leads to serious information loss during encoding phase. Second, to recover from the reduction of points, low-level features need to be carefully fused after upsampling for refinement. To address above issues, we propose a Cascaded Contextual Reasoning network (PointCCR) for large-scale point cloud semantic segmentation. We propose a Dilated Graph Convolution (DGC) module to exploit local structure of point clouds and efficiently increase receptive fields. To make compensation for the information loss caused by reducing points, we propose a Cross-level Feature Enrichment (CFE) module which models cross-level contextual dependencies. DGC and CFE together constitute the Multi-level Cascade Aggregation (MCA) module as encoder. Furthermore, we build a Class-aware Attentive Refinement (CAR) module to selectively fuse low-level features for refinement after upsampling. Extensive experimental results demonstrate our method achieves superior performance compared with other state-of-the-arts on three large-scale point cloud datasets (i.e., Semantic3D, SemanticKITTI and S3DIS).

Robust Extraction of Multiple-Type Support Positioning Devices in the Catenary System of Railway Dataset Based on MLS Point Clouds

Due to the external environment, the support positioning device (SPD) of the railway catenary is prone to break down. Therefore, the automatic detection of SPDs is of great research significance to ensure the safe operation of the railway system. However, the small-scale SPD expresses weak features due to the complexity of railway scenes. Therefore, we propose a new algorithm for automatically extracting multiple-type SPDs quickly. Firstly, a preprocessing approach of equidistant scene segmentation is proposed based on trajectory points to divide the whole scene into several independent region units for batch processing. Then, a new point spacing-based spatial clustering of applications with noise (PSBSCAN) algorithm is proposed for the extraction of pillars in the pillar corridor of each region unit. Adopting point spacing as the core standard and the parallel iteration mode with only accessing edge points can reduce the influence of the uneven point cloud density and the large-scale point clouds. Next, to locate SPDs, the spatial relationship between the trajectory point, the pillar, and the SPD in the scene is introduced into the corresponding objects to build the spatial index with semantic information. Finally, contact wires installed on the SPD are filtered through multiple-level voxel segmentation based on octree to extract SPD accurately. Through the test of 10km railway datasets, the average <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> by the proposed method is 98.61%, and the average recognition rate of SPD is 99.65%, which shows that this method can achieve efficient extraction of multiple-type SPDs in railway scenes with different point densities.

Multistage Scene-Level Constraints for Large-Scale Point Cloud Weakly Supervised Semantic Segmentation

Multistage Scene-Level Constraints for Large-Scale Point Cloud Weakly Supervised Semantic Segmentation

FAR-Net: Semantic Segmentation of Large-Scale Point Clouds Based on Feature Aggregation and Recoding for Aerial Computing

FAR-Net: Semantic Segmentation of Large-Scale Point Clouds Based on Feature Aggregation and Recoding for Aerial Computing

FG-Net: A Fast and Accurate Framework for Large-Scale LiDAR Point Cloud Understanding.

This work presents FG-Net, a general deep learning framework for large-scale point cloud understanding without voxelizations, which achieves accurate and real-time performance with a single NVIDIA GTX 1080 8G GPU and an i7 CPU. First, a novel noise and outlier filtering method is designed to facilitate the subsequent high-level understanding tasks. For effective understanding purpose, we propose a novel plug-and-play module consisting of correlated feature mining and deformable convolution-based geometric-aware modeling, in which the local feature relationships and point cloud geometric structures can be fully extracted and exploited. For the efficiency issue, we put forward a new composite inverse density sampling (IDS)-based and learning-based operation and a feature pyramid-based residual learning strategy to save the computational cost and memory consumption, respectively. Compared with current methods which are only validated on limited datasets, we have done extensive experiments on eight real-world challenging benchmarks, which demonstrates that our approaches outperform state-of-the-art (SOTA) approaches in terms of accuracy, speed, and memory efficiency. Moreover, weakly supervised transfer learning is also conducted to demonstrate the generalization capacity of our method.

MCTNet: Multiscale Cross-Attention-Based Transformer Network for Semantic Segmentation of Large-Scale Point Cloud

MCTNet: Multiscale Cross-Attention-Based Transformer Network for Semantic Segmentation of Large-Scale Point Cloud

Feature Graph Convolution Network With Attentive Fusion for Large-Scale Point Clouds Semantic Segmentation

Feature Graph Convolution Network With Attentive Fusion for Large-Scale Point Clouds Semantic Segmentation

CFSA-Net: Efficient Large-Scale Point Cloud Semantic Segmentation Based on Cross-Fusion Self-Attention

CFSA-Net: Efficient Large-Scale Point Cloud Semantic Segmentation Based on Cross-Fusion Self-Attention