Point Cloud Semantic Segmentation Research Articles

Filtering-Assisted Airborne Point Cloud Semantic Segmentation for Transmission Lines.

Point cloud semantic segmentation is crucial for identifying and analyzing transmission lines. Due to the number of point clouds being huge, complex scenes, and unbalanced sample proportion, the mainstream machine learning methods of point cloud segmentation cannot provide high efficiency and accuracy when extending to transmission line scenes. This paper proposes a filter-assisted airborne point cloud semantic segmentation for transmission lines. First, a large number of ground point clouds is identified by introducing the well-developed cloth simulation filter to alleviate the impact of the imbalance of the target object proportion on the classifier's performance. The multi-dimensional features are then defined, and the classification model is trained to achieve the multi-element semantic segmentation of the transmission line scene. The experimental results and analysis indicate that the proposed filter-assisted algorithm can significantly improve the semantic segmentation performance of the transmission line point cloud, enhancing both the point cloud segmentation efficiency and accuracy by more than 25.46% and 3.15%, respectively. The filter-assisted point cloud semantic segmentation method reduces the volume of sample data, the number of sample classes, and the sample imbalance index in power line scenarios to a certain extent, thereby improving the classification accuracy of classifiers and reducing time consumption. This research holds significant theoretical reference value and engineering application potential for scene reconstruction and intelligent understanding of airborne laser point cloud transmission lines.

Multi-Modal LiDAR Point Cloud Semantic Segmentation with Salience Refinement and Boundary Perception

Point cloud segmentation is essential for scene understanding, which provides advanced information for many applications, such as autonomous driving, robots, and virtual reality. To improve the accuracy and robustness of point cloud segmentation, many researchers have attempted to fuze camera images to complement the color and texture information. The common fusion strategy is the combination of convolutional operations with concatenation, element-wise addition or element-wise multiplication. However, conventional convolutional operators tend to confine the fusion of modal features within their receptive fields, which can be incomplete and limited. In addition, the inability of encoder–decoder segmentation networks to explicitly perceive segmentation boundary information results in semantic ambiguity and classification errors at object edges. These errors are further amplified in point cloud segmentation tasks, significantly affecting the accuracy of point cloud segmentation. To address the above issues, we propose a novel self-attention multi-modal fusion semantic segmentation network for point cloud semantic segmentation. Firstly, to effectively fuze different modal features, we propose a self-cross fusion module (SCF), which models long-range modality dependencies and transfers complementary image information to the point cloud to fully leverage the modality-specific advantages. Secondly, we design the salience refinement module (SR), which calculates the importance of channels in the feature maps and global descriptors to enhance the representation capability of salient modal features. Finally, we propose the local-aware anisotropy loss measure the element-level importance in the data and explicitly provide boundary information for the model, which alleviates the inherent semantic ambiguity problem in segmentation networks. Extensive experiments on two benchmark datasets demonstrate that our proposed method surpasses current state-of-the-art methods.

Adversarial Unsupervised Domain Adaptation for 3D Semantic Segmentation with 2D Image Fusion of Dense Depth

AbstractUnsupervised domain adaptation (UDA) is increasingly used for 3D point cloud semantic segmentation tasks due to its ability to address the issue of missing labels for new domains. However, most existing unsupervised domain adaptation methods focus only on uni‐modal data and are rarely applied to multi‐modal data. Therefore, we propose a cross‐modal UDA on multi‐modal datasets that contain 3D point clouds and 2D images for 3D Semantic Segmentation. Specifically, we first propose a Dual discriminator‐based Domain Adaptation (Dd‐bDA) module to enhance the adaptability of different domains. Second, given that the robustness of depth information to domain shifts can provide more details for semantic segmentation, we further employ a Dense depth Feature Fusion (DdFF) module to extract image features with rich depth cues. We evaluate our model in four unsupervised domain adaptation scenarios, i.e., dataset‐to‐dataset (A2D2 → SemanticKITTI), Day‐to‐Night, country‐to‐country (USA → Singapore), and synthetic‐to‐real (VirtualKITTI → SemanticKITTI). In all settings, the experimental results achieve significant improvements and surpass state‐of‐the‐art models.

EIDU-Net: edge-preserved inception DenseGCN U-Net for LiDAR point cloud segmentation

With the development of laser scanners and machine learning, point cloud semantic segmentation plays a significant role in autonomous driving, scene reconstruction, human-computer interaction, and other fields. In recent years, point cloud semantic segmentation based on deep learning has become one of the key research directions in point cloud processing. Due to the limited ability to exploit geometric details and contextual information in point clouds, most methods that adopt encoder-decoder architecture lose local structural information easily, especially detailed features, and extract features insufficiently. To address this issue, the edge-preserving inception DenseGCN U-Net (entitled as EIDU-Net) is proposed. EIDU-Net makes full use of the complementation between geometric details in the original point cloud and high-level features. The edge-preserved graph pooling (EGP) layer, the key module of the EIDU-Net, is designed to retain additional edge feature information from the original point cloud during pooling operations. Accordingly, the edge-preserved graph unpooling (EGU) layer can restore the feature graph more efficiently based on the additionally retained edge features. Extensive experiments demonstrate that our proposed EIDU-Net has remarkable improvements on semantic segmentation tasks under whatever S3DIS or Terracotta Warrior fragments. Our code is publicly available at https://github.com/caoxin918/EIDU-Net.

Weakly supervised 3D point cloud semantic segmentation for architectural heritage using teacher-guided consistency and contrast learning

Point cloud semantic segmentation is significant for managing and protecting architectural heritage. Currently, fully supervised methods require a large amount of annotated data, while weakly supervised methods are difficult to transfer directly to architectural heritage. This paper proposes an end-to-end teacher-guided consistency and contrastive learning weakly supervised (TCCWS) framework for architectural heritage point cloud semantic segmentation, which can fully utilize limited labeled points to train network. Specifically, a teacher-student framework is designed to generate pseudo labels and a pseudo label dividing module is proposed to distinguish reliable and ambiguous point sets. Based on it, a consistency and contrastive learning strategy is designed to fully utilize supervision signals to learn the features of point clouds. The framework is tested on the ArCH dataset and self-collected point cloud, which demonstrates that the proposed method can achieve effective semantic segmentation of architectural heritage using only 0.1 % of annotated points.

Key technology for digital twins in the architecture, engineering, and construction industry: new advances in point cloud semantic segmentation algorithms for buildings

Key technology for digital twins in the architecture, engineering, and construction industry: new advances in point cloud semantic segmentation algorithms for buildings

Class agnostic and specific consistency learning for weakly-supervised point cloud semantic segmentation

This paper focuses on Weakly Supervised 3D Point Cloud Semantic Segmentation (WS3DSS), which involves annotating only a few points while leaving a large number of points unlabeled in the training sample. Existing methods roughly force point-to-point predictions across different augmented versions of inputs close to each other. While this paper introduces a carefully-designed approach for learning class agnostic and specific consistency, based on the teacher–student framework. The proposed class-agnostic consistency learning, to bring the features of student and teacher models closer together, enhances the model robustness by replacing the traditional point-to-point prediction consistency with the group-to-group consistency based on the perturbed local neighboring points’ features. Furthermore, to facilitate learning under class-wise supervisions, we propose a class-specific consistency learning method, pulling the feature of the unlabeled point towards its corresponding class-specific memory bank feature. Such a class of the unlabeled point is determined as the one with the highest probability predicted by the classifier. Extensive experimental results demonstrate that our proposed method surpasses the SOTA method SQN (Huet al., 2022) by 2.5% and 8.3% on S3DIS dataset, and 4.4% and 13.9% on ScanNetV2 dataset, on the 0.1% and 0.01% settings, respectively. Code is available at https://github.com/jasonwjw/CASC.

Deep learning network for indoor point cloud semantic segmentation with transferability

Semantic segmentation is crucial for interpreting point cloud data and plays a fundamental role in automating the creation of as-built BIM. Existing neural network models for semantic segmentation often heavily rely on the training dataset, resulting in a significant performance drop when applied to new datasets. This paper presents AttTransNet, a neural network model for automated point cloud semantic segmentation. Its attention-based pooling module improves local feature extraction from point clouds while reducing computational costs. The transfer learning framework enhances segmentation accuracy with minimal training on target datasets. Comparative experiments show that AttTransNet reduces training time by 80 % and improves segmentation accuracy by over 20 % compared with other SOTA methods. Cross-dataset experiments reveal that the transfer learning framework increases accuracy on new datasets by 150 %. By adding semantic information to point clouds, AttTransNet aids BIM modelers with direct reference, encouraging broader application of automated point cloud segmentation in the industry.

3D Point Cloud Semantic Segmentation based on Multi-scale Dense Nested Networks

Aiming at the problem that the relationship between geometric features and semantic features is ignored in the point cloud data downsampling process, which leads to inaccurate segmentation of object boundaries and structural details, this paper proposes a 3D point cloud semantic segmentation network based on multi-scale dense nested type. Firstly, a dense nested network architecture is constructed by nesting multiple multi-scale feature fusion modules to fuse multi-scale features of different directions between encoder-decoder paths, so as to effectively propagate local ge-ometric context information and enhance the ability of cross-scale information interaction. Secondly, a local feature aggregation unit is constructed in the multi-scale feature fusion module, which strengthens the structural awareness within the local point set based on graph convolution and attention mechanism, and promotes the complementarity of local geometric features and abstract semantic information. Then, the cross-layer multi-loss supervision module is combined to further optimize the multi-scale feature propagation, which makes the network training more stable and improves the point cloud segmentation accuracy. Finally, this paper verified the proposed network on the S3DIS dataset. The experimental results show that the proposed network has the mean intersection over Union of 71.2% and the overall accuracy of 88.7%, which is 1.2 and 0.7 percentage points higher than RandLA-Net, respectively, which proves that the proposed network can effectively improve the accuracy of 3D point cloud semantic segmentation.

Weakly supervised point cloud semantic segmentation based on scene consistency

Weakly supervised point cloud semantic segmentation based on scene consistency

Multi-Scale Classification and Contrastive Regularization: Weakly Supervised Large-Scale 3D Point Cloud Semantic Segmentation

With the proliferation of large-scale 3D point cloud datasets, the high cost of per-point annotation has spurred the development of weakly supervised semantic segmentation methods. Current popular research mainly focuses on single-scale classification, which fails to address the significant feature scale differences between background and objects in large scenes. Therefore, we propose MCCR (Multi-scale Classification and Contrastive Regularization), an end-to-end semantic segmentation framework for large-scale 3D scenes under weak supervision. MCCR first aggregates features and applies random downsampling to the input data. Then, it captures the local features of a random point based on multi-layer features and the input coordinates. These features are then fed into the network to obtain the initial and final prediction results, and MCCR iteratively trains the model using strategies such as contrastive learning. Notably, MCCR combines multi-scale classification with contrastive regularization to fully exploit multi-scale features and weakly labeled information. We investigate both point-level and local contrastive regularization to leverage point cloud augmentor and local semantic information and introduce a Decoupling Layer to guide the loss optimization in different spaces. Results on three popular large-scale datasets, S3DIS, SemanticKITTI and SensatUrban, demonstrate that our model achieves state-of-the-art (SOTA) performance on large-scale outdoor datasets with only 0.1% labeled points for supervision, while maintaining strong performance on indoor datasets.

Enhancing explainability of deep learning models for point cloud analysis: a focus on semantic segmentation

ABSTRACT Semantic segmentation of point clouds plays a critical role in various applications, such as urban planning, infrastructure management, environmental analyses and autonomous navigation. Understanding the behaviour of deep neural networks (DNNs) in analysing point cloud data is essential for improving segmentation accuracy and developing effective network architectures and acquisition strategies. In this paper, we investigate the traits of some state-of-the-art neural networks using indoor and urban outdoor point cloud datasets. We compare PointNet, DGCNN, and BAAF-Net on specifically selected datasets, including synthetic and real-world environments. The chosen datasets are S3DIS, SynthCity, Semantic3D, and KITTI. We analyse the impact of different factors such as dataset type (synthetic vs. real), scene type (indoor vs. outdoor), and acquisition system (static vs. mobile sensors). Through detailed analyses and comparisons, we provide insights into the strengths and limitations not only of different network architectures in handling urban point clouds but also of their data structure. This study contributes to going beyond the mere and unconditional use of AI algorithms, trying to explain DNNs behaviour in point cloud analysis and paving the way for future research to enhance segmentation accuracy and develop possible guidelines both for network design and data acquisition in the geomatics field.

Semantic Segmentation of Point Cloud Scene via Multi-Scale Feature Aggregation and Adaptive Fusion

Point cloud semantic segmentation is a key step in 3D scene understanding and analysis. In recent years, deep learning–based point cloud semantic segmentation methods have received extensive attention from researchers. Multi-scale neighborhood feature learning methods are suitable for inhomogeneous density point clouds, but different scale branching feature learning increases the computational complexity and makes it difficult to accurately fuse different scale features to express local information. In this study, a point cloud semantic segmentation network based on RandLA-Net with multi-scale local feature aggregation and adaptive fusion is proposed. The designed structure can reduce computational complexity and accurately express local features. The mean intersection-over-union is improved by 1.1% on the SemanticKITTI data set with an inference speed of nine frames per second, while the mean intersection-over-union is improved by 0.9% on the S3DIS data set, compared with RandLA-Net. We also conduct ablation studies to validate the effectiveness of the proposed key structure.

LESA-Net: Semantic segmentation of multi-type road point clouds in complex agroforestry environment

Point-cloud semantic segmentation is a visual task essential for agricultural robots to comprehend natural agroforestry environments. However, owing to the extremely large amount of point-cloud data in agroforestry environments, learning effective features for semantic segmentation from large-scale point clouds is challenging. Therefore, to address this issue and achieve accurate semantic segmentation of different types of road-surface point clouds in large-scale agroforestry environments, this study proposes a point-cloud semantic segmentation network framework based on double-distance self-attention. First, a point-cloud local feature enhancement module is proposed. This module primarily extends the receptive field and enhances the generalizability of multidimensional features by incorporating reflection intensity information and a spatial feature-encoding block that is enhanced with contextual semantic information. Second, we introduce a dual-distance attention pooling (DDAPS) block based on the self-attention mechanism. This block initially learns the feature representation of the local neighborhood of each point through the self-attention mechanism. Then, it uses the DDAPS block to aggregate more discriminative local neighborhood point features. Finally, extensive experimental results on large-scale point-cloud datasets, SemanticKITTI and RELLIS-3D, demonstrate that our algorithm outperforms similar algorithms in large-scale agroforestry environments.

Accurate 3-D Semantic Segmentation of Point Clouds for Intelligent Vehicles Based on Multiview Edge Guidance and Fusion

Accurate 3-D Semantic Segmentation of Point Clouds for Intelligent Vehicles Based on Multiview Edge Guidance and Fusion

PA-Net: Trustworthy weakly supervised point cloud semantic segmentation with primary–auxiliary structure

Weakly supervised point cloud segmentation is a hot research topic with significant implications for practical applications such as autonomous driving. However, existing weakly supervised approaches primarily focus on improving prediction results while neglecting the reliability of the inference outcomes. To address this issue, this paper proposes a primary–auxiliary network PA-Net for point cloud segmentation and uncertainty estimation. PA-Net achieves excellent segmentation results and enables effective uncertainty assessment. Specifically, we first train PA-Net on sparsely annotated point cloud data, which allows our network to capture multi-scale information of labeled points and provide additional supervisory signals. Subsequently, we utilize the trained model for inference. We mainly conducted experiments on the indoor point cloud dataset S3DIS in a 1% labeled setting. PA-Net achieved a mean Intersection over Union (mIoU) of 64.5%, which surpasses most weakly-supervised methods. In particular, it exceeded PSD by 3.5% mIoU and SQN by 0.8% mIoU. Similarly, on the outdoor dataset Toronto3D, PA-Net also outperformed most weakly supervised methods. With only 0.01% labeling, it surpassed PSD by 11.99% mIoU and SQN by 20.34% mIoU. Lastly, to validate the effectiveness of uncertainty estimation, we visualized scene uncertainty, thereby demonstrating the practicality of our uncertainty estimation approach. Code will be made publicly available at https://github.com/NiuYingchun/PA-Net.

RAAFNet: Reverse Attention Adaptive Fusion Network for Large-Scale Point Cloud Semantic Segmentation

Point cloud semantic segmentation is essential for comprehending and analyzing scenes. However, performing semantic segmentation on large-scale point clouds presents challenges, including demanding high memory requirements, a lack of structured data, and the absence of topological information. This paper presents a novel method based on the Reverse Attention Adaptive Fusion network (RAAFNet) for segmenting large-scale point clouds. RAAFNet consists of a reverse attention encoder–decoder module, an adaptive fusion module, and a local feature aggregation module. The reverse attention encoder–decoder module is applied to extract point cloud features at different scales. The adaptive fusion module enhances fine-grained representation within multi-resolution feature maps. Furthermore, a local aggregation classifier is introduced, which aggregates the features of neighboring points to the center point in order to leverage contextual information and enhance the classifier’s perceptual capability. Finally, the predicted labels are generated. Notably, our method excels at extracting point cloud features across different dimensions and produces highly accurate segmentation results. Experimental results on the Semantic3D dataset achieved an overall accuracy of 89.9% and a mIoU of 74.4%.

Improved building MEP systems semantic segmentation in point clouds using a novel multi-class dataset and local–global vector transformer network

Point cloud semantic segmentation for mechanical, electrical, and plumbing (MEP) systems is crucial for establishing MEP systems digital twins. Deep learning has shown promise in automatically deriving point-wise semantic labels from point clouds. However, the utilization of 3D deep learning methods for directly segmenting complex, occluded, and densely distributed multi-class MEP systems in common building point clouds remains unexplored due to the lack of specialized public datasets. To address this, our study introduces a public, richly annotated, and multi-class point cloud dataset for MEP systems in common buildings, consisting of 92.10 million points across four buildings, 56 areas, and nine component categories. Subsequently, we propose Trans2Net, a novel and customized 3D deep learning method for the automatic and precise segmentation of MEP components, featuring local vector transformer module, global-aid vector transformer module, and learnable feature fusion operator that enhance the model’s sensitivity to local features, capture global context with sparse points, and dynamically fuse weighted local and global features. Experimental results demonstrate that Trans2Net significantly outperforms seven other state-of-the-art methods on the MEP dataset, achieving an overall accuracy of 97.39%, a mean accuracy of 95.45%, and a mean intersection over union (mIoU) of 90.76%. Notably, it surpasses the second-ranked method, PointMeta, by a margin of 9.91% in mIoU. This study provides a novel paradigm for automatically understanding MEP systems in point clouds and offers robust support for digital twin management of MEP systems, contributing to the prolonged lifespans of MEP systems.

Improving Aerial Targeting Precision: A Study on Point Cloud Semantic Segmentation with Advanced Deep Learning Algorithms

The integration of technological advancements has significantly impacted artificial intelligence (AI), enhancing the reliability of AI model outputs. This progress has led to the widespread utilization of AI across various sectors, including automotive, robotics, healthcare, space exploration, and defense. Today, air defense operations predominantly rely on laser designation. This process is entirely dependent on the capability and experience of human operators. Considering that UAV systems can have flight durations exceeding 24 h, this process is highly prone to errors due to the human factor. Therefore, the aim of this study is to automate the laser designation process using advanced deep learning algorithms on 3D point clouds obtained from different sources, thereby eliminating operator-related errors. As different data sources, dense 3D point clouds produced with photogrammetric methods containing color information, and point clouds produced with LiDAR systems were identified. The photogrammetric point cloud data were generated from images captured by the Akinci UAV’s multi-axis gimbal camera system within the scope of this study. For the point cloud data obtained from the LiDAR system, the DublinCity LiDAR dataset was used for testing purposes. The segmentation of point cloud data utilized the PointNet++ and RandLA-Net algorithms. Distinct differences were observed between the evaluated algorithms. The RandLA-Net algorithm, relying solely on geometric features, achieved an approximate accuracy of 94%, while integrating color features significantly improved its performance, raising its accuracy to nearly 97%. Similarly, the PointNet++ algorithm, relying solely on geometric features, achieved an accuracy of approximately 94%. Notably, the model developed as a unique contribution in this study involved enriching the PointNet++ algorithm by incorporating color attributes, leading to significant improvements with an approximate accuracy of 96%. The obtained results demonstrate a notable improvement in the PointNet++ algorithm with the proposed approach. Furthermore, it was demonstrated that the methodology proposed in this study can be effectively applied directly to data generated from different sources in aerial scanning systems.

Point cloud semantic segmentation of grotto scenes using the knowledge-guided deep learning method

ABSTRACT In recent years, the deep learning-based semantic segmentation for point clouds has demonstrated remarkable capabilities in processing 3D urban scenes for applications such as three-dimensional reconstruction, semantic modeling, and augmented reality. However, research on grottoes scenes is very limited. It is currently unclear how existing neural architectures for point cloud semantic segmentation perform in grotto scenes, and how to effectively incorporate the unique characteristics of grotto scenes to enhance the performance of deep neural networks. This study proposed a method for point cloud semantic segmentation of grotto scenes, combining knowledge with deep learning approaches. The method adopted knowledge to guide the creation of benchmark datasets, the design of a neural network called GSS-Net, and the correction of segmentation errors in the results of deep learning. The results show that the proposed method outperforms four existing mainstream models without the correction of segmentation results. Moreover, a set of ablation studies verified the effectiveness of each proposed module. This method not only improves the accuracy of point cloud semantic segmentation in grotto scenes but also enhances the interpretability of network designs. It provides new insights into the application of knowledge-guided deep learning models in grotto scenes.