Segmentation Of Point Cloud Data Research Articles

Corrosion segmentation method of concrete drainage pipes based on point transformer

Deep learning based Closed Circuit Television (CCTV) image detection has achieved remarkable accuracy in identifying and segmenting concrete pipeline damages. However, the lack of in-depth information and requirement for large amounts of training data are two major drawbacks. Therefore, this study proposes a point cloud data segmentation method that incorporates depth information to address the issue of insufficient training data for depth models, enabling efficient and accurate segmentation. Firstly, 3D simulation technology is employed to construct a simulation dataset. Secondly, an optimized point transformation model is established to identify and segment the point cloud data. The proposed method achieved a test accuracy of 94.36 % by optimizing the network structure and the training strategy. The Mean Intersection over Union (MIoU) increased by 3.62 % and 10.3 %, respectively, compared with the classic PointNet++ and before adding simulation data, reaching 86.31 %. These results demonstrate that the proposed method exhibits high precision defect detection capability on three-dimensional point clouds of drainage pipelines.

Research on deep learning-based point cloud semantic segmentation for offshore drilling platforms

Offshore drilling platforms (ODP) require effective deformation monitoring and damage detection in the wellhead space for operational maintenance. LiDAR technology, replacing traditional methods, plays a key role in damage detection through 3D reconstruction, and the accurate semantic segmentation of point clouds is fundamental to this process. Following advancements in deep learning, neural network models have emerged as effective tools for the semantic segmentation of point clouds in a range of settings. ODP point cloud data, characterized by large volumes, high density, class imbalance, and complex structures, present unique challenges for feature extraction. To address these challenges, this paper proposed a multi-scale bounding box (MSB-Box) feature extraction strategy based on spatial relation dynamic encoding (SRD-encoding) and a multi-scale point cloud down-sampling (MPDS) strategy to enhance computational efficiency. Utilizing these proposed strategies, we developed an ODP-Net, a deep learning neural network based on the attention mechanism. The ODP-Net's semantic segmentation performance was evaluated using a self-constructed ODP scene point cloud dataset and the S3DIS dataset. The results demonstrate that ODP-Net, based on MPDS and MSB-Box, outperforms other networks in diverse scenes, confirming its high accuracy and wide applicability in the semantic segmentation of point cloud data in ODP scenes.

Improved deep learning segmentation of outdoor point clouds with different sampling strategies and using intensities

Abstract The rapid growth of outdoor digital twin data sets and advancements in 3D data acquisition technology have sparked interest in improving segmentation performance using deep learning. This research aims to analyze and evaluate different sampling strategies and optimization techniques while exploring the intensity information of outdoor point cloud data. Two sampling strategies, random and stratified sampling, are employed to divide a limited data set. Additionally, the data set is divided into point cloud data with and without intensity. The PointNet++ model is used to segment the point cloud data into two classes, vegetation and structure. The results indicate that stratified sampling outperforms random sampling, yielding a considerable improvement in mean intersection over union scores of up to 10%. Interestingly, the inclusion of intensity information in the data set does not universally enhance performance. Although the use of intensity improves the performance of random sampling, it does not benefit stratified sampling. This research provides insights into the effectiveness of different sampling strategies for outdoor point cloud data segmentation. The findings can contribute to the development of optimized approaches to improving segmentation accuracy in outdoor digital twin applications using deep learning techniques.

Visualization of regional seismic response based on oblique photography and point cloud data

Efficient and realistic visualization of regional seismic response is of great significance for risk education and emergency response. This study proposes an efficient visualization framework for seismic response of regional buildings, which consists of three parts: (1) oblique photography-based three-dimensional (3D) point cloud modelling of regional buildings; (2) time-history response analysis of regional buildings; and (3) hybrid processing for dynamic visualization. Specifically, the unmanned aerial vehicle (UAV) is adopted for oblique photography. The building coordinate extraction with the least square method and point cloud data segmentation with the clustering algorithm are implemented for the pre-processing. The simplified multi-degree-of-freedom (MDOF) model is developed for the time-history displacement analysis of buildings. Then, the MDOF model is used to match the point cloud model of the buildings for visualization. Preset rendering parameters and dynamic geometrical updating are used to further improve the visualization quality. Taking the Southeast University Jiulonghu campus in Nanjing, China for example, a case study is conducted to validate the feasibility of the developed framework.

Deep-Learning-Based Semantic Segmentation Approach for Point Clouds of Extra-High-Voltage Transmission Lines

The accurate semantic segmentation of point cloud data is the basis for their application in the inspection of extra high-voltage transmission lines (EHVTL). As deep learning evolves, point-wise-based deep neural networks have shown great potential for the semantic segmentation of EHVTL point clouds. However, EHVTL point cloud data are characterized by a large data volume and significant class imbalance. Therefore, the down-sampling method and point cloud feature extraction method used in current point-wise-based deep neural networks hardly meet the needs of computational accuracy and efficiency. In this paper, we proposed a two-step down-sampling method and a point cloud feature extraction method based on local feature aggregation of the point clouds after down-sampling in each layer of the model (LFAPAD). We then established a deep neural network named PowerLine-Net for the semantic segmentation of the EHVTL point clouds. Furthermore, in order to test and analyze the performance of PowerLine-Net, we constructed a point cloud dataset for the EHVTL scenes. Using this dataset and the Semantic3D dataset, we implemented network parameter testing, semantic segmentation, and an accuracy comparison of different networks based on PowerLine-Net. The results illustrate that the semantic segmentation model proposed in this paper has a high computational efficiency and accuracy in the semantic segmentation of EHVTL point clouds. Compared with conventional deep neural networks, including PointCNN, KPConv, SPG, PointNet++, and RandLA-Net, PowerLine-Net also achieves a higher accuracy in the semantic segmentation of EHVTL point clouds. Moreover, based on the results predicted by PowerLine-Net, the risk point detection for EHVTL point clouds has been achieved, which demonstrates the important value of this network in practical applications. In addition, as shown by the results of Semantic3D, PowerLine-Net also achieves a high segmentation accuracy, which proves its powerful capability and wide applicability in semantic segmentation for the point clouds of large-scale scenes.

Point Cloud Deep Learning Network Based on Balanced Sampling and Hybrid Pooling

The automatic semantic segmentation of point cloud data is important for applications in the fields of machine vision, virtual reality, and smart cities. The processing capability of the point cloud segmentation method with PointNet++ as the baseline needs to be improved for extremely imbalanced point cloud scenes. To address this problem, in this study, we designed a weighted sampling method based on farthest point sampling (FPS), which adjusts the sampling weight value according to the loss value of the model to equalize the sampling process. We also introduced the relational learning of the neighborhood space of the sampling center point in the feature encoding process, where the feature importance is distinguished by using a self-attention model. Finally, the global-local features were aggregated and transmitted using the hybrid pooling method. The experimental results of the six-fold crossover experiment showed that on the S3DIS semantic segmentation dataset, the proposed network achieved 9.5% and 11.6% improvement in overall point-wise accuracy (OA) and mean of class-wise intersection over union (MIoU), respectively, compared with the baseline. On the Vaihingen dataset, the proposed network achieved 4.2% and 3.9% improvement in OA and MIoU, respectively, compared with the baseline. Compared with the segmentation results of other network models on public datasets, our algorithm achieves a good balance between OA and MIoU.

Implementing PointNet for point cloud segmentation in the heritage context

Automated Heritage Building Information Modelling (HBIM) from the point cloud data has been researched in the last decade as HBIM can be the integrated data model to bring together diverse sources of complex cultural content relating to heritage buildings. However, HBIM modelling from the scan data of heritage buildings is mainly manual and image processing techniques are insufficient for the segmentation of point cloud data to speed up and enhance the current workflow for HBIM modelling. Artificial Intelligence (AI) based deep learning methods such as PointNet are introduced in the literature for point cloud segmentation. Yet, their use is mainly for manufactured and clear geometric shapes and components. To what extent PointNet based segmentation is applicable for heritage buildings and how PointNet can be used for point cloud segmentation with the best possible accuracy (ACC) are tested and analysed in this paper. In this study, classification and segmentation processes are performed on the 3D point cloud data of heritage buildings in Gaziantep, Turkey. Accordingly, it proposes a novel approach of activity workflow for point cloud segmentation with deep learning using PointNet for the heritage buildings. Twenty-eight case study heritage buildings are used, and AI training is performed using five feature labelling for segmentation namely, walls, roofs, floors, doors, and windows for each of these 28 heritage buildings. The dataset is divided into clusters with 80% training dataset and 20% prediction test dataset. PointNet algorithm was unable to provide sufficient accuracy in segmenting the point clouds due to deformation and deterioration on the existing conditions of the heritage case study buildings. However, if PointNet algorithm is trained with the restitution-based heritage data, which is called synthetic data in the research, PointNet algorithm provides high accuracy. Thus, the proposed approach can build the baseline for the accurate classification and segmentation of the heritage buildings.

Semantic Segmentation of Substation Site Cloud Based on Seg-PointNet

3D point cloud semantic segmentation has been widely used in industrial scenes and has attracted continuous attention as a critical technology for understanding the intelligent robot scene. However, extracting visual semantics in complex environments remains a challenge. We propose the Seg-PointNet model based on multi-layer residual structure and feature pyramid for the LiDAR point cloud data semantic segmentation task in the complex substations scene. The model is based on the PointNet network and introduces a multi-scale residual structure. The residual structure multilayer perception (RES-MLP) model is proposed to fully excavate features at different levels and improve the characterization capabilities of complex features. Moreover, the 3D point cloud feature pyramid module is proposed to characterize the substation scene’s semantic features. We tested and verified the Seg-PointNet model on a self-built substation cloud point (SCP) dataset. The results show that the proposed Seg-PointNet model effectively improves the point cloud data segmentation accuracy, with an accuracy of 89.23% and mean intersection over union (mIoU) of 63.57%. This shows that the model can be applied to substation scenarios and provide technical support to intelligent robots in complex substation environments.

Understanding the Imperfection of 3D point Cloud and Semantic Segmentation algorithms for 3D Models of Indoor Environment

Abstract. Point clouds data provides new potentials for automated construction of more geometrically accurate and semantically rich 3D models for indoor environments. Recent advances in deep learning methods on point cloud semantic segmentation demonstrated impressive accuracy in labeling points of 3D surfaces with object classes. However, it remains challenging to reconstruct the shape of semantic objects from semantically-labeled 3D points, due to imperfection of such data and the under-determination of object construction algorithms. We have little empirical knowledge about how data imperfections affect the reconstruction of 3D indoor room objects. This paper contributes to understanding the nature of such imperfection of 3D point cloud data and semantic segmentation algorithms by analyzing the reconstructability of indoor room objects from semantically-labeled point cloud. 181 rooms from Stanford Large-Scale 3D Indoor Spaces Dataset (S3DIS) were used in our experiment. After generating semantic labels on point-clouds using PointNet++ segmentic segmentation algorithm, we use human coders to judge the reconstructability of indoor objects, following a qualitative coding scheme. Human exploration of object shape imperfection was assisted by a visual analytic tool in making their judgement. We found that high point-level accuracy achieved through semantic segmentation of point cloud data does not guarantee high object-level accuracy. The extent of this problem varies widely among different spatial settings and configurations. We discuss the significance of these findings on the choice of 3D reconstruction methods.

ROBUST INDOOR POINT CLOUD CLASSIFICATION BY FUSING LSTM NEURAL NETWORKS WITH SUPERVOXEL CLUSTERING

Abstract. To address the problems of lack of training data and inaccurate classification of existing 3D point cloud data segmentation and classification methods, this paper proposes a high-precision classification algorithm for indoor point clouds by fusing LSTM neural network and super voxels. The algorithm first performs super voxel segmentation on the original point cloud and uses it as the basic unit for machine learning classification, and then introduces LSTM (Long Short-Term Memory) neural network to model the super voxel domain relationship and optimize the classification results. Finally, the accuracy of the proposed method is evaluated based on open dataset, and the experimental results show that 83.2% classification accuracy can be achieved in the open dataset.

SnapshotNet: Self-supervised feature learning for point cloud data segmentation using minimal labeled data

Manually annotating complex scene point cloud datasets is both costly and error-prone. To reduce the reliance on labeled data, a new model called SnapshotNet is proposed as a self-supervised feature learning approach, which directly works on the unlabeled point cloud data of a complex 3D scene. The SnapshotNet pipeline includes three stages. In the snapshot capturing stage, snapshots, which are defined as local collections of points, are sampled from the point cloud scene. A snapshot could be a view of a local 3D scan directly captured from the real scene, or a virtual view of such from a large 3D point cloud dataset. Snapshots could also be sampled at different sampling rates or fields of view (FOVs), thus multi-FOV snapshots, to capture scale information from the scene. In the feature learning stage, a new pre-text task called multi-FOV contrasting is proposed to recognize whether two snapshots are from the same object or not, within the same FOV or across different FOVs. Snapshots go through two self-supervised learning steps: the contrastive learning step with both part contrasting and scale contrasting, followed by a snapshot clustering step to extract higher level semantic features. Then a weakly-supervised segmentation stage is implemented by first training a standard SVM classifier on the learned features with a small fraction of labeled snapshots. Then trained SVM is further used to predict labels for input snapshots and predicted labels are converted into point-wise label assignments for semantic segmentation of the entire scene using a voting procedure. The experiments are conducted on the Semantic3D dataset and the results have shown that the proposed method is capable of learning effective features from snapshots of complex scene data without any labels. Moreover, the proposed weakly-supervised method has shown advantages when comparing to the state of the art method on weakly-supervised point cloud semantic segmentation.

Semantic segmentation of point cloud data using raw laser scanner measurements and deep neural networks

Deep learning methods based on convolutional neural networks have shown to give excellent results in semantic segmentation of images, but the inherent irregularity of point cloud data complicates their usage in semantically segmenting 3D laser scanning data. To overcome this problem, point cloud networks particularly specialized for the purpose have been implemented since 2017 but finding the most appropriate way to semantically segment point clouds is still an open research question. In this study we attempted semantic segmentation of point cloud data with convolutional neural networks by using only the raw measurements provided by a multiple echo detection capable profiling laser scanner. We formatted the measurements to a series of 2D rasters, where each raster contains the measurements (range, reflectance, echo deviation) of a single scanner mirror rotation to be able to use the rich research done on semantic segmentation of 2D images with convolutional neural networks. Similar approach for profiling laser scanner in forest context has never been proposed before. A boreal forest in Evo region near Hämeenlinna in Finland was used as experimental study area. The data was collected with FGI Akhka-R3 backpack laser scanning system, georeferenced and then manually labelled to ground, understorey, tree trunk and foliage classes for training and evaluation purposes. The labelled points were then transformed back to 2D rasters and used for training three different neural network architectures. Further, the same georeferenced data in point cloud format was used for training the state-of-the-art point cloud semantic segmentation network RandLA-Net and the results were compared with those of our method. Our best semantic segmentation network reached the mean Intersection-over-Union value of 80.1% and it is comparable to the 80.6% reached by the point cloud -based RandLA-Net. The numerical results and visual analysis of the resulting point clouds show that our method is a valid way of doing semantic segmentation of point clouds at least in the forest context. The labelled datasets were also released to the research community.

Combining data-and-model-driven 3D modelling (CDMD3DM) for small indoor scenes using RGB-D data

To solve problems in 3D modelling, including unclear 3D edges in point clouds, lack of geometric semantics, confusing topological relations between 3D models, and low degree of automation in traditional model-driven 3D modelling, a method for high-accuracy and automatic monomer 3D modelling must be developed. This paper is the first to propose a 3D modelling strategy, called combining data-and-model-driven 3D modelling (CDMD3DM), for small regular objects in indoor scenes using RGB-D data. The proposed method’s workflow is as follows: generation of initial 3D point cloud data using data-driven Kinect v2; segmentation of point cloud data, based on deep learning, that improves the accuracy and automation of geometric model recognition; definition of the initial model-driven parameters based on the instance segmentation results; optimization of geometric model parameters, based on generalized point photogrammetry theory, to generate monomer models in indoor scenes to overcome the shortcomings of confusing topological relationships and inaccurate 3D model edges; and finally, fusion of the results of data-driven and model-driven 3D modelling. The experimental results demonstrate that CDMD3DM is feasible, automatic, more accurate, more reliable, semantically richer and capable of producing clearer topological relationships than current 3D modelling results using indoor RGB-D data. These outcomes promote interdisciplinary integration between computer vision and photogrammetry.

Learning Polynomial-Based Separable Convolution for 3D Point Cloud Analysis.

Shape classification and segmentation of point cloud data are two of the most demanding tasks in photogrammetry and remote sensing applications, which aim to recognize object categories or point labels. Point convolution is an essential operation when designing a network on point clouds for these tasks, which helps to explore 3D local points for feature learning. In this paper, we propose a novel point convolution (PSConv) using separable weights learned with polynomials for 3D point cloud analysis. Specifically, we generalize the traditional convolution defined on the regular data to a 3D point cloud by learning the point convolution kernels based on the polynomials of transformed local point coordinates. We further propose a separable assumption on the convolution kernels to reduce the parameter size and computational cost for our point convolution. Using this novel point convolution, a hierarchical network (PSNet) defined on the point cloud is proposed for 3D shape analysis tasks such as 3D shape classification and segmentation. Experiments are conducted on standard datasets, including synthetic and real scanned ones, and our PSNet achieves state-of-the-art accuracies for shape classification, as well as competitive results for shape segmentation compared with previous methods.

TDA 3D Data Model and Feature Simplification Analysis

Topology data analysis (TDA) is a combination of topology and data science. The analysis process will not cause information loss. It has been widely used in various industries in recent years. However, how to display TDA data with simpler data structure is still under exploration. This paper designs and simulates a method of TDA point cloud data segmentation and classification data topology information simplification. Firstly, the point cloud data is segmented based on the region growing method. Secondly, the idea of “topological compression” is used to express the richer data information of each segmentation part with a simpler data structure; in the experimental model, the experiment is completed on the platform of MATLAB 2016 (b), and the simple and complex representation of segmented vehicle targets completely includes all the pixel positions and gray 3D data information of the vehicle.

Research on Automatic Denoising of LIDAR Point Cloud Data for Substation Equipment Based on Spatial Grid Density

Aiming at the shortcomings of the existing 3D point cloud data automatic extraction methods of substation equipment, which are highly dependent on big data algorithms and low efficiency, this paper proposes a 3D LIDAR point cloud data segmentation method and process based on the multidimensional subspace grid density difference. The proposed method is based on eliminating the flying spots of 3D point cloud data, and is divided into equipment point cloud data and ground point cloud data based on point cloud data characteristics for 3D real-world modeling and accurate positioning of the model; Among them, the equipment point cloud data uses a multi-dimensional density difference segmentation method. The long-distance terrain is divided in the XOY and YOZ planes, and converted into a combination of multiple small-scale scale spaces. Effective segmentation, so that automatic extraction of substation equipment can be realized; The ground point cloud data uses a single-dimensional density difference segmentation method to dilute the ground point cloud data to obtain clear positioning points. The feasibility verification results of cloud data of a UHV substation show that the proposed method can effectively suppress the noise interference of interference points, realize accurate extraction and location of substation equipment, and the algorithm has high efficiency and strong engineering application.

Fuzzy Neighborhood Learning for Deep 3-D Segmentation of Point Cloud

Semantic segmentation of point cloud data, an efficient 3-D scattered point representation, is a fundamental task for various applications, such as autonomous driving and 3-D telepresence. In recent years, deep learning techniques have achieved significant progress in semantic segmentation, especially in the 2-D image setting. However, due to the irregularity of point clouds, most of them cannot be applied to this special data representation directly. While recent works are able to handle the irregularity problem and maintain the permutation invariance, most of them fail to capture the valuable high-dimensional local feature in fine granularity. Inspired by fuzzy mathematical methods and the analysis on the drawbacks of current state-of-the-art works, in this article, we propose a novel deep neural model, Fuzzy3DSeg, that is able to directly feed in the point clouds while maintaining invariant to the permutation of the data feeding order. We deeply integrate the learning of the fuzzy neighborhood feature of each point into our network architecture, so as to perform operations on high-dimensional features. We demonstrate the effectiveness of this network architecture level integration, compared with methods of the fuzzy data preprocessing cascading neural network. Comprehensive experiments on two challenging datasets demonstrate that the proposed Fuzzy3DSeg significantly outperforms the state-of-the-art methods.

Boosting Minority Class Prediction on Imbalanced Point Cloud Data

Data imbalance during the training of deep networks can cause the network to skip directly to learning minority classes. This paper presents a novel framework by which to train segmentation networks using imbalanced point cloud data. PointNet, an early deep network used for the segmentation of point cloud data, proved effective in the point-wise classification of balanced data; however, performance degraded when imbalanced data was used. The proposed approach involves removing between-class data point imbalances and guiding the network to pay more attention to majority classes. Data imbalance is alleviated using a hybrid-sampling method involving oversampling, as well as undersampling, respectively, to decrease the amount of data in majority classes and increase the amount of data in minority classes. A balanced focus loss function is also used to emphasize the minority classes through the automated assignment of costs to the various classes based on their density in the point cloud. Experiments demonstrate the effectiveness of the proposed training framework when provided a point cloud dataset pertaining to six objects. The mean intersection over union (mIoU) test accuracy results obtained using PointNet training were as follows: XYZRGB data (91%) and XYZ data (86%). The mIoU test accuracy results obtained using the proposed scheme were as follows: XYZRGB data (98%) and XYZ data (93%).

Review of 3D Point Cloud Data Segmentation Methods

Abstract 3D point cloud segmentation is one of the key steps in point cloud processing, which is the technology and process of dividing the point cloud data set into several specific regions with unique properties and proposing interesting targets. It has important applications in medical image processing, industrial inspection, cultural relic’s identification and 3D visualization. Despite widespread use, point cloud segmentation still faces many challenges because of uneven sampling density, high redundancy, and lack explicit structure of point cloud data. The main goal of this paper is to analyse the most popular algorithms and methodologies to segment point clouds. To facilitate analysis and summary, according to the principle of segmentation we divide the 3D point cloud segmentation methods into edge-based methods, region-based methods, graph-based methods, model-based methods, and machine learning-based methods. Then analyze and discuss the advantages, disadvantages and application scenarios of these segmentation methods. For some algorithms the results of the segmentation and classification is shown. Finally, we outline the issues that need to be addressed and important future research directions.

Point Cloud Data Segmentation Using RANSAC and Localization

In this paper, we present 3D point cloud data segmentation from a Terrestrial Laser Scanner (TLS) using RANSAC and localization approaches. Our work is using the real world data acquired from outdoor and indoor scene of Tawau Bell Tower (also known as the Belfry), one of the heritage building located in Sabah. We adapt the methods to segment ground and nonground data for data reduction. Based on the non-ground information and geometrical constraint, the data managed to show the building structure. The aim is to use and handle a smaller set of data rather than using the entire point cloud data in processing the region of interest. This will also help to reduce processing time as point cloud data from a TLS is usually large in size. The ground plane was fitted with RANSAC algorithm restricted by distance to describe the geometrical of the plane. As a result, the point cloud data was reduced to the environment around Bell Tower and able to highlight the region of interest. Building segmentation is then retrieved from the reference centre location to form a bounding box and data were organized using kd-tree to denote the Bell Tower structure. The framework is tested for outdoor and indoor scan of the Bell Tower and able to segment the data appropriately.