Point Cloud Datasets Research Articles

A real-time automatic rail extraction algorithm for low-density mobile laser scanning data

Automatic detection of railroad infrastructure using Mobile Laser Scanning systems is a key technology for both advanced rail driver assistance and intelligent track maintenance. The recent research into railway facility extraction or condition monitoring usually relies on high-density point cloud dataset with known sensor parameters but ignores the processing performance in actual deployment and has high requirements for the acquisition device. To address these limitations, a novel rail extraction algorithm is proposed for processing Mobile Laser Scanning data in real time during acquisition, which could extract rail features by a hierarchical coarse-to-fine method with basic structural parameters. Using the geometric and global statistical characteristics of rail in raw data, a new rail descriptor is defined based on the quantitative statistics of points satisfying height difference in generalized local neighborhood. The approach is evaluated experimentally by a simulated real-time acquisition data and compared with a reference method. The experimental results show that the proposed algorithm has a finer extraction effect and good real-time performance.

A high-accuracy multi-temporal laser point cloud registration method for providing a detailed expression of the urban environment

In consideration of the significant changes in urban environmental scenes and the impact of factors such as signal obstruction on positioning, the spatial positions of multi-temporal Mobile Laser Scanning (MLS) point clouds collected in the same area are inconsistent, which makes it challenging to provide a complete expression of the urban road environment. To address this issue, a precise registration algorithm based on pole-like targets is proposed. The core steps include registration primitive extraction: a cylindrical spatial neighborhood is established based on the distinctive key points, and precise extraction of pole-like geospatial point cloud data is performed using the Euclidean clustering method with the normal curvature constraint. Achieving optimal spatial transformation: a circle center fitting method with additional Mean Absolute Error (MAE) loss function constraints was developed based on the stable elemental unit which was utilized to accurately construct homonymous feature point pairs. Validation results from multiple real urban point cloud datasets collected from different experimental areas and platforms demonstrate that our method achieves superior registration performance, with mean residuals below 5 cm in both the planar and elevation directions. The proposed method exhibits strong universality and registration robustness for multi-temporal point cloud data of the homologous or cross-sources.

A multilayered urban tree dataset of point clouds, quantitative structure and graph models

The significance of urban trees in promoting human health and well-being has been amplified by urbanization and the climate change effects. Simultaneously, advancements in remote sensing techniques have enhanced the opportunities for studying urban trees. The TreeML-Data has been compiled to support these efforts. It consists of labelled point clouds of 40 scanning projects of streets in Munich, 3,755 leaf-off (scans in winter) point clouds of individual trees, quantitative structure models (QSM), tree structure measurements, and tree graph structure models of these trees. The dataset offers valuable data for generating and evaluating models in various scientific disciplines, which include remote sensing, computer vision, machine learning, urban forestry, urban ecosystem, green architecture, and graph analysis. To ensure its quality, the tree structure measurements and QSM have been crosschecked. For instance, the tree diameter at breast height (DBH) in the sample dataset exhibits a deviation of approximately 1.5 cm (4.3%) when compared to manual measurements. In conclusion, the quality checks confirm its reliability for subsequent studies when compared to manual measurements.

Feature extraction and representation learning of 3D point cloud data

Three-dimensional point cloud data serves as a critical source of information in various real-world application domains, such as computer vision, robotics, geographic information systems, and medical image processing. Due to the discrete and unordered nature of point clouds, applying 2D image feature extractors directly to the extraction of 3D point cloud features is challenging. Therefore, we propose a novel variational feature component extraction method called PointFEA. This paper aims to research and propose a series of methods to enhance the feature extraction and representation learning of 3D point cloud data. Firstly, in terms of feature extraction, local neighborhood encoding is combined with the local latent representation of point clouds to obtain more correlated point cloud features. Secondly, in the domain of point cloud representation learning, the multi-scale representation learning method maps point cloud data into a high-dimensional space to better capture critical features and adapt to different granularities of point cloud data. Lastly, features of different dimensions are input into a cross-fusion transformer to obtain local attention coefficients. We validate our methods on commonly used point cloud datasets, and the experiments demonstrate the effectiveness of our approach, achieving accuracies of 94.8% on ModelNet40 and 89.1% on ScanObjectNN.

Extraction of Building Roof Contours from Airborne LiDAR Point Clouds Based on Multidirectional Bands

Because of the complex structure and different shapes of building contours, the uneven density distribution of airborne LiDAR point clouds, and occlusion, existing building contour extraction algorithms are subject to such problems as poor robustness, difficulty with setting parameters, and low extraction efficiency. To solve these problems, a building contour extraction algorithm based on multidirectional bands was proposed in this study. Firstly, the point clouds were divided into bands with the same width in one direction, the points within each band were vertically projected on the central axis in the band, the two projection points with the farthest distance were determined, and their corresponding original points were regarded as the roof contour points; given that the contour points obtained based on single-direction bands were sparse and discontinuous, different banding directions were selected to repeat the above contour point marking process, and the contour points extracted from the different banding directions were integrated as the initial contour points. Then, the initial contour points were sorted and connected according to the principle of joining the nearest points in the forward direction, and the edges with lengths greater than a given threshold were recognized as long edges, which remained to be further densified. Finally, each long edge was densified by selecting the noninitial contour point closest to the midpoint of the long edge, and the densification process was repeated for the updated long edge. In the end, a building roof contour line with complete details and topological relationships was obtained. In this study, three point cloud datasets of representative building roofs were chosen for experiments. The results show that the proposed algorithm can extract high-quality outer contours from point clouds with various boundary structures, accompanied by strong robustness for point clouds differing in density and density change. Moreover, the proposed algorithm is characterized by easily setting parameters and high efficiency for extracting outer contours. Specific to the experimental data selected for this study, the PoLiS values in the outer contour extraction results were always smaller than 0.2 m, and the RAE values were smaller than 7%. Hence, the proposed algorithm can provide high-precision outer contour information on buildings for applications such as 3D building model reconstruction.

Surface and underwater human pose recognition based on temporal 3D point cloud deep learning

Airborne surface and underwater human pose recognition are crucial for various safety and surveillance applications, including the detection of individuals in distress or drowning situations. However, airborne optical cameras struggle to achieve simultaneous imaging of the surface and underwater because of limitations imposed by visible-light wavelengths. To address this problem, this study proposes the use of light detection and ranging (LiDAR) to simultaneously detect humans on the surface and underwater, whereby human poses are recognized using a neural network designed for irregular data. First, a temporal point-cloud dataset was constructed for surface and underwater human pose recognition to enhance the recognition of comparable movements. Subsequently, radius outlier removal (ROR) and statistical outlier removal (SOR) were employed to alleviate the impact of noise and outliers in the constructed dataset. Finally, different combinations of secondary sampling methods and sample sizes were tested to improve recognition accuracy using PointNet++. The experimental results show that the highest recognition accuracy reached 97.5012%, demonstrating the effectiveness of the proposed human pose detection and recognition method.

PointCHD: A Point Cloud Benchmark for Congenital Heart Disease Classification and Segmentation.

Congenital heart disease (CHD) is one of the most common birth defects. With the development of medical imaging analysis technology, medical image analysis for CHD has become an important research direction. Due to the lack of data and the difficulty of labeling, CHD datasets are scarce. Previous studies focused on CT and other medical image modes, while point cloud is still unstudied. As a representative type of 3D data, point cloud can intuitively model organ shapes, which has obvious advantages in medical analysis and can assist doctors in diagnosis. However, the production of a medical point cloud dataset is more complex than that of an image dataset, and the 3D modeling of internal organs needs to be reconstructed after scanning by high-precision instruments. We propose PointCHD, the first point cloud dataset for CHD diagnosis, with a large number of high precision-annotated and wide-categorized data. PointCHD includes different types of three-dimensional data with varying degrees of distortion, and supports multiple analysis tasks, i.e. classification, segmentation, reconstruction, etc. We also construct a benchmark on PointCHD with the goal of medical diagnosis, we design the analysis process and compare the performances of the mainstream point cloud analysis methods. In view of the complex internal and external structure of the heart point cloud, we propose a point cloud representation learning method based on manifold learning. By introducing normal lines to consider the continuity of the surface to construct a manifold learning method of the adaptive projection plane, fully extracted the structural features of the heart, and achieved the best performance on each task of the PointCHD benchmark. Finally, we summarize the existing problems in the analysis of the CHD point cloud and prospects for potential research directions in the future. The benchmark will be released soon.

Descriptor: A Comprehensive LiDAR Point Cloud Dataset of Telecommunication Towers (TowerScope)

Descriptor: A Comprehensive LiDAR Point Cloud Dataset of Telecommunication Towers (TowerScope)

Curriculumformer: Taming Curriculum Pre-Training for Enhanced 3-D Point Cloud Understanding.

Learning universal representations of 3-D point clouds is essential for reducing the need for manual annotation of large-scale and irregular point cloud datasets. The current modus operandi for representative learning is self-supervised learning, which has shown great potential for improving point cloud understanding. Nevertheless, it remains an open problem how to employ auto-encoding for learning universal 3-D representations of irregularly structured point clouds, as previous methods focus on either global shapes or local geometries. To this end, we present a cascaded self-supervised point cloud representation learning framework, dubbed Curriculumformer, aiming to tame curriculum pre-training for enhanced point cloud understanding. Our main idea lies in devising a progressive pre-training strategy, which trains the Transformer in an easy-to-hard manner. Specifically, we first pre-train the Transformer using an upsampling strategy, which allows it to learn global information. Then, we follow up with a completion strategy, which enables the Transformer to gain insight into local geometries. Finally, we propose a Multi-Modal Multi-Modality Contrastive Learning (M4CL) strategy to enhance the ability of representation learning by enriching the Transformer with semantic information. In this way, the pre-trained Transformer can be easily transferred to a wide range of downstream applications. We demonstrate the superior performance of Curriculumformer on various discriminant and generative tasks, outperforming state-of-the-art methods. Moreover, Curriculumformer can also be integrated into other off-the-shelf methods to promote their performance. Our code is available at https://github.com/Fayeben/Curriculumformer.

An investigation of geometric feature recognition in 3D ship data

The intelligent recognition of ship geometric features is a prerequisite for enabling computers to automatically generate and deform ship hull surfaces according to requirements, thereby replacing the work of human designers to improve design efficiency. This paper aims to research the recognition of geometric features in three-dimensional ship data using PointNet. To achieve this goal, we first construct two ship point cloud datasets suitable for global feature classification and feature part segmentation of three-dimensional hulls. Subsequently, we conducted recognition capability testing to determine the optimal hyperparameters for identifying ship feature networks. Finally, we employ ship models with non-standard positions to implement data augmentation, enhancing the network's robustness in recognizing the initial positions of ships and achieving rapid cognition of three-dimensional ship geometric features. The findings of this research will provide technical support for ship design based on artificial intelligence technology.

Segmentation of LiDAR point cloud data in urban areas using adaptive neighborhood selection technique.

Semantic segmentation of urban areas using Light Detection and Ranging (LiDAR) point cloud data is challenging due to the complexity, outliers, and heterogeneous nature of the input point cloud data. The machine learning-based methods for segmenting point clouds suffer from the imprecise computation of the training feature values. The most important factor that influences how precisely the feature values are computed is the neighborhood chosen by each point. This research addresses this issue and proposes a suitable adaptive neighborhood selection approach for individual points by completely considering the complex and heterogeneous nature of the input LiDAR point cloud data. The proposed approach is evaluated on high-density mobile and low-density aerial LiDAR point cloud datasets using the Random Forest machine learning classifier. In the context of performance evaluation, the proposed approach confirms the competitive performance over the state-of-the-art approaches. The computed accuracy and F1-score for the high-density Toronto and low-density Vaihingen datasets are greater than 91% and 82%, respectively.

PointWavelet: Learning in Spectral Domain for 3-D Point Cloud Analysis.

With recent success of deep learning in 2-D visual recognition, deep-learning-based 3-D point cloud analysis has received increasing attention from the community, especially due to the rapid development of autonomous driving technologies. However, most existing methods directly learn point features in the spatial domain, leaving the local structures in the spectral domain poorly investigated. In this article, we introduce a new method, PointWavelet, to explore local graphs in the spectral domain via a learnable graph wavelet transform. Specifically, we first introduce the graph wavelet transform to form multiscale spectral graph convolution to learn effective local structural representations. To avoid the time-consuming spectral decomposition, we then devise a learnable graph wavelet transform, which significantly accelerates the overall training process. Extensive experiments on four popular point cloud datasets, ModelNet40, ScanObjectNN, ShapeNet-Part, and S3DIS, demonstrate the effectiveness of the proposed method on point cloud classification and segmentation.

New Reference Bathymetric Point Cloud Datasets Derived From ICESat-2 Observations: A Case in the Caribbean Sea

New Reference Bathymetric Point Cloud Datasets Derived From ICESat-2 Observations: A Case in the Caribbean Sea

AI-based landing zone detection for vertical takeoff and land LiDAR localization and mapping pipelines

This work describes a deep learning-based autonomous landing zone identification module for a vertical takeoff and landing vehicle. The proposed module is developed using LiDAR point cloud data and can be integrated into a visual LiDAR odometry and mapping pipeline implemented in the vehicle. “ConvPoint,” the top-performing neural network architecture in an online point cloud segmentation benchmark leaderboard at the time of writing, was chosen as the reference architecture. Semantic labeling of the datasets was done using the terrain geometry characteristics and manual adjustment of labels through visual observation. Point clouds captured by the Memorial University and online point cloud datasets were used to transfer-learn the neural network model and to evaluate the accuracy-runtime trade-off for the proposed pipeline. The selected neural network model generated accuracy values of 89.7% and 92.1% on two selected datasets, while it computed 3940.15 points per second and 3633.85 points per second to predict landing zone labels, respectively. Hyperparameter tuning was carried out to obtain a higher throughput with an update rate of 1 Hz for the landing zone map of the point cloud inputs from the visual LiDAR odometry and mapping pipeline. The proposed system is validated by evaluating its performance on three variations of point clouds. The results validate the accuracy-runtime trade-off of the proposed system and show that further optimization can improve performance.

Cucumber Seedling Segmentation Network Based on a Multiview Geometric Graph Encoder from 3D Point Clouds.

Plant phenotyping plays a pivotal role in observing and comprehending the growth and development of plants. In phenotyping, plant organ segmentation based on 3D point clouds has garnered increasing attention in recent years. However, using only the geometric relationship features of Euclidean space still cannot accurately segment and measure plants. To this end, we mine more geometric features and propose a segmentation network based on a multiview geometric graph encoder, called SN-MGGE. First, we construct a point cloud acquisition platform to obtain the cucumber seedling point cloud dataset, and employ CloudCompare software to annotate the point cloud data. The GGE module is then designed to generate the point features, including the geometric relationships and geometric shape structure, via a graph encoder over the Euclidean and hyperbolic spaces. Finally, the semantic segmentation results are obtained via a downsampling operation and multilayer perceptron. Extensive experiments on a cucumber seedling dataset clearly show that our proposed SN-MGGE network outperforms several mainstream segmentation networks (e.g., PointNet++, AGConv, and PointMLP), achieving mIoU and OA values of 94.90% and 97.43%, respectively. On the basis of the segmentation results, 4 phenotypic parameters (i.e., plant height, leaf length, leaf width, and leaf area) are extracted through the K-means clustering method; these parameters are very close to the ground truth, and the R 2 values reach 0.98, 0.96, 0.97, and 0.97, respectively. Furthermore, an ablation study and a generalization experiment also show that the SN-MGGE network is robust and extensive.

An Improved PointNet++ Based Method for 3D Point Cloud Geometric Features Segmentation in Mechanical Parts

The extraction of geometric features such as holes, arcs, and surfaces of mechanical parts is crucial for quality control. The existing methods for geometrical feature segmentations on 3D point clouds still have limitations, especially for simultaneously extracting multiple types of geometric features from comprehensive workpieces. To this end, this study investigates segmentation methods that take 3D point cloud datasets of mechanical parts as inputs, and employs an improved PointNet++ deep learning model to solve this extraction difficulty. Firstly, the Set Abstraction module in PointNet++ was modified by incorporating Self-Attention mechanisms to increase interactivity and global correlation among data points. Then, the local feature extraction Multilayer Perceptron (MLP) from PointNet-Transformer was integrated to enhance the feature extraction accuracy. Due to the inherent class imbalance issue, the Focal Tversky Loss is employed as the loss function to ensure that geometric features with relatively lower proportions can be fully trained. Finally, the Statistical filtering algorithm is utilized to mitigate noise and attenuate subtle irregularities, such that the smoothness of geometric features can be substantially enhanced. The experimental results demonstrate that the proposed model achieves an accuracy of 86.6% on geometric feature segmentations and a mean Intersection over Union (mIoU) of 0.84. The comparison with the original PointNet++ proves that the proposed method can improve accuracy and mIoU by 3.7% and 0.03 respectively.

Tree-PointNet: A Novel Neural Network for Dormant Apple Trees Using Semi-Circle-Based Point Cloud for Robotic Pruning

HighlightsTree-PointNet, a novel neural network, integrates geometric down-sampling for efficient pruning.A Semi-Circle-Based point cloud dataset of apple trees was collected to aid in automated pruning.Enhanced algorithm performance was investigated, showing significant improvements in model accuracy. Abstract. Pruning during the dormant season is one of the most costly and labor-intensive operations in the production of specialty crops. In winter, many trained seasonal workers have to carefully remove branches from hundreds of trees according to pre-defined rules. The aim of automated pruning is to reduce this reliance on the large amount of labor currently required for the operation. The first step in the automatic application of pruning rules is the acquisition of point cloud data from trees. To match the optical vision system of an ideal automatic pruning robot, a brand new and challenging apple tree dataset was collected in which the point clouds were all based on semicircular shapes, rather than the carefully crafted point clouds of complete apple trees. 1000 Semi-Circle-Based point clouds of apple trees were used. For the brand-new structure of point cloud part segmentation, a neural network model (namely Tree-PointNet) was proposed. A Semi-Circle-Based point cloud of apple tree was segmented by the Tree-PointNet with the segmentation result consisting of a trunk and primary branches. With a weight size of only 7.9% of PointStack, the best OA of 95.6% was obtained by the Tree-PointNet network proposed in this paper. The GDS was also used, which could obtain more key points in the Branch-to-Trunk connection region as input to the proposed network. On the other hand, the OA and mIoU of the model improved by 10.2% and 7.98%, respectively, compared to the PointNet++. Keywords: Dormant apple trees, Part segmentation, PointNet++, Semi-circle-based point cloud.

A Survey of Label-Efficient Deep Learning for 3D Point Clouds.

In the past decade, deep neural networks have achieved significant progress in point cloud learning. However, collecting large-scale precisely-annotated point clouds is extremely laborious and expensive, which hinders the scalability of existing point cloud datasets and poses a bottleneck for efficient exploration of point cloud data in various tasks and applications. Label-efficient learning offers a promising solution by enabling effective deep network training with much-reduced annotation efforts. This paper presents the first comprehensive survey of label-efficient learning of point clouds. We address three critical questions in this emerging research field: i) the importance and urgency of label-efficient learning in point cloud processing, ii) the subfields it encompasses, and iii) the progress achieved in this area. To this end, we propose a taxonomy that organizes label-efficient learning methods based on the data prerequisites provided by different types of labels. We categorize four typical label-efficient learning approaches that significantly reduce point cloud annotation efforts: data augmentation, domain transfer learning, weakly-supervised learning, and pretrained foundation models. For each approach, we outline the problem setup and provide an extensive literature review that showcases relevant progress and challenges. Finally, we share our views on the current research challenges and potential future directions. A project associated with this survey has been built at https://github.com/xiaoaoran/3D label efficient learning.

Intelligent Recognition of Component Building Elements in Assembly Buildings with BIM-based Virtual Point Cloud Dataset

Assembly building is one of the most important directions for the transformation and upgrading of the construction industry as a whole, and the traditional inspection methods of assembly components are low in efficiency and precision. How to conduct rapid batch high-precision size inspection of the actual production of assembly components has become a key problem that must be solved for the popularization of assembly building. In this research, 3D computer vision and point cloud deep learning technology were applied to the intelligent identification of assembly components, combined with BIM technology. We proposed a method to produce virtual point cloud dataset based on BIM model, and on this basis, we used the point cloud deep learning technology to automatically identify the dimensions of prefabricated components, which further improves the inspection efficiency and accuracy and has a stronger significance for promoting the application of deep learning technology in the field of engineering. The research has strong significance for promoting the application of deep learning technology in the engineering field.

Statistically assessing vertical change on a sandy beach from permanent laser scanning time series

In the view of climate change, understanding and managing effects on coastal areas and adjacent cities is essential. Permanent Laser Scanning (PLS) is a successful technique to not only observe notably sandy coasts incidentally or once every year, but (nearly) continuously over extended periods of time. The collected point cloud observations form a 4D point cloud data set representing the evolution of the coast provide the opportunity to assess change processes at high level of detail. For an exemplary location in Noordwijk, The Netherlands, three years of hourly point clouds were acquired on a 1 km long section of a typical Dutch urban sandy beach. Often, the so-called level of detection is used to assess point cloud differences from two epochs. To explicitly incorporate the temporal dimension of the height estimates from the point cloud data set, we revisit statistical testing theory. We apply multiple hypothesis testing on elevation time series in order to identify different coastal processes, like aeolian sand transport or bulldozer works. We then estimate the minimal detectable bias for different alternative hypotheses, to quantify the minimal elevation change that can be estimated from the PLS observations over a certain period of time. Additionally, we analyse potential error sources and influences on the elevation estimations and provide orders of magnitudes and possible ways to deal with them. Finally we conclude that elevation time series from a long term PLS data set are a suitable input to identify aeolian sand transport with the help of multiple hypothesis testing. In our example case, slopes of 0.032 m/day and sudden changes of 0.031 m can be identified with statistical power of 80% and with 95% significance in 24-h time series on the upper beach. In the intertidal area the presented method allows to classify daily elevation time series over one month according to the dominating model (sudden change or linear trend) in either eroding or accreting behaviour.