Cloud Classification Research Articles

DPFANet: Deep Point Feature Aggregation Network for Classification of Irregular Objects in LIDAR Point Clouds

Point cloud data acquired by scanning with Light Detection and Ranging (LiDAR) devices typically contain irregular objects, such as trees, which lead to low classification accuracy in existing point cloud classification methods. Consequently, this paper proposes a deep point feature aggregation network (DPFANet) that integrates adaptive graph convolution and space-filling curve sampling modules to effectively address the feature extraction problem for irregular object point clouds. To refine the feature representation, we utilize the affinity matrix to quantify inter-channel relationships and adjust the input feature matrix accordingly, thereby improving the classification accuracy of the object point cloud. To validate the effectiveness of the proposed approach, a TreeNet dataset was created, comprising four categories of tree point clouds derived from publicly available UAV point cloud data. The experimental findings illustrate that the model attains a mean accuracy of 91.4% on the ModelNet40 dataset, comparable to prevailing state-of-the-art techniques. When applied to the more challenging TreeNet dataset, the model achieves a mean accuracy of 88.0%, surpassing existing state-of-the-art methods in all classification metrics. These results underscore the high potential of the model for point cloud classification of irregular objects.

A novel method: YOLO-CE and 3D point cloud-based feature extraction for welding seams of tower bases

Abstract Robotic automated welding of non-standard steel structures presents significant challenges, particularly for electric power tower bases. This study introduces a novel approach that integrates the You Only Look Once—Compact Invert Block and Efficient Local Attention (YOLO-CE) model, an enhanced version of YOLOV8 for 2D image segmentation, with 3D point cloud technology. The YOLO-CE model is used to accurately extract point cloud data from the target area, which is then processed using the MSAC algorithm for efficient plane segmentation. Weld lines are identified through plane equations, allowing for initial weld point cloud extraction. To further refine accuracy, an optimized evaluation equation is developed that accounts for both the distance between the weld point cloud and the fitted plane, and the angle between their normal vectors. This enables precise classification of the weld point cloud. From this classification, key weld feature points are identified, and their exact positions are determined by calculating the distances between these points and their intersections with three planes. The reliability of the proposed method was validated using a robot for precise measurements, with a total error margin of less than 1.5084 mm, demonstrating high accuracy and stability. Post-operation inspections confirmed that the welds were filled and free from defects, meeting all process requirements. The YOLO-CE model achieved a mIoU of 96.38% and a precision of 99.8%, highlighting its effectiveness. This method provides an efficient and precise solution for the automated welding of non-standard steel structural components and has promising application potential.

Airborne LiDAR Point Cloud Classification Using Ensemble Learning for DEM Generation

Airborne laser scanning (ALS) point clouds have emerged as a predominant data source for the generation of digital elevation models (DEM) in recent years. Traditionally, the generation of DEM using ALS point clouds involves the steps of point cloud classification or ground point filtering to extract ground points and labor-intensive post-processing to correct the misclassified ground points. The current deep learning techniques leverage the ability of geometric recognition for ground point classification. However, the deep learning classifiers are generally trained using 3D point clouds with simple geometric terrains, which decrease the performance of model inferencing. In this study, a point-based deep learning model with boosting ensemble learning and a set of geometric features as the model inputs is proposed. With the ensemble learning strategy, this study integrates specialized ground point classifiers designed for different terrains to boost classification robustness and accuracy. In experiments, ALS point clouds containing various terrains were used to evaluate the feasibility of the proposed method. The results demonstrated that the proposed method can improve the point cloud classification and the quality of generated DEMs. The classification accuracy and F1 score are improved from 80.9% to 92.2%, and 82.2% to 94.2%, respectively, by using the proposed methods. In addition, the DEM generation error, in terms of mean squared error (RMSE), is reduced from 0.318–1.362 m to 0.273–1.032 m by using the proposed ensemble learning.

Cloud classification through machine learning and global horizontal irradiance data analysis

AbstractCloud observations and characterization are crucial owing to their influence on energy balance, climate, and weather. Their particular effects on radiation vary depending on different cloud parameters, such as cloud base or top height, water content, and cloud optical thickness, all of them closely related to the specific cloud type. Cloud classification therefore becomes a crucial task in meteorology, although it remains challenging for weather services worldwide owing to the intensive associated labor and cost. In this study we introduce a new low‐cost method for automating cloud classification based on a combination of ground‐based global horizontal irradiance (GHI) measurements, a clear‐sky model, and machine learning. Based on the hypothesis that different cloud types have their own GHI signatures, we trained different supervised learning algorithms using GHI data manually labeled by meteorological observers from time‐synchronized all‐sky images. Multiple time windows were extracted from each GHI series, with eight features defined in each case to characterize the sequence. The best outcome was achieved using an XGBoost model on features extracted on time windows of 33 min, obtaining an accuracy of 0.88 and a Cohen's kappa of 0.84 in a held‐out test set. The development presented in this study has the ability to provide low‐cost cloud classification from ground‐based observations, which is a challenge for weather services worldwide owing to intensive labor and cost.

Climatology of Cirrus Clouds over Observatory of Haute-Provence (France) Using Multivariate Analyses on Lidar Profiles

This study aims to achieve the classification of the cirrus clouds over the Observatory of Haute-Provence (OHP) in France. Rayleigh–Mie–Raman lidar measurements, in conjunction with the ERA5 dataset, are analyzed to provide geometrical morphology and optical cirrus properties over the site. The method of cirrus cloud climatology presented here is based on a threefold classification scheme based on the cirrus geometrical and optical properties and their formation history. Principal component analysis (PCA) and subsequent clustering provide four morphological cirrus classes, three optical groups, and two origin-related categories. Cirrus clouds occur approximately 37% of the time, with most being single-layered (66.7%). The mean cloud optical depth (COD) is 0.39 ± 0.46, and the mean heights range around 10.8 ± 1.35 km. Thicker tropospheric cirrus are observed under higher temperature and humidity conditions than cirrus observed in the vicinity of the tropopause level. Monthly cirrus occurrences fluctuate irregularly, whereas seasonal patterns peak in spring. Concerning the mechanism of the formation, it is found that the majority of cirrus clouds are of in situ origin. The liquid-origin cirrus category consists nearly entirely of thick cirrus. Overall results suggest that in situ origin thin cirrus, located in the upper tropospheric and tropopause regions, have the most noteworthy occurrence over the site.

HBIM MODELLING PROCESS FROM 3D POINT CLOUDS BY APPLYING ARTIFICIAL INTELLIGENCE ALGORITHMS IN CULTURAL HERITAGE

In the context of Cultural Heritage (CH), the widespread adoption of 3D point cloud technology, coupled with Artificial Intelligence (AI) algorithms, plays a pivotal role. These technologies facilitate the creation of as-built models by integrating Building Information Modelling (BIM) strategies, enhancing collaboration within the Architecture, Engineering, and Construction (AEC) sector. Leveraging computer vision, robotics, and remote sensing, 3D points clouds provide rich data. However, manual segmentation and classification are labor-intensive and error prone. Consequently, researchers increasingly turn to machine learning (ML) and deep learning (DL) techniques for automating these tasks. The transition from manual reconstruction to automated procedures is crucial. Despite progress, gaps remain, particularly in incorporating 3D point cloud segmentation into Historical Building Information Modelling (HBIM). The lack of conclusive evidence regarding automated derivation of parametric attributes from segmentation outcomes underscores the need for further exploration. Addressing this gap is essential for cultural asset documentation, conservation, and upkeep. By automating the segmentation and classification of 3D point clouds, efficient communication via a shared database becomes feasible. The article aims to review studies on semantically parsing and classifying 3D point clouds using AI algorithms, particularly within complex cultural heritage geometries, shedding light on potential benefits and barriers.

Intelligent processing of UAV remote sensing data for building high-precision DEMs in complex terrain: A case study of Loess Plateau in China

The Loess Plateau in China is renowned for its dense gullies and complex terrain, with drastic changes primarily due to soil erosion and human activities, significantly affecting the evolution of the ecological environment. The complex terrains and dense vegetation make precise terrain measurement and modeling challenging. Although the development of Unmanned Aerial Vehicle (UAV) light detection and ranging (LiDAR) scanning and photogrammetry technologies has improved data acquisition precision, relying solely on one remote sensing technology struggles with accurately extracting bare earth information. This study adopted a method that fuses UAV lidar scanning with aerial photogrammetric imagery, generating detailed lidar point cloud data that includes coordinate, reflectance, true color, and texture information to enhance data classifiability and interpretability. Subsequently, a point cloud classification model based on the Transformer architecture (Stratified Transformer) is introduced to intelligently complete the initial ground point cloud extraction in complex gully terrains. Further, to address residual non-ground noise in the initial ground point clouds, a new point cloud classification optimization algorithm (MDD, Multi-scale C2M Distance Difference) is proposed. This algorithm, based on the characteristics of discrete and non-continuous with the ground surface of the noisy point clouds, effectively eliminates the discrete noisy point clouds by analyzing the distances between the point clouds and TINs (Triangular Irregular Networks) of different scales and their differences. This study effectively addresses the technical challenges of ground point cloud extraction in the mixed environment of complex terrain and vegetation, solving the problem of precise terrain measurement and intelligent data processing in complex gully terrains, and offering new technical pathways for detecting geomorphological changes.

Hyperbolic prototype rectification for few-shot 3D point cloud classification

Few-shot point cloud classification is a challenging task in 3D computer vision and has received widespread attention from researchers. Most of the works on deep learning models rely heavily on Euclidean spatial metrics. However, point cloud objects often have complex non-Euclidean geometric structures, with underlying inter/intra-class hierarchical structures, which are difficult to capture by current Euclidean-based deep learning models. Moreover, due to the lack of training samples, many few-shot learning methods often suffer from the overfitting problem. Given the Hyperbolic metric of non-Euclidean geometry offering hierarchical structural prior, as we assume to be able to assist FSL task, we propose Hyperbolic Prototype Rectification (HPR) for few-shot point cloud classification, without requiring extra learnable parameter. Firstly, point clouds are embedded into hyperbolic space to better describe hierarchical similarity relationships in data. Secondly, the HPR utilizes hyperbolic spatial and distributional information to enhance the feature representation and improve the generalization capability, with more appropriate hyperbolic prototypes. The few-shot classification experiments and further ablation studies conducted on widely used point cloud datasets demonstrate the effectiveness of our method. On the real-world ScanObjectNN(-PB) datasets, the average classification accuracy outperforms the SOTA method by 2.08%(0.66%), respectively, indicating that the proposed HPR has great generalization capability and strong robustness against perturbed data. Our code is available at: https://github.com/Jonathan-UCAS/HPR.

Detection and Tracking in Human Monitoring Framework Using Modified Direct 3D LiDAR Point Cloud Classifier Based on Region Cluster Proposal

Detection and Tracking in Human Monitoring Framework Using Modified Direct 3D LiDAR Point Cloud Classifier Based on Region Cluster Proposal

Geometric characterization and segmentation of historic buildings using classification algorithms and convolutional networks in HBIM

Building Information Models (BIM) are essential for managing information and creating 3D digital representations, especially in the study of historic buildings. However, generating BIM models from point clouds in these structures is challenging due to complex algorithms and architectural forms. Artificial Intelligence (AI) technologies are beginning to automate point cloud classification and segmentation, but fully effective methods for historic buildings are still lacking. This study compares Machine Learning (ML) methodologies and a Deep Learning (DL) classifier. It evaluates the effectiveness of a neighbourhood algorithm with commercial software used by geometers and surveyors, and the applicability of convolutional networks. The methods tested include the Random Forest algorithm in MATLAB, commercial geomatics software, and a variant of the PointNet architecture for DL. The results are evaluated by BIM experts, highlighting the high effectiveness of these approaches and their potential contributions to the field.

MLF-PointNet++: A Multifeature-Assisted and Multilayer Fused Neural Network for LiDAR-UAS Point Cloud Classification in Estuarine Areas

LiDAR-unmanned aerial system (LiDAR-UAS) technology can accurately and efficiently obtain detailed and accurate three-dimensional spatial information of objects. The classification of objects in estuarine areas is highly important for management, planning, and ecosystem protection. Owing to the presence of slopes in estuarine areas, distinguishing between dense vegetation (lawns and trees) on slopes and the ground at the tops of slopes is difficult. In addition, the imbalance in the number of point clouds also poses a challenge for accurate classification directly from point cloud data. A multifeature-assisted and multilayer fused neural network (MLF-PointNet++) is proposed for LiDAR-UAS point cloud classification in estuarine areas. First, the 3D shape features that characterize the geometric characteristics of targets and the visible-band difference vegetation index (VDVI) that can characterize vegetation distribution are used as auxiliary features to enhance the distinguishability of dense vegetation (lawns and trees) on slopes and the ground at the tops of slopes. Second, to enhance the extraction of target spatial information and contextual relationships, the feature vectors output by different layers of set abstraction in the PointNet++ model are fused to form a combined feature vector that integrates low and high-level information. Finally, the focal loss function is adopted as the loss function in the MLF-PointNet++ model to reduce the effect of imbalance in the number of point clouds in each category on the classification accuracy. A classification evaluation was conducted using LiDAR-UAS data from the Moshui River estuarine area in Qingdao, China. The experimental results revealed that MLF-PointNet++ had an overall accuracy (OA), mean intersection over union (mIOU), kappa coefficient, precision, recall, and F1-score of 0.976, 0.913, 0.960, 0.953, 0.953, and 0.953, respectively, for object classification in the three representative areas, which were better than the corresponding values for the classification methods of random forest, BP neural network, Naive Bayes, PointNet, PointNet++, and RandLA-Net. The study results provide effective methodological support for the classification of objects in estuarine areas and offer a scientific basis for the sustainable development of these areas.

A novel method for the classification of 3D point clouds based on the improved PointNet++

Abstract In deep learning, point clouds are used as the primary input format for 3D data, which can provide detailed geometric information about objects in the original 3D space. PointNet++ is a deep learning network that uses point cloud data as an input format, which avoids the losses associated with the previous conversion of point cloud into 3D voxelization and a collection of 2D images. Although PointNet++ can directly process point cloud data in various ways, due to the disordered, irregular, and unevenly distributed nature of point cloud data, the effect of extracting point cloud features could be better. The large amount of point cloud data also leads to the training model falling into the local optimal solution, which affects the training results. In recent years, some effective methods and strategies have emerged to address these problems. In this thesis, three methods are proposed based on the PointNet++ network: feature similarity-based attention pooling, small kernel convolution, and diverse branch block method to improve the performance of the PointNet++ network. Experiments show that the improvement methods proposed in this paper effectively improve the feature extraction accuracy, which improves the accuracy of the PointNet++ network for classification on the ModelNet40_Normal_Resampled dataset, with an overall improvement of 1% compared with PointNet++.

3DFFL: privacy-preserving Federated Few-Shot Learning for 3D point clouds in autonomous vehicles

This paper presents a comprehensive study of 3D point cloud Federated Few-Shot Learning (3DFFL), focusing on addressing challenges such as limited data availability and privacy concerns in point cloud classification for applications such as autonomous vehicles. We introduce a novel approach that integrates Federated Learning with Few-Shot Learning techniques, with a special emphasis on optimizing network architectures for 3D point cloud data. Our method capitalizes on the strengths of PointNet++ for feature extraction and ProtoNet for classification, all within a federated learning framework to ensure data privacy and collaborative learning. Significantly, the approach is augmented with the use of attention and SoftMax layers, enhancing the feature extraction and classification processes. Extensive experiments on the ModelNet40, ShapeNet, and ScanOnjectNN datasets validate our method’s accuracy and adaptability in handling 3D point cloud classification, especially in privacy-sensitive and collaborative scenarios. This study not only demonstrates the potential of integrating attention mechanisms and SoftMax layers in 3DFFL but also lays a robust foundation for future advancements in this evolving field, particularly in technologies dependent on 3D data processing.

Long-range attention classification for substation point cloud

Point cloud classification for substation is challenging due to the occluding layout as well as the complexity of methods. Most previous methods trade inference speed for precision by building complex extractors. To alleviate the paradox of performance and complexity trade-off, this paper proposes the lightweight long-range attention classification for substation point cloud, including shuffle channel attention(SCA) and dilated channel attention(DCA). First, SCA captures local cross-channel interaction via 1D convolution and global interaction via channel shuffling, shown promising performance. Second, to further reduce the amount of computation involved in shuffling, we propose a more elegant method DCA by resizing the 1D vector after pooling into 2D feature map. Note that proposed DCA are implemented by just 2D convolution, determining the coverage of cross-channel interaction, with a significantly higher inference speed. Furthermore, we develop a heuristic algorithm to adaptively determine parameters like kernel size, shuffle group and size of 2D feature map. Besides ModelNet40 and ScanObjectNN(PB_T50_RS), this paper also selects ten of main objects in substation for training and testing. Finally, experimental results suggest that proposed methods bring notable performance gain, nearly 1%. DCA reaches 93.801% and just increases 0.0003M parameters but 15.5% faster than SCA. We also make an attempt to simplify proposed methods via reducing the coding dimensions and coding blocks. And simplified DCA achieves 93.4% performance with almost quarter parameters and 112% faster, guaranteeing both efficiency and effectiveness. A simplified version is used to achieve the highest accuracy of 85.115% and 83.304% on ScanObjectNN, with the highest improvements of 1.735% and 2.391%. This paper also conducts robustness test where different proportions of points are missing. Results show that when 93.75% missing, the accuracy only decreases by 4.093%, and can still reach 89%, which is significantly ahead of other methods.

Graph Transformer for 3D point clouds classification and semantic segmentation

Recently, graph-based and Transformer-based deep learning have demonstrated excellent performances on various point cloud tasks. Most of the existing graph-based methods rely on static graph, which take a fixed input to establish graph relations. Moreover, many graph-based methods apply maximizing and averaging to aggregate neighboring features, so that only a single neighboring point affects the feature of centroid or different neighboring points own the same influence on the centroid’s feature, which ignoring the correlation and difference between points. Most Transformer-based approaches extract point cloud features based on global attention and lack the feature learning on local neighbors. To solve the above issues of graph-based and Transformer-based models, we propose a new feature extraction block named Graph Transformer and construct a 3D point cloud learning network called GTNet to learn features of point clouds on local and global patterns. Graph Transformer integrates the advantages of graph-based and Transformer-based methods, and consists of Local Transformer that use intra-domain cross-attention and Global Transformer that use global self-attention. Finally, we use GTNet for shape classification, part segmentation and semantic segmentation tasks in this paper. The experimental results show that our model achieves good learning and prediction ability on most tasks. The source code and pre-trained model of GTNet will be released on https://github.com/NWUzhouwei/GTNet.

Granformer: A granular transformer net with linear complexity

Recently, transformer models have demonstrated excellent performance across various intelligent applications owing to their ability to understand global context through self-attention mechanism. However, the extensively investigated multiplicative-based attention mechanism is inadequate for capturing relationship-based feature representations, as the dot product cannot depict the intricate semantic information between objects. Moreover, it has a great computational burden with a complexity of O(n2), due to the global feature representation capability achieved by calculating the relationship between each token within the entire feature sequence. To solve the current problem, this paper proposes a granular transformer framework with linear complexity, wherein diverse granulation functions can be employed to supersede the prevailing multiplicative relationships, and an innovative linearization methodology in the form of matrix factorization is designed to reduce the computational burden. Relying on the intricate semantics information embedded within granular structures, the capacity for feature extraction is significantly more comprehensive. Then, a novel matrix factorization methodology is developed for the linearity of granulation-based attention, accomplished by implementing separate deformable convolution sampling and using an approximate iterative algorithm based on cubic equations to calculate the Moore–Penrose inverse. The mathematical proof that our method is approximate with the complete granulation-based attention matrix is investigated in detail. Finally, the performance of Granformer, an innovative reconfiguration of plug-and-play transformer block, is evaluated on representative intelligent applications, including 3D point cloud classification, emotion recognition and sentiment analysis, and object detection. The experimental results suggest that our methodologies outperform the state-of-the-art models.

Compressed point cloud classification with point-based edge sampling

3D point cloud data, as an immersive detailed data source, has been increasingly used in numerous applications. To deal with the computational and storage challenges of this data, it needs to be compressed before transmission, storage, and processing, especially in real-time systems. Instead of decoding the compressed data stream and subsequently conducting downstream tasks on the decompressed data, analyzing point clouds directly in their compressed domain has attracted great interest. In this paper, we dive into the realm of compressed point cloud classification (CPCC), aiming to achieve high point cloud classification accuracy in a bitrate-saving way by ensuring the bit stream contains a high degree of representative information of the point cloud. Edge information is one of the most important and representative attributes of the point cloud because it can display the outlines or main shapes. However, extracting edge points or information from point cloud models is challenging due to their irregularity and sparsity. To address this challenge, we adopt an advanced edge-sampling method that enhances existing state-of-the-art (SOTA) point cloud edge-sampling techniques based on attention mechanisms and consequently develop a novel CPCC method “CPCC-PES” that focuses on point cloud’s edge information. The result obtained on the benchmark ModelNet40 dataset shows that our model has superior rate-accuracy trade-off performance than SOTA works. Specifically, our method achieves over 90% Top-1 Accuracy with a mere 0.08 bits-per-point (bpp), marking a remarkable over 96% reduction in BD-bitrate compared with specialized codecs. This means that our method only consumes 20% of the bitrate of other SOTA works while maintaining comparable accuracy. Furthermore, we propose a new evaluation metric named BD-Top-1 Accuracy to evaluate the trade-off performance between bitrate and Top-1 Accuracy for future CPCC research.

Operational application of Fengyun geostationary meteorological satellites to cloud observation products

Cloud products from geostationary satellites are the main alternative to surface synoptic cloud observations (SYNOP), and have become the baseline products for the development and construction of the Quality Management System (QMS) of integrated meteorological observation in China. This study addresses the needs of the China Meteorological Administration (CMA) for such an operational reform, and it is carried out using real-time observations obtained from the Fengyun-2E (FY-2E) geostationary satellite in 2012 to derive cloud total amount and classification using two different methods. Compared to surface SYNOP observations, the cloud total amount estimated by FY-2E is generally significantly lower (about 30% lower on average). The cloud classification resulting from the two methods used in this study is also significantly different from the classification obtained from surface observations (difference between 22 and 32%). The difference is smaller for the classification method, which uses additional auxiliary temperature profiles.

Push-and-Pull: A General Training Framework With Differential Augmentor for Domain Generalized Point Cloud Classification

Push-and-Pull: A General Training Framework With Differential Augmentor for Domain Generalized Point Cloud Classification

Longwave Radiative Feedback Due To Stratiform and Anvil Clouds

AbstractStudies have implicated the importance of longwave (LW) cloud‐radiative forcing (CRF) in facilitating or accelerating the upscale development of tropical moist convection. While different cloud types are known to have distinct CRF, their individual roles in driving upscale development through radiative feedback is largely unexplored. Here we examine the hypothesis that CRF from stratiform regions has the greatest positive effect on upscale development of tropical convection. We do so through numerical model experiments using convection‐permitting ensemble WRF (Weather Research and Forecasting) simulations of tropical cyclone formation. Using a new column‐by‐column cloud classification scheme, we identify the contributions of five cloud types (shallow, congestus, and deep convective; and stratiform and anvil clouds). We examine their relative impacts on longwave radiation moist static energy (MSE) variance feedback and test the removal of this forcing in additional mechanism‐denial simulations. Our results indicate the importance stratiform and anvil regions in accelerating convective upscale development.