3D Point Cloud Research Articles

Automatic identification of discontinuities and refined modeling of rock blocks from 3D point cloud data of rock surfaces

Automatic identification of discontinuities and refined modeling of rock blocks from 3D point cloud data of rock surfaces

Defect focused Harris3D & boundary fine-tuning optimized region growing: Lithium battery pole piece defect segmentation

As a vital component of lithium battery, the pole piece is prone to defects during the production process, which adversely affects performance and even causes safety accidents. Therefore, conducting quality inspections for lithium battery pole pieces is crucial. In this research, an optimized region growing algorithm based on 3D point cloud data is proposed to automatically detect pole piece defects. Specifically, a potential defect-focused Harris3D method (DFHarris3D) for keypoint detection and a boundary fine-tuning algorithm (BFT) are introduced. Comparative experimental results prove that our method outperforms others in terms of completeness, accuracy, and robustness. Therefore, this novel non-destructive detection technique demonstrates the potential to accurately segment surface defects in lithium battery pole pieces, which can be integrated into smart factories for fully automated defect detection.

Pothole formation on Rock Window Mesa, Chinle valley, AZ

In areas of the Colorado Plateau, rock basins, or potholes, occur in abundance. Because of their impressive size and often enigmatic location, their presence begs an explanation. Our goal in undertaking this research is to suggest a model for the origin of Rock Window Mesa's potholes, by which we might better interpret the geomorphological history of the mesa and Chinle Valley. Data collection consisted of both field work and analysis of a high-resolution 3D point cloud of the mesa. The southern and western edges of the mesa, closest to the Chinle Wash, and close to where knickpoints may have been, are the areas of highest pothole density. The pothole-forming processes disrupted drainage systems, and pothole sizes do not correlate with the catchment areas of the potholes. There are abundant, fluvially deposited erratic pebbles and cobbles on the mesa, and the joints and cracks that are present in the bedrock exert little to no control over pothole morphology. These data suggest that the potholes were formed subaqueously. A proposal that may point us to a possible mechanism is that similar potholes, in the Navajo Sandstone exposed in the Henry Mountains, appear to have been formed via fluvial processes in over-steepened tributary junctions. At our research location, the water that carved out the potholes appears to have come from ENE, and the point at which this water met the Chinle Wash may have been over-steepened as a result of rapid incision of the Chinle Wash through the poorly lithified Navajo Sandstone.

Research on the Identification of Rock Mass Structural Planes and Extraction of Dominant Orientations Based on 3D Point Cloud

The different spatial distribution forms of rock mass structural planes create weak zones in the rock mass, which is also a key factor in controlling rock mass stability. Accurately and efficiently identifying rock mass structural planes and obtaining their dominant orientations is critical for rock mass engineering design and construction. Traditional surveying methods for high and steep rock mass structural planes pose high safety risks, offer limited data, and make comprehensive statistical analysis difficult. This paper utilizes complex rock mass surface 3D point cloud data obtained through 3D laser scanning technology and uses the Hough space transform method to calculate the normal vectors of the 3D point cloud. Based on the difference in normal vectors and surface variation, region growing segmentation is applied to identify and extract rock mass structural planes. Additionally, the fast search and density peak clustering method (CFSFDP) is used for clustering analysis of the rock mass structural planes to obtain dominant orientations. This method was applied to a highway’s high and steep rock slope, successfully identifying 281 structural planes and two sets of dominant structural planes. The orientation of the dominant structural planes identified through RocScience Dips 7.0 analysis showed a deviation of no more than ±3°, complying with engineering standards. The research results offer a feasible solution for the identification of high and steep rock mass structural planes and the extraction of the orientation of dominant structural planes.

A Novel Panorama Depth Estimation Framework for Autonomous Driving Scenarios Based on a Vision Transformer.

An accurate panorama depth estimation result is crucial to risk perception in autonomous driving practice. In this paper, an innovative framework is presented to address the challenges of imperfect observation and projection fusion in panorama depth estimation, enabling the accurate capture of distances from surrounding images in driving scenarios. First, the Patch Filling method is proposed to alleviate the imperfect observation of panoramic depth in autonomous driving scenarios, which constructs a panoramic depth map based on the sparse distance data provided by the 3D point cloud. Then, in order to tackle the distortion challenge faced by outdoor panoramic images, a method for image context learning, ViT-Fuse, is proposed and specifically designed for equirectangular panoramic views. The experimental results show that the proposed ViT-Fuse reduces the estimation error by 9.15% on average in driving scenarios compared with the basic method and exhibits more robust and smoother results on the edge details of the depth estimation maps.

Research on the Laser Scattering Characteristics of Three-Dimensional Imaging Based on Electro–Optical Crystal Modulation

In this paper, we construct a laser 3D imaging simulation model based on the 3D imaging principle of electro–optical crystal modulation. Unlike the traditional 3D imaging simulation method, this paper focuses on the laser scattering characteristics of the target scene. To accurately analyze and simulate the scattering characteristic model of the target under laser irradiation, we propose a BRDF (Bidirectional Reflectance Distribution Function) model fitting algorithm based on the hybrid BBO–Firefly model, which can accurately simulate the laser scattering distribution of the target at different angles. Finally, according to the fitted scattering characteristic model, we inverted the target imaging gray map. We used the laser 3D imaging restoration principle to reconstruct the 3D point cloud of the target to realize the laser 3D imaging of the target.

Immersive remote telerobotics: foveated unicasting and remote visualization for intuitive interaction

Abstract Precise and efficient performance in remote robotic teleoperation relies on intuitive interaction. This requires both accurate control actions and complete perception (vision, haptic, and other sensory feedback) of the remote environment. Especially in immersive remote teleoperation, the complete perception of remote environments in 3D allows operators to gain improved situational awareness. Color and Depth (RGB-D) cameras capture remote environments as dense 3D point clouds for real-time visualization. However, providing enough situational awareness needs fast, high-quality data transmission from acquisition to virtual reality rendering. Unfortunately, dense point-cloud data can suffer from network delays and limits, impacting the teleoperator’s situational awareness. Understanding how the human eye works can help mitigate these challenges. This paper introduces a solution by implementing foveation, mimicking the human eye’s focus by smartly sampling and rendering dense point clouds for an intuitive remote teleoperation interface. This provides high resolution in the user’s central field, which gradually reduces toward the edges. However, this systematic visualization approach in the peripheral vision may benefit or risk losing information and burdening the user’s cognitive load. This work investigates these advantages and drawbacks through an experimental study and describes the overall system, with its software, hardware, and communication framework. This will show significant enhancements in both latency and throughput, surpassing 60% and 40% improvements in both aspects when compared with state-of-the-art research works. A user study reveals that the framework has minimal impact on the user’s visual quality of experience while helping to reduce the error rate significantly. Further, a 50% reduction in task execution time highlights the benefits of the proposed framework in immersive remote telerobotics applications.

MSSA: Multi-Representation Semantics-Augmented Set Abstraction for 3D Object Detection

Accurate recognition and localization of 3D objects is a fundamental research problem in 3D computer vision. Benefiting from transformation-free point cloud processing and flexible receptive fields, point-based methods have become accurate in 3D point cloud modeling, but still fall behind voxel-based competitors in 3D detection. We observe that the set abstraction module, commonly utilized by point-based methods for downsampling points, tends to retain excessive irrelevant background information, thus hindering the effective learning of features for object detection tasks. To address this issue, we propose MSSA, a Multi-representation Semantics-augmented Set Abstraction for 3D object detection. Specifically, we first design a backbone network to encode different representation features of point clouds, which extracts point-wise features through PointNet to preserve fine-grained geometric structure features, and adopts VoxelNet to extract voxel features and BEV features to enhance the semantic features of key points. Second, to efficiently fuse different representation features of keypoints, we propose a Point feature-guided Voxel feature and BEV feature fusion (PVB-Fusion) module to adaptively fuse multi-representation features and remove noise. At last, a novel Multi-representation Semantic-guided Farthest Point Sampling (MS-FPS) algorithm is designed to help set abstraction modules progressively downsample point clouds, thereby improving instance recall and detection performance with more important foreground points. We evaluate MSSA on the widely used KITTI dataset and the more challenging nuScenes dataset. Experimental results show that compared to PointRCNN, our method improves the AP of “moderate” level for three classes of objects by 7.02%, 6.76%, and 5.44%, respectively. Compared to the advanced point-voxel-based method PV-RCNN, our method improves the AP of “moderate” level by 1.23%, 2.84%, and 0.55% for the three classes, respectively.

Graph-based robust 3D point cloud map merging approach for large scale

Graph-based robust 3D point cloud map merging approach for large scale

A Calculating Method for the Height of Multi-Type Buildings Based on 3D Point Cloud

Building height is a critical variable in urban studies, and the automated acquisition of the precise building height is essential for intelligent construction, safety, and the sustainable development of cities. The building height is often approximated by the building’s highest point. However, the calculation method of the building height of the various roof types differs according to building codes, making it challenging to accurately calculate the height of buildings with complex roof structures or multiple upper appendages. Consequently, this paper utilizes point clouds to propose an automated method for calculating building heights conforming to design codes. The model considers roof types and allows for fast, automated, and highly accurate building height estimation. First, roofs are extracted from the point cloud by combining normal vector density clustering with a region-growing algorithm. Second, combined with variational Bayes, a Gaussian mixture model is employed to segment the roof surfaces. Finally, roofs are classified based on slope characteristics, achieving the automatic acquisition of building heights for various roof types over large areas. Experiments were conducted on Vaihingen and STPLS3D datasets. In the Vaihingen area, the maximum error, root-mean-square-error (RMSE), and mean absolute error (MAE) of the measured heights are 1.92 cm, 1.18 cm, and 1.13 cm, respectively. In the STPLS3D area, these values are 1.79 cm, 0.82 cm, and 0.68 cm, respectively. The results demonstrate that the proposed method is reliable and effective, which offers valuable data for the development, construction, and planning of three-dimensional (3D) cities.

Comprehensive Review of Tunnel Blasting Evaluation Techniques and Innovative Half Porosity Assessment Using 3D Image Reconstruction

To meet the increasing demand for rapid and efficient evaluation of tunnel blasting quality, this study presents To meet the increasing demand for rapid and efficient evaluation of tunnel blasting quality, this study presents a comprehensive review of the current state of the art in tunnel blasting evaluation, organized into five key areas: Blasting Techniques and Optimization, 3D Reconstruction and Visualization, Monitoring and Assessment Technologies, Automation and Advanced Techniques, and Half Porosity in Tunnel Blasting. Each section provides an indepth analysis of the latest research and developments, offering insights into enhancing blasting efficiency, improving safety, and optimizing tunnel design. Building on this foundation, we introduce a digital identification method for assessing half porosity through 3D image reconstruction. Utilizing the Structure from Motion (SFM) technique, we re-construct the 3D contours of tunnel surfaces and bench faces after blasting. Curvature values are employed as key indicators for extracting 3D point cloud data from boreholes. The acquired postblasting point cloud data is processed using advanced software that incorporates the RANSAC algorithm to accurately project and fit the borehole data, leading to the determination of the target circle and borehole axis. The characteristics of the boreholes are analyzed based on the fitting results, culminating in the calculation of half porosity. Field experiments conducted on the Huangtai Tunnel (AK20 + 970.5 to AK25 + 434), part of the new National Highway 109 project, provided data from shell holes generated during blasting. These data were analyzed and compared with traditional onsite measurements to validate the proposed method’s effectiveness. The computed half porosity value using this technique was 58.7%, showing minimal deviation from the traditional measurement of 60%. This methodology offers significant advantages over conventional measurement techniques, including easier equipment acquisition, non-interference with construction activities, a comprehensive detection range, rapid processing speed, reduced costs, and improved accuracy. The findings demonstrate the method’s potential for broader application in tunnel blasting assessments.

3D coherent single shot lidar imaging beyond coherence length

Advancements in remote sensing and autonomous vehicle technologies made lidars equally important for unmanned objects alongside cameras. Therefore, precise 3D lidar imaging and point cloud generation have become important subjects. Although existing coherent lidar technologies provide precise imaging results, the spectral linewidth of the laser sources becomes a key limitation over long distances as it defines the maximum detection range. Here, we present long-distance 3D lidar imaging which removes the coherence length limitations and therefore the necessity of high-coherence laser sources. Mainly, we generate optical sidebands, by modulating a continuous wave (CW) laser source with multiple radio-frequency (RF) tones. Then, using our own post-processing and triangulation methods, we use the relative phase changes between the sidebands which are free from laser phase noise to determine the target distance. We prove that the multi-tone coherent Lidar technique can perform precise 3D imaging and point cloud generation of various targets at sub-10pW optical power reception and distances up to ∼12× beyond the coherence length of the CW laser employed in the lidar architecture. Overall, it is demonstrated that coherence length restriction is removed by the suggested method, which makes precise long-distance 3D lidar imaging possible, particularly for applications such as spacecraft and aerial coherent lidars.

Comprehensive Analysis of Phenotypic Traits in Chinese Cabbage Using 3D Point Cloud Technology

Studies on the phenotypic traits and their associations in Chinese cabbage lack precise and objective digital evaluation metrics. Traditional assessment methods often rely on subjective evaluations and experience, compromising accuracy and reliability. This study develops an innovative, comprehensive trait evaluation method based on 3D point cloud technology, with the aim of enhancing the precision, reliability, and standardization of the comprehensive phenotypic traits of Chinese cabbage. By using multi-view image sequences and structure-from-motion algorithms, 3D point clouds of 50 plants from each of the 17 Chinese cabbage varieties were reconstructed. Color-based region growing and 3D convex hull techniques were employed to measure 30 agronomic traits. Comparisons between 3D point cloud-based measurements of the plant spread, plant height, leaf area, and leaf ball volume and traditional methods yielded R2 values greater than 0.97, with root mean square errors of 1.27 cm, 1.16 cm, 839.77 cm3, and 59.15 cm2, respectively. Based on the plant spread and plant height, a linear regression prediction of Chinese cabbage weights was conducted, yielding an R2 value of 0.76. Integrated optimization algorithms were used to test the parameters, reducing the measurement time from 55 min when using traditional methods to 3.2 min. Furthermore, in-depth analyses including variation, correlation, principal component analysis, and clustering analyses were conducted. Variation analysis revealed significant trait variability, with correlation analysis indicating 21 pairs of traits with highly significant positive correlations and 2 pairs with highly significant negative correlations. The top six principal components accounted for 90% of the total variance. Using the elbow method, k-means clustering determined that the optimal number of clusters was four, thus classifying the 17 cabbage varieties into four distinct groups. This study provides new theoretical and methodological insights for exploring phenotypic trait associations in Chinese cabbage and facilitates the breeding and identification of high-quality varieties. Compared with traditional methods, this system provides significant advantages in terms of accuracy, speed, and comprehensiveness, with its low cost and ease of use making it an ideal replacement for manual methods, being particularly suited for large-scale monitoring and high-throughput phenotyping.

Airborne LiDAR Point Cloud Classification Using Ensemble Learning for DEM Generation.

Airborne laser scanning (ALS) point clouds have emerged as a predominant data source for the generation of digital elevation models (DEM) in recent years. Traditionally, the generation of DEM using ALS point clouds involves the steps of point cloud classification or ground point filtering to extract ground points and labor-intensive post-processing to correct the misclassified ground points. The current deep learning techniques leverage the ability of geometric recognition for ground point classification. However, the deep learning classifiers are generally trained using 3D point clouds with simple geometric terrains, which decrease the performance of model inferencing. In this study, a point-based deep learning model with boosting ensemble learning and a set of geometric features as the model inputs is proposed. With the ensemble learning strategy, this study integrates specialized ground point classifiers designed for different terrains to boost classification robustness and accuracy. In experiments, ALS point clouds containing various terrains were used to evaluate the feasibility of the proposed method. The results demonstrated that the proposed method can improve the point cloud classification and the quality of generated DEMs. The classification accuracy and F1 score are improved from 80.9% to 92.2%, and 82.2% to 94.2%, respectively, by using the proposed methods. In addition, the DEM generation error, in terms of mean squared error (RMSE), is reduced from 0.318-1.362 m to 0.273-1.032 m by using the proposed ensemble learning.

A Unified Virtual Model for Real-Time Visualization and Diagnosis in Architectural Heritage Conservation

The aim of this paper is to propose a workflow for the real-time visualization of virtual environments that supports diagnostic tasks in heritage buildings. The approach integrates data from terrestrial laser scanning (3D point clouds and meshes), along with panoramic and thermal images, into a unified virtual model. Additionally, the methodology incorporates several post-processing stages designed to enhance the user experience in visualizing both the building and its associated damage. The methodology was tested on the Medieval Templar Church of Vera Cruz in Segovia, utilizing a combination of visible and infrared data, along with manually prepared damage maps. The project results demonstrate that the use of a hybrid digital model—combining 3D point clouds, polygonal meshes, and panoramic images—is highly effective for real-time rendering, providing detailed visualization while maintaining adaptability for mobile devices with limited computational power.

Stereo and LiDAR Loosely Coupled SLAM Constrained Ground Detection.

In many robotic applications, creating a map is crucial, and 3D maps provide a method for estimating the positions of other objects or obstacles. Most of the previous research processes 3D point clouds through projection-based or voxel-based models, but both approaches have certain limitations. This paper proposes a hybrid localization and mapping method using stereo vision and LiDAR. Unlike the traditional single-sensor systems, we construct a pose optimization model by matching ground information between LiDAR maps and visual images. We use stereo vision to extract ground information and fuse it with LiDAR tensor voting data to establish coplanarity constraints. Pose optimization is achieved through a graph-based optimization algorithm and a local window optimization method. The proposed method is evaluated using the KITTI dataset and compared against the ORB-SLAM3, F-LOAM, LOAM, and LeGO-LOAM methods. Additionally, we generate 3D point cloud maps for the corresponding sequences and high-definition point cloud maps of the streets in sequence 00. The experimental results demonstrate significant improvements in trajectory accuracy and robustness, enabling the construction of clear, dense 3D maps.

Adversarial Unsupervised Domain Adaptation for 3D Semantic Segmentation with 2D Image Fusion of Dense Depth

AbstractUnsupervised domain adaptation (UDA) is increasingly used for 3D point cloud semantic segmentation tasks due to its ability to address the issue of missing labels for new domains. However, most existing unsupervised domain adaptation methods focus only on uni‐modal data and are rarely applied to multi‐modal data. Therefore, we propose a cross‐modal UDA on multi‐modal datasets that contain 3D point clouds and 2D images for 3D Semantic Segmentation. Specifically, we first propose a Dual discriminator‐based Domain Adaptation (Dd‐bDA) module to enhance the adaptability of different domains. Second, given that the robustness of depth information to domain shifts can provide more details for semantic segmentation, we further employ a Dense depth Feature Fusion (DdFF) module to extract image features with rich depth cues. We evaluate our model in four unsupervised domain adaptation scenarios, i.e., dataset‐to‐dataset (A2D2 → SemanticKITTI), Day‐to‐Night, country‐to‐country (USA → Singapore), and synthetic‐to‐real (VirtualKITTI → SemanticKITTI). In all settings, the experimental results achieve significant improvements and surpass state‐of‐the‐art models.

Underwater point cloud transmission framework: hybrid encoder implementation based on CNN and transformer

Abstract In underwater multi-robot systems, 3D point cloud data generated by sonar and depth sensors facilitates the execution of more complex collaborative tasks among robots, which partly rely on efficient data transmission. In this work, we propose a hybrid encoder framework based on convolutional neural network and Transformer for underwater point cloud transmission, aimed at dealing with large-scale point cloud data. Our encoder allows setting lower compression rate on regions or objects of interest for semantic point clouds, preserving crucial information in the point cloud. For underwater acoustic communication, we employ orthogonal frequency division multiplexing combined with deep joint source-channel coding for transmission to enhance the system’s error-resilience. Compared to state-of-the-art methods in the simulation experiments, our end-to-end framework achieves considerable compression performance while eliminating certain cliff and leveling effects, also demonstrating robustness even with changing channels.

DSGI‐Net: Density‐based Selective Grouping Point Cloud Learning Network for Indoor Scene

AbstractIndoor scene point clouds exhibit diverse distributions and varying levels of sparsity, characterized by more intricate geometry and occlusion compared to outdoor scenes or individual objects. Despite recent advancements in 3D point cloud analysis introducing various network architectures, there remains a lack of frameworks tailored to the unique attributes of indoor scenarios. To address this, we propose DSGI‐Net, a novel indoor scene point cloud learning network that can be integrated into existing models. The key innovation of this work is selectively grouping more informative neighbor points in sparse regions and promoting semantic consistency of the local area where different instances are in proximity but belong to distinct categories. Furthermore, our method encodes both semantic and spatial relationships between points in local regions to reduce the loss of local geometric details. Extensive experiments on the ScanNetv2, SUN RGB‐D, and S3DIS indoor scene benchmarks demonstrate that our method is straightforward yet effective.

Point‐AGM : Attention Guided Masked Auto‐Encoder for Joint Self‐supervised Learning on Point Clouds

AbstractMasked point modeling (MPM) has gained considerable attention in self‐supervised learning for 3D point clouds. While existing self‐supervised methods have progressed in learning from point clouds, we aim to address their limitation of capturing high‐level semantics through our novel attention‐guided masking framework, Point‐AGM. Our approach introduces an attention‐guided masking mechanism that selectively masks low‐attended regions, enabling the model to concentrate on reconstructing more critical areas and addressing the limitations of random and block masking strategies. Furthermore, we exploit the inherent advantages of the teacher‐student network to enable cross‐view contrastive learning on augmented dual‐view point clouds, enforcing consistency between complete and partially masked views of the same 3D shape in the feature space. This unified framework leverages the complementary strengths of masked point modeling, attention‐guided masking, and contrastive learning for robust representation learning. Extensive experiments have shown the effectiveness of our approach and its well‐transferable performance across various downstream tasks. Specifically, our model achieves an accuracy of 94.12% on ModelNet40 and 87.16% on the PB‐T50‐RS setting of ScanObjectNN, outperforming other self‐supervised learning methods.