3D Point Cloud Research Articles

Breaking barriers in 3D point cloud data processing: A unified system for efficient storage and high-throughput loading

Breaking barriers in 3D point cloud data processing: A unified system for efficient storage and high-throughput loading

Cross-modal knowledge transfer for 3D point clouds via graph offset prediction

Cross-modal knowledge transfer for 3D point clouds via graph offset prediction

ChatIOS: Improving automatic 3-dimensional tooth segmentation via GPT-4V and multimodal pre-training.

ChatIOS: Improving automatic 3-dimensional tooth segmentation via GPT-4V and multimodal pre-training.

Combining SfM and deep learning to construct 3D point cloud models of shield tunnels and Realize spatial localization of water leakages

Combining SfM and deep learning to construct 3D point cloud models of shield tunnels and Realize spatial localization of water leakages

Comprehensive review on 3D point cloud segmentation in plants

Comprehensive review on 3D point cloud segmentation in plants

3D location estimation and tunnel mapping of autonomous driving robots through 3D point cloud registration on underground mine rampways

3D location estimation and tunnel mapping of autonomous driving robots through 3D point cloud registration on underground mine rampways

LCASAFormer: Cross-attention enhanced backbone network for 3D point cloud tasks

LCASAFormer: Cross-attention enhanced backbone network for 3D point cloud tasks

Automatic segmentation and measurement system of 3D point cloud images based on RGB-D camera for rat wounds

Automatic segmentation and measurement system of 3D point cloud images based on RGB-D camera for rat wounds

Modification of an automated precision farming robot for high temporal resolution measurement of leaf angle dynamics using stereo vision.

Modification of an automated precision farming robot for high temporal resolution measurement of leaf angle dynamics using stereo vision.

Exploring high-contrast areas context for 3D point cloud segmentation via MLP-driven Discrepancy mechanism

Exploring high-contrast areas context for 3D point cloud segmentation via MLP-driven Discrepancy mechanism

Infra-3DRC-FusionNet: Deep Fusion of Roadside Mounted RGB Mono Camera and Three-Dimensional Automotive Radar for Traffic User Detection

Mono RGB cameras and automotive radar sensors provide a complementary information set that makes them excellent candidates for sensor data fusion to obtain robust traffic user detection. This has been widely used in the vehicle domain and recently introduced in roadside-mounted smart infrastructure-based road user detection. However, the performance of the most commonly used late fusion methods often degrades when the camera fails to detect road users in adverse environmental conditions. The solution is to fuse the data using deep neural networks at the early stage of the fusion pipeline to use the complete data provided by both sensors. Research has been carried out in this area, but is limited to vehicle-based sensor setups. Hence, this work proposes a novel deep neural network to jointly fuse RGB mono-camera images and 3D automotive radar point cloud data to obtain enhanced traffic user detection for the roadside-mounted smart infrastructure setup. Projected radar points are first used to generate anchors in image regions with a high likelihood of road users, including areas not visible to the camera. These anchors guide the prediction of 2D bounding boxes, object categories, and confidence scores. Valid detections are then used to segment radar points by instance, and the results are post-processed to produce final road user detections in the ground plane. The trained model is evaluated for different light and weather conditions using ground truth data from a lidar sensor. It provides a precision of 92%, recall of 78%, and F1-score of 85%. The proposed deep fusion methodology has 33%, 6%, and 21% absolute improvement in precision, recall, and F1-score, respectively, compared to object-level spatial fusion output.

Evolution of Rockfall Based on Structure from Motion Reconstruction of Street View Imagery and Unmanned Aerial Vehicle Data: Case Study from Koto Panjang, Indonesia

This study explores the growing application of 3D remote sensing in geohazard studies, particularly for rock slope monitoring. It highlights the use of cost-effective Street View Imagery (SVI) and Unmanned Aerial Vehicles (UAV) through Structure-from-Motion (SfM) photogrammetry as tools for 3D rockfall monitoring. Using multi-temporal SVI and UAV Imagery from the Koto Panjang cliff in Indonesia, we quantify rockfall volume changes over seven years and assess associated geohazards. The results reveal a total rockfall retreat of 5270 m3, with an average annual rate of 7.53 m3/year. Structural analysis identified six major discontinuity sets and confirmed inherent instability within the rock mass. Kinematic simulations using SVI and UAV-derived data further assessed rockfall trajectories and potential impact zones. Results indicate that 40% of simulated rockfall deposits accumulated near existing roads, with significant differences in distribution based on scree slope angles. This emphasizes the role of scree slope in influencing rockfall propagation. In conclusion, SVI and UAV imagery presents a valuable tool for 3D point cloud reconstruction and rockfall hazard assessment, particularly in areas lacking historical data. The study showcases the effectiveness of using SVI and UAV imagery in quantifying historical past rockfall volume and identifies critical areas for mitigation strategies, highlighting the importance of scree slope angle in managing rockfall hazard.

Optimizing Camera Settings and Unmanned Aerial Vehicle Flight Methods for Imagery-Based 3D Reconstruction: Applications in Outcrop and Underground Rock Faces

The structure from motion (SfM) and multiview stereo (MVS) techniques have proven effective in generating high-quality 3D point clouds, particularly when integrated with unmanned aerial vehicles (UAVs). However, the impact of image quality—a critical factor for SfM–MVS techniques—has received limited attention. This study proposes a method for optimizing camera settings and UAV flight methods to minimize point cloud errors under illumination and time constraints. The effectiveness of the optimized settings was validated by comparing point clouds generated under these conditions with those obtained using arbitrary settings. The evaluation involved measuring point-to-point error levels for an indoor target and analyzing the standard deviation of cloud-to-mesh (C2M) and multiscale model-to-model cloud comparison (M3C2) distances across six joint planes of a rock mass outcrop in Seoul, Republic of Korea. The results showed that optimal settings improved accuracy without requiring additional lighting or extended survey time. Furthermore, we assessed the performance of SfM–MVS under optimized settings in an underground tunnel in Yeoju-si, Republic of Korea, comparing the resulting 3D models with those generated using Light Detection and Ranging (LiDAR). Despite challenging lighting conditions and time constraints, the results suggest that SfM–MVS with optimized settings has the potential to produce 3D models with higher accuracy and resolution at a lower cost than LiDAR in such environments.

From Trees to Data: A Standardised Approach to Forest Monitoring

Forest ecosystems provide a wide range of services, including timber production, biodiversity conservation, protection against natural hazards, and human recreation. However, they are increasingly under pressure from direct human interventions and climate change. Strengthening forest resilience to changing environmental conditions requires appropriate adaptation measures, which in turn depend on a deep understanding of near-natural forest structures and functions. High-resolution in situ data, both temporally and spatially, are essential for understanding and effective planning and management. This presentation introduces a potential eLTER service offering standardised terrestrial data collection, good practice guides and white papers to assess high resolution 3D structural forest ecosystem properties. Recent advances in proximate sensing like LiDAR and photogrammetry enable high-precision, three-dimensional forest reconstructions at unprecedented levels of detail. Among these, terrestrial laser scanning (TLS) has become the gold standard for generating very high-resolution, accurate 3D point clouds in forest environments. With improvements in portability and usability, TLS technology is now more accessible for widespread application. Mobile laser scanning (MLS), including handheld or backpack-mounted personal laser scanning (PLS) systems, has recently emerged as a much faster alternative, capable of mapping a view hectares per day. However, this speed comes at the expense of reduced precision compared to TLS. For even larger-scale mapping, unmanned aerial vehicle laser scanning (UAVLS) is increasingly used to assess high resolution 3D forest structures across entire landscapes. Additionally, UAV-based and terrestrial photogrammetry based on spectral data provide detailed 3D structural information from overlapping images, while also enabling functional trait assessment through spectral data analysis. A variety of sensors and methods are now available to capture detailed structural and functional information on trees and forests. These data acquisition techniques, along with their derived products, can significantly enhance eLTER standard observations by improving information on e.g. vegetation structure, biomass, energy flows and light availability. Furthermore, such standardised and scalable approaches can add value to various satellite missions by supporting the calibration and validation of satellite-derived data products. In this presentation, we will outline the potential data products, documents and guidelines of this proposed service, discuss the benefits for different eLTER user groups, and explore its role in improving satellite data calibration and validation.

DSF-Net: semantic segmentation of large-scale point clouds based on integrating deep and shallow networks

PurposeWith the upgrading of three-dimensional (3D) sensing devices, the amount of point cloud data collected has also increased exponentially. However, most of the existing methods also have unbalanced optimizations in memory consumption and semantic segmentation efficiency. This research addresses the need for a more balanced approach in processing large-scale point cloud data efficiently.Design/methodology/approachThis research used a network framework (DSF-Net) based on dual-path deep and shallow networks and designed a point cloud space pyramid pooling module based on hole convolution. The 3D point cloud data are trained separately by integrating the deep branch and shallow branch networks. Besides, a deep and shallow fusion module fuses the deep and shallow feature relationships and outputs several loss functions for convergence training.FindingsIt is found that DSF-Net solves the problem of segmentation efficiency, achieves a balanced effect while ensuring the ability of a large range of point cloud input and reduces the memory consumption.Originality/valueThe deep network can extract high-level semantic information, while the shallow neural network has fewer neural network layers and faster inference speed. Meanwhile, random sampling and point-atrous spatial pyramid pool modules are used, respectively, for deep and shallow networks to capture multi-scale local context information in point cloud.

Geometrically aware transformer for point cloud analysis

With the increasing use of 3D point cloud data in autonomous driving, robotic perception, and remote sensing, efficient and accurate point cloud analysis remains a critical challenge. This study presents PointGA, a lightweight Transformer-based model that enhances geometric perception for improved feature extraction and representation. First, PointGA expands the original 3D coordinates into various geometric information, introducing more prior knowledge into the network. Second, a trigonometric position encoding suitable for point clouds is designed, which effectively enhances the expressive capability of positional information and performs preliminary feature extraction through pooling layers, significantly improving the model’s robustness across various tasks. Finally, a positional differential self-attention (PDA) mechanism with linear complexity is developed to optimize feature representation and achieve efficient computation. Experimental results demonstrate that PointGA achieves 87.6% overall accuracy on the ScanObjectNN dataset for classification and 66.2% mean intersection over union(mIoU) on the S3DIS Area 5 dataset for segmentation, outperforming existing methods. These findings highlight the model’s capability to balance efficiency and accuracy, offering a promising solution for point cloud analysis tasks.

Image-based laser point cloud building facade structure extraction method by considering semantic information

Building facade structures form the foundation for 3D model reconstructions, making the extraction of facade structures from 3D point clouds a key research area. A method for extracting the building facade structure from image-based laser point clouds by considering semantic information is proposed. First, point cloud segmentation and clustering are applied to organize the data into distinct planes. Second, semantic images and corresponding semantic image laser point cloud models are generated from each plane. Finally, an enhanced method named as SemColorED extracts the facade structures, and followed by optimization based on building morphology. Evaluation of the method using actual 3D laser point cloud data and the Semantic3D dataset shows improved accuracy, recall, and integrity compared to the current methods.

Fusion of feature vectors for enhanced transformer-based applied to 3D point cloud classification

Fusion of feature vectors for enhanced transformer-based applied to 3D point cloud classification

Extraction of individual tree attributes using ultra-high-density point clouds acquired by low-cost UAV-LiDAR in Eucalyptus plantations

Key messageIn this paper, we first introduced a novel method for directly measuring tree diameters from UAV-LiDAR point clouds utilizing the χ2-filtering technique and a technique for measuring tree heights using pseudo-waveforms.ContextEucalyptus plantation forests constitute the largest expanse of planted broad-leaved forests worldwide. Detailed and accurate individual tree attributes are essential for precision forestry. Terrestrial laser scanning (TLS) and mobile laser scanning (MLS) are frequently employed to acquire information on individual trees. However, both technologies suffer from low efficiency. Therefore, the challenge remains how to access this information efficiently.AimsConsequently, this paper investigated a novel technical approach to automatically extract individual tree attributes using low-cost UAV-LiDAR technology.MethodsThe framework consists of three independent yet interrelated approaches. Firstly, the tree trunks were detected using an approach based on the hierarchical density-based spatial clustering of applications with noise (HDBSCAN) algorithm. It utilized 3D point clouds to achieve precise tree counts and their approximate locations. These locations then enabled cylindrical segmentation of the point clouds at the trunk level, facilitating diameter measurement. Secondly, stem diameters were directly measured using the probability density function of the chi-square distribution. This process produced precise stem diameters, trunk positions, and growth directions, which were subsequently used to determine the center of the crown top for tree height extraction. Lastly, the tree height was estimated based on the pseudo-waveforms. We validated this framework by acquiring ultra-high-density UAV-LiDAR data in an Eucalyptus plantation.ResultsThe result indicated a precision of 91.1% for individual tree detection, with an F-score of 0.916. The root mean square errors (RMSEs) for direct measurements of diameter at breast height (DBH) and tree height were 14.60% (2.18 cm) and 2.69% (0.31 m), respectively. Furthermore, this study suggested that the classical circle-fitting method might not be suitable for directly measuring tree diameter using low-cost UAV-LiDAR data.ConclusionThe proposed framework facilitates automated inventory and monitoring in Eucalyptus plantation forests. However, more trials are needed to verify the framework’s applicability in other planted and natural forests.

Metric Scaling and Extrinsic Calibration of Monocular Neural Network-Derived 3D Point Clouds in Railway Applications

Three-dimensional reconstruction using monocular camera images is a well-established research topic. While multi-image approaches like Structure from Motion produce sparse point clouds, single-image depth estimation via machine learning promises denser results. However, many models estimate relative depth, and even those providing metric depth often struggle with unseen data due to unfamiliar camera parameters or domain-specific challenges. Accurate metric 3D reconstruction is critical for railway applications, such as ensuring structural gauge clearance from vegetation to meet legal requirements. We propose a novel method to scale 3D point clouds using the track gauge, which typically only varies in very limited values between large areas or countries worldwide (e.g., 1.435 m in Europe). Our approach leverages state-of-the-art image segmentation to detect rails and measure the track gauge from a train driver’s perspective. Additionally, we extend our method to estimate a reasonable railway-specific extrinsic camera calibration. Evaluations show that our method reduces the average Chamfer distance to LiDAR point clouds from 1.94 m (benchmark UniDepth) to 0.41 m for image-wise calibration and 0.71 m for average calibration.