Dense Point Clouds Research Articles

3D Point Cloud Fusion Method Based on EMD Auto-Evolution and Local Parametric Network

Although the development of high-resolution remote sensing satellite technology has made it possible to reconstruct the 3D structure of object-level features using satellite imagery, the results from a single reconstruction are often insufficient to comprehensively describe the 3D structure of the target. Therefore, developing an effective 3D point cloud fusion method can fully utilize information from multiple observations to improve the accuracy of 3D reconstruction. To this end, this paper addresses the problems of shape distortion and sparse point cloud density in existing 3D point cloud fusion methods by proposing a 3D point cloud fusion method based on Earth mover’s distance (EMD) auto-evolution and local parameterization network. Our method is divided into two stages. In the first stage, EMD is introduced as a key metric for evaluating the fusion results, and a point cloud fusion method based on EMD auto-evolution is constructed. The method uses an alternating iterative technique to sequentially update the variables and produce an initial fusion result. The second stage focuses on point cloud optimization by constructing a local parameterization network for the point cloud, mapping the upsampled point cloud in the 2D parameter domain back to the 3D space to complete the optimization. Through these two steps, the method achieves the fusion of two sets of non-uniform point cloud data obtained from satellite stereo images into a single, denser 3D point cloud that more closely resembles the true target shape. Experimental results demonstrate that our fusion method outperforms other classical comparison algorithms for targets such as buildings, planes, and ships, and achieves a fused RMSE of approximately 2 m and an EMD accuracy better than 0.5.

Enhancing environmental modeling and 3D point cloud construction for unmanned vehicles using laser SLAM and stereo vision with Convolutional Neural Networks

Abstract. This research presents an innovative SLAM algorithm that integrates Convolutional Neural Networks (CNNs) with LIDAR and stereo vision to significantly enhance the accuracy of environmental modeling and the construction of dense 3D point cloud maps in complex and dynamic surroundings. By processing pre-recorded video data and employing advanced image segmentation techniques, this study achieves a deep fusion of visual and geometric data, resulting in highly detailed and precise 3D representations of the environment. The experimental results demonstrate that this approach effectively detects and excludes dynamic objects, thereby significantly improving the overall quality, robustness, and reliability of the constructed maps. This work represents a substantial advancement in SLAM technology, particularly in its capability to model and capture intricate environmental details under varying conditions, making it a powerful tool for precise environmental mapping.

Evaluation of Two-Dimensional DBH Estimation Algorithms Using TLS

Terrestrial laser scanning (TLS) has become a vital tool in forestry for accurately measuring tree parameters, such as diameter at breast height (DBH). However, its application in Mexican forests remains underexplored. This study evaluates the performance of five two-dimensional DBH estimation algorithms (Nelder–Mead, least squares, Hough transform, RANSAC, and convex hull) within a temperate Mexican forest and explores their broader applicability across diverse ecosystems, using published point cloud data from various scanning devices. Results indicate that algorithm accuracy is influenced by local factors like point cloud density, occlusion, vegetation, and tree structure. In the Mexican study area, the Nelder–Mead algorithm achieved the highest accuracy (R² = 0.98, RMSE = 1.59 cm, MAPE = 6.12%), closely followed by least squares (R² = 0.98, RMSE = 1.67 cm, MAPE = 6.42%), with different outcomes in other sites. These findings advance DBH estimation methods by highlighting the importance of tailored algorithm selection and environmental considerations, thereby contributing to more accurate and efficient forest management across various landscapes.

Low-Cost and Contactless Survey Technique for Rapid Pavement Texture Assessment Using Mobile Phone Imagery

Collecting pavement texture information is crucial to understand the characteristics of a road surface and to have essential data to support road maintenance. Traditional texture assessment techniques often require expensive equipment and complex operations. To ensure cost sustainability and reduce traffic closure times, this study proposes a rapid, cost-effective, and non-invasive surface texture assessment technique. This technology consists of capturing a set of images of a road surface with a mobile phone; then, the images are used to reconstruct the 3D surface with photogrammetric processing and derive the roughness parameters to assess the pavement texture. The results indicate that pavement images taken by a mobile phone can reconstruct the 3D surface and extract texture features with accuracy, meeting the requirements of a time-effective documentation. To validate the effectiveness of this technique, the surface structure of the pavement was analyzed in situ using a 3D structured light projection scanner and rigorous photogrammetry with a high-end reflex camera. The results demonstrated that increasing the point cloud density can enhance the detail level of the real surface 3D representation, but it leads to variations in road surface roughness parameters. Therefore, appropriate density should be chosen when performing three-dimensional reconstruction using mobile phone images. Mobile phone photogrammetry technology performs well in detecting shallow road surface textures but has certain limitations in capturing deeper textures. The texture parameters and the Abbott curve obtained using all three methods are comparable and fall within the same range of acceptability. This finding demonstrates the feasibility of using a mobile phone for pavement texture assessments with appropriate settings.

Boosting point cloud understanding through graph convolutional network with scale measurement and high-frequency enhancement

Graph-based methods have exhibited exceptional performance in point cloud understanding by capturing local geometric relationships. However, existing approaches often struggle to characterize the overall spatial scale of local graphs. In addition, they fail to capture the differences between nodes effectively, which is crucial for distinguishing different classes. This study introduces SM-HFEGCN, a novel graph convolutional network that addresses these limitations through two key innovations: scale measurement and high-frequency enhancement. First, we introduce a spatial scale feature derived from the diagonal vectors of the neighborhood, which serves as a unique graph-specific property related to the geometry and density of the local point cloud. This feature can characterize the overall spatial scale of the local point cloud. Second, we enhance the high-frequency information to capture node variations and integrate it with smoothed information to represent the differences and similarities between nodes simultaneously. Extensive experiments demonstrate the effectiveness of SM-HFEGCN in point cloud classification and segmentation tasks.

Bi-directional information interaction for multi-modal 3D object detection in real-world traffic scenes

Multimodal 3D object detection methods are poorly adapted to real-world traffic scenes due to sparse distribution of point clouds and misalignment multimodal data during actual collection. Among the existing methods, they focus on high-quality open-source datasets, with performance relying on the accurate structural representation of point clouds and the precise mapping relationship between point clouds and images. To solve the above challenges, this paper proposes a multimodal feature-level fusion method based on the bi-directional interaction between image and point cloud. To overcome the sparsity issue in asynchronous multi-modal data, a point cloud densification scheme based on visual guidance and point cloud density guidance is proposed. This scheme can generate object-level virtual point clouds even when the point cloud and image are misaligned. To deal with the unalignment issue between point cloud and image, a bi-directional interaction module based on image-guided interaction with key points of point clouds and point cloud-guided interaction with image context information is proposed. It achieves effective feature fusion even when the point cloud and image are misaligned. The experiments on the VANJEE and KITTI datasets demonstrated the effectiveness of the proposed method, with average precision improvements of 6.20% and 1.54% compared to the baseline.

Immersive remote telerobotics: foveated unicasting and remote visualization for intuitive interaction

Abstract Precise and efficient performance in remote robotic teleoperation relies on intuitive interaction. This requires both accurate control actions and complete perception (vision, haptic, and other sensory feedback) of the remote environment. Especially in immersive remote teleoperation, the complete perception of remote environments in 3D allows operators to gain improved situational awareness. Color and Depth (RGB-D) cameras capture remote environments as dense 3D point clouds for real-time visualization. However, providing enough situational awareness needs fast, high-quality data transmission from acquisition to virtual reality rendering. Unfortunately, dense point-cloud data can suffer from network delays and limits, impacting the teleoperator’s situational awareness. Understanding how the human eye works can help mitigate these challenges. This paper introduces a solution by implementing foveation, mimicking the human eye’s focus by smartly sampling and rendering dense point clouds for an intuitive remote teleoperation interface. This provides high resolution in the user’s central field, which gradually reduces toward the edges. However, this systematic visualization approach in the peripheral vision may benefit or risk losing information and burdening the user’s cognitive load. This work investigates these advantages and drawbacks through an experimental study and describes the overall system, with its software, hardware, and communication framework. This will show significant enhancements in both latency and throughput, surpassing 60% and 40% improvements in both aspects when compared with state-of-the-art research works. A user study reveals that the framework has minimal impact on the user’s visual quality of experience while helping to reduce the error rate significantly. Further, a 50% reduction in task execution time highlights the benefits of the proposed framework in immersive remote telerobotics applications.

Map construction algorithm based on dense point cloud

AbstractIn order to address the issue of the difficulty in constructing maps from sensor‐acquired point clouds, a map construction algorithm is proposed for the effective conversion from sparse point clouds to dense point clouds. First, it constructs a dense point cloud map in real‐time using colour and depth information of keyframes. Then, it generates an accurate point cloud map by voxel filtering and coordinate conversion. Finally, the point cloud map is converted into an octree map using OctoMap. This article uses the TUM dataset for simulation analysis to verify the effectiveness of the algorithm. The results demonstrate that the algorithm can construct dense point cloud maps with high positioning accuracy in sparse point cloud scenarios. The algorithm improves the positioning accuracy by more than 15% under the fr1_xyz, fr1_room, fr1_desk2, fr1_desk1, fr2_desk and fr3_str_tex_near sequences, compared to ORB‐SLAM2, ORB‐SLAM3 and GX ORB‐SLAM2 algorithm

Arch dam point cloud segmentation based on deep feature learning and normal vector data optimization

Separating the dam body, spillway, and other structures from the point cloud in the dam area is an important step in dam deformation monitoring. Manual segmentation is time consuming and inaccurate. This study proposes a point cloud segmentation neural network model based on normal vector optimization suitable for dam environment: (1) This model utilizes the voxel uniform sampling method of equal length cubes to solve the problem of uneven point cloud density caused by wide range and long-distance measurement during point cloud measurement in dam areas. (2) Designed block input and combined output modules in the model, achieving efficient input of large volume point cloud and eliminating the impact of interpolation points offset during seq2seq model decoding process. (3) In response to the diverse characteristics of point cloud normal vectors presented by vegetation, rock mass, and complex dam structures in the dam area, this paper proposes an adaptive radius plane fitting vector estimation method based on eigenvalue method to improve the accuracy of segmentation. The experiment on the prototype arch dam shows that the proposed normal estimation method improves the classification accuracy of PointNet + + from 96.26 to 98.27%. Compared with the other three normal estimation methods (2-jets, Hough CNN, iterative PCA), the overall accuracy is improved by 0.82%, 1.22%, and 0.22%, and the joint average intersection is improved by 0.0293, 0.0325, and 0.0104. The prototype arch dam experiment shows that our proposed method has a segmentation accuracy of 98.27%. Compared with 2-jets, Hough CNN, and iterative PCA, the overall accuracy has been improved by 0.82%, 1.22%, and 0.22%. This study provides a high-precision segmentation scheme for applications such as deformation detection of dam components based on point clouds.

A new technique mapping submerged beachrocks using low-altitude UAV photogrammetry, the Altınova region, northern coast of the Sea of Marmara (NW Türkiye)

High-resolution aerial imagery captured from low altitudes enables the detailed reconstruction of the geomorphological features of submerged beachrocks and the quantification of their topographic complexity. These coastal deposits serve as vital indicators for estimating past sea-level positions and deformation, contributing to our understanding of climate and tectonic processes across various temporal and spatial scales. This study focuses on submerged beachrocks identified in the nearshore coastal area of Tekirdağ city (Altınova), and the existing knowledge regarding such geological formations is limited.Situated in the tectonically active western Marmara region, the coastal zone of The Tekirdağ-Altınova is influenced by the North Anatolian Fault Zone (NAFZ), which has significantly shaped the coastline over time. Utilizing unmanned aerial vehicle (UAV) data, we successfully isolated submerged beachrocks from other coastal deposits, identifying them at depths of 2 m below current sea level and extending approximately 5 km along the coast. This identification was facilitated by integrating high-resolution aerial imagery with morphological analysis techniques, specifically employing structure-from-motion photogrammetry to generate a dense point cloud for bathymetric mapping.The utilization of cost-effective UAV imagery facilitated the efficient monitoring of beachrocks. Geographic information systems (GIS) and the TerraceM application were employed to analyze the high-resolution bathymetry data (with a 5 cm resolution), enabling the precise estimation of shoreline angles and associated errors. These shoreline angles, which correspond to the past sea levels high stand, were mapped using swath profiles oriented perpendicular to the isobaths. Our findings reveal that the majority of submerged beachrocks exhibit shoreline angles between ∼ −0.7 m and −1.1 m. This study presents innovative methodologies for mapping the height and spatial distribution of beachrocks, as well as for reliably estimating reliable uplift rates in the nearshore area of Tekirdağ-Altınova.

On the use of semi-georeferenced photogrammetric dense point clouds in the investigation of rock mass discontinuity properties

On the use of semi-georeferenced photogrammetric dense point clouds in the investigation of rock mass discontinuity properties

Influence of Structure from Motion Algorithm Parameters on Metrics for Individual Tree Detection Accuracy and Precision

Uncrewed aerial system (UAS) structure from motion (SfM) monitoring strategies for individual trees has rapidly expanded in the early 21st century. It has become common for studies to report accuracies for individual tree heights and DBH, along with stand density metrics. This study evaluates individual tree detection and stand basal area accuracy and precision in five ponderosa pine sites against the range of SfM parameters in the Agisoft Metashape, Pix4DMapper, and OpenDroneMap algorithms. The study is designed to frame UAS-SfM individual tree monitoring accuracy in the context of data processing and storage demands as a function of SfM algorithm parameter levels. Results show that when SfM algorithms are properly tuned, differences between software types are negligible, with Metashape providing a median F-score improvement over OpenDroneMap of 0.02 and PIX4DMapper of 0.06. However, tree extraction performance varied greatly across algorithm parameters, with the greatest extraction rates typically coming from parameters causing increased density in dense point clouds and minimal point cloud filtering. Transferring UAS-SfM forest monitoring into management will require tradeoffs between accuracy and efficiency. Our analysis shows that a one-step reduction in dense point cloud quality saves 77–86% in point cloud processing time without decreasing tree extraction (F-score) or basal area precision using Metashape and PIX4DMapper but the same parameter change for OpenDroneMap caused a ~5% loss in precision. Providing reproducible processing strategies is a vital step in successfully transferring these technologies into usage as management tools.

An Improved Adaptive Grid-Based Progressive Triangulated Irregular Network Densification Algorithm for Filtering Airborne LiDAR Data

Ground filtering is crucial for airborne Light Detection and Ranging (LiDAR) data post-processing. The progressive triangulated irregular network densification (PTD) algorithm and its variants outperform others in accuracy, stability, and robustness, using grid-based seed point selection, TIN construction, and iterative rules for ground point identification. However, these methods still face limitations in removing low points and accurately preserving terrain details, primarily due to their sensitivity to grid size. To overcome this issue, a novel PTD filtering algorithm based on an adaptive grid (AGPTD) was proposed. The main contributions of the proposed method include an outlier removal method using a radius outlier removal algorithm and Kd-tree, a method for establishing an adaptive two-level grid based on point cloud density and terrain slope, and an adaptive selection method for angle and distance thresholds in the iterative densification processing. The performance of the AGPTD algorithm was assessed based on widely used benchmark datasets. Results show that the AGPTD algorithm outperforms the classical PTD algorithm in retaining ground feature points, especially in reducing Type I error and average total error significantly. In comparison with other advanced algorithms developed in recent years, the novel algorithm showed the lowest average Type I error, the minimal average total error, and the greatest average Kappa coefficient, which were 1.11%, 2.28%, and 90.86%, respectively. Additionally, the average accuracy, precision, and recall of AGPTD were 97.69%, 97.52%, and 98.98%, respectively.

A point cloud segmentation algorithm based on multi-feature training and weighted random forest

Abstract Point cloud segmentation is the process of dividing point cloud data into a series of coherent subsets according to its attributes. It has been widely used in target recognition, digital protection of cultural relics, medical research and other fields. To improve the classification accuracy of point cloud and achieve accurate segmentation of objects or scenes, a point cloud segmentation algorithm based on multi–features training and weighted random forest (RF) is proposed. Firstly, the feature vector composed of 3D coordinate value, RGB value, echo intensity, point cloud density, normal direction and average curvature is used to train the SVM classifier, and the ‘one–to–one’ strategy is adopted to achieve the initial multivariate rough segmentation of point cloud. Then, the maximum information coefficient and sample correlation coefficient (SCC) are used to evaluate the correlation of the decision tree, and the decision tree is weighted accordingly to build a weak correlation weighted RF, so as to achieve further accurate segmentation of the point cloud. The experiment verifies the effectiveness of the proposed algorithm by segmenting the outdoor scene point cloud data model. The results show that the segmentation algorithm based on multi–features training and weighted RF can achieve accurate point cloud segmentation, and is an effective point cloud segmentation method.

Point cloud upsampling network based on pyramid pooling and self-attention mechanism

For the problems of irregular point cloud data and non-unique input size, a point-cloud upsampling network (PPSA-PU) based on Pyramid Pooling and Self-Attention mechanism is proposed. Multi-scale features are first extracted from the input point cloud using dense map convolution with different expansion rates, and the pyramid pooling module and self-attention mechanism are added. By applying the same pooling operation to regions of different sizes and numbers, the network is made to accept inputs of arbitrary size. Self-attention is used to weight the features to establish global correlation and improve the feature extraction capability. The features are then extended by the Self-Attention NodeShuffle (SA-NS) module, which uses a graph convolution layer to fuse neighbour information and learn new point features from the latent space. Finally, the coordinate reconstruction module is used to generate a dense up-sampled point cloud. Using the existing dataset to train and test the network and compare it with the existing up-sampling network, the experimental results show that PPSA-PU is better in terms of evaluation metrics and performance, and has better generalisation and robustness.

CDP-MVS: Forest Multi-View Reconstruction with Enhanced Confidence-Guided Dynamic Domain Propagation

Using multi-view images of forest plots to reconstruct dense point clouds and extract individual tree parameters enables rapid, high-precision, and cost-effective forest plot surveys. However, images captured at close range face challenges in forest reconstruction, such as unclear canopy reconstruction, prolonged reconstruction times, insufficient accuracy, and issues with tree duplication. To address these challenges, this paper introduces a new image dataset creation process that enhances both the efficiency and quality of image acquisition. Additionally, a block-matching-based multi-view reconstruction algorithm, Forest Multi-View Reconstruction with Enhanced Confidence-Guided Dynamic Domain Propagation (CDP-MVS), is proposed. The CDP-MVS algorithm addresses the issue of canopy and sky mixing in reconstructed point clouds by segmenting the sky in the depth maps and setting its depth value to zero. Furthermore, the algorithm introduces a confidence calculation method that comprehensively evaluates multiple aspects. Moreover, CDP-MVS employs a decentralized dynamic domain propagation sampling strategy, guiding the propagation of the dynamic domain through newly defined confidence measures. Finally, this paper compares the reconstruction results and individual tree parameters of the CDP-MVS, ACMMP, and PatchMatchNet algorithms using self-collected data. Visualization results show that, compared to the other two algorithms, CDP-MVS produces the least sky noise in tree reconstructions, with the clearest and most detailed canopy branches and trunk sections. In terms of parameter metrics, CDP-MVS achieved 100% accuracy in reconstructing tree quantities across the four plots, effectively avoiding tree duplication. The accuracy of breast diameter extraction values of point clouds reconstructed by CDPMVS reached 96.27%, 90%, 90.64%, and 93.62%, respectively, in the four sample plots. The positional deviation of reconstructed trees, compared to ACMMP, was reduced by 0.37 m, 0.07 m, 0.18 m and 0.33 m, with the average distance deviation across the four plots converging within 0.25 m. In terms of reconstruction efficiency, CDP-MVS completed the reconstruction of the four plots in 1.8 to 3.1 h, reducing the average reconstruction time per plot by six minutes compared to ACMMP and by two to three times compared to PatchMatchNet. Finally, the differences in tree height accuracy among the point clouds reconstructed by the different algorithms were minimal. The experimental results demonstrate that CDP-MVS, as a multi-view reconstruction algorithm tailored for forest reconstruction, shows promising application potential and can provide valuable support for forestry surveys.

3D reconstruction and depth estimation method for local anomalies of rail surface based on multi-view stereo matching

Abstract Detecting rail surface anomalies has become crucial for ensuring the safety of train operations. However, manual detection methods are time-consuming and labor-intensive. Although traditional detection models based on one-dimensional signals or two-dimensional images can effectively identify the existence of defects, they are difficult to use to obtain local depth characteristic information, leading to difficulties in accurately assessing the extent of damage in abnormal regions. Rail track maintenance strategies are typically developed based on the detected different depth ranges of defects. To address this issue, this paper proposes a local depth estimation method based on the point cloud of the target rail surface reconstructed by PatchmatchNet. Firstly, according to the acquisition method of multi-view images and the sampling environment at the site, a practical data collection protocol is established for generating a multi-view dataset of rail surface. Next, dense point cloud of the target rail surface were reconstructed by PatchmatchNet. Subsequently, a method for estimating the local anomaly depth is developed based on the point cloud. Experimental results demonstrate that the proposed method achieves a minimum error of approximately 5.5% within a maximum depth range of 0.35mm, when compared to depth measurements obtained using a structured light camera. Finally, this paper presents a more comprehensive three-dimensional representation of the local abnormal physical structure, surpassing traditional one-dimensional and two-dimensional forms. This enhanced representation offers richer information for the effective determination of whether the anomalies constitute defects and aids in distinguishing true defects from other surface irregularities. Such advancement significantly improves the precision of defect assessment and supports the development of highly precise repair strategies.

Dense 3D Point Cloud Environmental Mapping Using Millimeter-Wave Radar

To address the challenges of sparse point clouds in current MIMO millimeter-wave radar environmental mapping, this paper proposes a dense 3D millimeter-wave radar point cloud environmental mapping algorithm. In the preprocessing phase, a radar SLAM-based approach is introduced to construct local submaps, which replaces the direct use of radar point cloud frames. This not only reduces data dimensionality but also enables the proposed method to handle scenarios involving vehicle motion with varying speeds. Building on this, a 3D-RadarHR cross-modal learning network is proposed, which uses LiDAR as the target output to train the radar submaps, thereby generating a dense millimeter-wave radar point cloud map. Experimental results across multiple scenarios, including outdoor environments and underground tunnels, demonstrate that the proposed method can increase the point cloud density of millimeter-wave radar environmental maps by over 50 times, with a point cloud accuracy better than 0.1 m. Compared to existing algorithms, the proposed method achieves superior environmental map reconstruction performance while maintaining a real-time processing rate of 15 Hz.

Multi-Feature-Filtering-Based Road Curb Extraction from Unordered Point Clouds

Road curb extraction is a critical component of road environment perception, being essential for calculating road geometry parameters and ensuring the safe navigation of autonomous vehicles. The existing research primarily focuses on extracting curbs from ordered point clouds, which are constrained by their structure of point cloud organization, making it difficult to apply them to unordered point cloud data and making them susceptible to interference from obstacles. To overcome these limitations, a multi-feature-filtering-based method for curb extraction from unordered point clouds is proposed. This method integrates several techniques, including the grid height difference, normal vectors, clustering, an alpha-shape algorithm based on point cloud density, and the MSAC (M-Estimate Sample Consensus) algorithm for multi-frame fitting. The multi-frame fitting approach addresses the limitations of traditional single-frame methods by fitting the curb contour every five frames, ensuring more accurate contour extraction while preserving local curb features. Based on our self-developed dataset and the Toronto dataset, these methods are integrated to create a robust filter capable of accurately identifying curbs in various complex scenarios. Optimal threshold values were determined through sensitivity analysis and applied to enhance curb extraction performance under diverse conditions. Experimental results demonstrate that the proposed method accurately and comprehensively extracts curb points in different road environments, proving its effectiveness and robustness. Specifically, the average curb segmentation precision, recall, and F1 score values across scenarios A, B (intersections), C (straight road), and scenarios D and E (curved roads and ghosting) are 0.9365, 0.782, and 0.8523, respectively.

3D-DCSNet: reconstruction of dense point clouds of objects with complex surface structures from a single image

3D-DCSNet: reconstruction of dense point clouds of objects with complex surface structures from a single image