3D Point Research Articles

Improved 3D laser point cloud reconstruction for autonomous mobile robot applications by using SVM-R technique

PurposeIn autonomous mobile robots, high-level accuracy and precision in 3D perception are required for object detection, shape estimation and obstacle distance measurement. However, the existing methods suffer from limitations like inaccurate point clouds, noise in sensor data and synchronization problems between 2D LiDAR and servomotor. These factors can lead to the wrong perception and also introduce noise during sensor registration. Thus, the purpose of this study is to address these limitations and enhance the perception in autonomous mobile robots.Design/methodology/approachA new sensor mounting structure is developed for 3D mapping by using a 2D LiDAR and servomotor. The proposed method uses a support vector machine regression (SVM-R) technique to optimize the waypoints of the servomotor for the point cloud reconstruction process and to obtain a highly accurate and detailed representation of the environment.FindingsThe study includes an analysis of the SVM-R model, including Linear, radial basis function (RBF) and Polynomial kernel. Results show that the Linear kernel performs better with the lowest 3.67 mean absolute error (MAE), 26.24 mean squared error (MSE) and 5.12 root mean squared error (RMSE) values than the RBF and Polynomial kernels. The 2D to 3D point cloud reconstruction shows that the proposed method with a new sensor mounting structure performs better in perception accuracy and achieves an error of 0.45% in measuring the height of the target objects whereas in previous techniques the error was very large.Originality/valueThe study shows the effectiveness of SVM-R in the 3D point cloud reconstruction process and exhibits remarkable performance for object height measurement. Further, the proposed technique is applicable for future advanced visual applications and has a superior performance over the other conventional methods.

Upsampling Algorithm for V-PCC-Coded 3D Point Clouds

Point cloud (PC) compression is crucial to immersive visual applications such as autonomous vehicles to classify objects on the roads. The Motion Picture Experts Group (MPEG) standardization group has achieved a notable compression efficiency, called video-based PC compression (V-PCC), which consists of an encoder-decoder. The V-PCC encoder takes original 3D PC data and projects them onto multiple 2D planes to generate several 2D feature images. These images are then compressed using the well-established High-Efficiency Video Coding (HEVC) method. The V-PCC decoder uses compressed information and decoding techniques to reconstruct the 3D PC. However, the PCs produced by V-PCC are often sparse, non-uniform, and contain artifacts. In many practical applications, it is necessary to recover complete PCs from partial ones in real time. This article presents a method for enhancing decoded PCs as a post-processing step in the V-PCC with reduced computational time. Our approach involves a 2D upsampling for the V-PCC occupancy image, which increases the density of the PC, and a 2D high-resolution auxiliary information modification algorithm for the 2D-3D conversion of high-resolution 3D PCs, which improves the uniformity and reduces the noise in the PC. The 3D high-resolution PC has been further enhanced using the developed 3D outlier removal and point regeneration algorithm. Our proposed work can significantly simplify the state-of-the-art super resolution methods for PCs and reduce the time complexity of 61–75% while maintaining a high level of quality in PCs.

Robust 3D point clouds classification based on declarative defenders

Robust 3D point clouds classification based on declarative defenders

Study of the application of UAV and LiDAR to topographic survey of mountainous forest areas – Case study of Win 3 Wind Power Plant Project, Huong Hoa District, Quang Tri Province, Vietnam

Abstract Unmanned Aerial Vehicle (UAV) technology in conjunction with aerial photography and aerial Light Detection and Ranging (LiDAR) was used for topographic surveying of the Win 3 Wind Power Project area, a complex mountainous terrain situated in the Huong Hoa district of Quang Tri province, Vietnam with the primary objective to produce a map with a scale of 1:1,000 with a contour interval of 1.0 m, accompanied by 3D point clouds, a Digital Elevation Model (DEM), and an orthomosaic. Employing the UAV LiDAR system of DJI Matrice 300 RTK, we conduct ed survey flights, achieving a high point density laser scanning of 229 points/m2 and capturing ultra-high-resolution imagery at 4.22 cm/pixel. The integration of aerial LiDAR and photographic data results in a rich and detailed information repository regarding the surveyed terrain demonstrating that UAV LiDAR technology is a robust tool for geographic data collection, particularly in challenging terrain conditions in Vietnam. We also evaluated the accuracy of the created topographic map, achieving results that surpass the technical requirements and quality standards of the project. This underscores the potential and effectiveness of UAV LiDAR technology in the field of large-scale geographic data collection and mapping in Vietnam.

3D Point Cloud Fusion Method Based on EMD Auto-Evolution and Local Parametric Network

Although the development of high-resolution remote sensing satellite technology has made it possible to reconstruct the 3D structure of object-level features using satellite imagery, the results from a single reconstruction are often insufficient to comprehensively describe the 3D structure of the target. Therefore, developing an effective 3D point cloud fusion method can fully utilize information from multiple observations to improve the accuracy of 3D reconstruction. To this end, this paper addresses the problems of shape distortion and sparse point cloud density in existing 3D point cloud fusion methods by proposing a 3D point cloud fusion method based on Earth mover’s distance (EMD) auto-evolution and local parameterization network. Our method is divided into two stages. In the first stage, EMD is introduced as a key metric for evaluating the fusion results, and a point cloud fusion method based on EMD auto-evolution is constructed. The method uses an alternating iterative technique to sequentially update the variables and produce an initial fusion result. The second stage focuses on point cloud optimization by constructing a local parameterization network for the point cloud, mapping the upsampled point cloud in the 2D parameter domain back to the 3D space to complete the optimization. Through these two steps, the method achieves the fusion of two sets of non-uniform point cloud data obtained from satellite stereo images into a single, denser 3D point cloud that more closely resembles the true target shape. Experimental results demonstrate that our fusion method outperforms other classical comparison algorithms for targets such as buildings, planes, and ships, and achieves a fused RMSE of approximately 2 m and an EMD accuracy better than 0.5.

Influence of the Inclusion of Off-Nadir Images on UAV-Photogrammetry Projects from Nadir Images and AGL (Above Ground Level) or AMSL (Above Mean Sea Level) Flights

UAV-SfM techniques are in constant development to address the challenges of accurate and precise mapping in terrains with complex morphologies. In contrast with the traditional photogrammetric processes, where only nadir images were considered, the combination of those with oblique imagery, also called off-nadir, has emerged as an optimal solution to achieve higher accuracy in these kinds of landscapes. UAV flights at a constant height above ground level (AGL) have also been considered a possible alternative to improve the resulting 3D point clouds compared to those obtained from constant height above mean sea level (AMSL) flights. The aim of this study is to evaluate the effect of incorporating oblique images as well as the type of flight on the accuracy and precision of the point clouds generated through UAV-SfM workflows for terrains with complex geometries. For that purpose, 58 scenarios with different camera angles and flight patterns for the oblique images were considered, 29 for each type of flight (AMSL and AGL). The 3D point cloud derived from each of the 58 scenarios was compared with a reference 3D point cloud acquired with a terrestrial laser scanner (TLS). The results obtained confirmed that both incorporating oblique images and using AGL flight mode have a positive effect on the mapping. Combination of nadir image blocks, obtained from an AGL crosshatch flight plan, with supplemental oblique images collected with a camera angle of between 20° and 35° yielded the best accuracy and precision records.

Remote Sensing and GIS in Natural Resource Management: Comparing Tools and Emphasizing the Importance of In-Situ Data

Remote sensing (RS) and Geographic Information Systems (GISs) provide significant opportunities for monitoring and managing natural resources across various temporal, spectral, and spatial resolutions. There is a critical need for natural resource managers to understand the expanding capabilities of image sources, analysis techniques, and in situ validation methods. This article reviews key image analysis tools in natural resource management, highlighting their unique strengths across diverse applications such as agriculture, forestry, water resources, soil management, and natural hazard monitoring. Google Earth Engine (GEE), a cloud-based platform introduced in 2010, stands out for its vast geospatial data catalog and scalability, making it ideal for global-scale analysis and algorithm development. ENVI, known for advanced multi- and hyperspectral image processing, excels in vegetation monitoring, environmental analysis, and feature extraction. ERDAS IMAGINE specializes in radar data analysis and LiDAR processing, offering robust classification and terrain analysis capabilities. Global Mapper is recognized for its versatility, supporting over 300 data formats and excelling in 3D visualization and point cloud processing, especially in UAV applications. eCognition leverages object-based image analysis (OBIA) to enhance classification accuracy by grouping pixels into meaningful objects, making it effective in environmental monitoring and urban planning. Lastly, QGIS integrates these remote sensing tools with powerful spatial analysis functions, supporting decision-making in sustainable resource management. Together, these tools when paired with in situ data provide comprehensive solutions for managing and analyzing natural resources across scales.

Enhancing environmental modeling and 3D point cloud construction for unmanned vehicles using laser SLAM and stereo vision with Convolutional Neural Networks

Abstract. This research presents an innovative SLAM algorithm that integrates Convolutional Neural Networks (CNNs) with LIDAR and stereo vision to significantly enhance the accuracy of environmental modeling and the construction of dense 3D point cloud maps in complex and dynamic surroundings. By processing pre-recorded video data and employing advanced image segmentation techniques, this study achieves a deep fusion of visual and geometric data, resulting in highly detailed and precise 3D representations of the environment. The experimental results demonstrate that this approach effectively detects and excludes dynamic objects, thereby significantly improving the overall quality, robustness, and reliability of the constructed maps. This work represents a substantial advancement in SLAM technology, particularly in its capability to model and capture intricate environmental details under varying conditions, making it a powerful tool for precise environmental mapping.

Enhancing Fluvial Modeling and Flood Prediction Accuracy through the Fusion of 3D Point Clouds, Multispectral, and RGB Data

Enhancing Fluvial Modeling and Flood Prediction Accuracy through the Fusion of 3D Point Clouds, Multispectral, and RGB Data

GETr: A Geometric Equivariant Transformer for Point Cloud Registration

AbstractAs a fundamental problem in computer vision, 3D point cloud registration (PCR) aims to seek the optimal transformation to align point cloud pairs. Meanwhile, the equivariance lies at the core of matching point clouds at arbitrary pose. In this paper, we propose GETr, a geometric equivariant transformer for PCR. By learning the point‐wise orientations, we decouple the coordinate to the pose of the point clouds, which is the key to achieve equivariance in our framework. Then we utilize attention mechanism to learn the geometric features for superpoints matching, the proposed novel self‐attention mechanism encodes the geometric information of point clouds. Finally, the coarse‐to‐fine manner is used to obtain high‐quality correspondence for registration. Extensive experiments on both indoor and outdoor benchmarks demonstrate that our method outperforms various existing state‐of‐the‐art methods.

A unified LOD model for river network and DEM based on an improved 3D_DP algorithm

The level of detail (LOD) modelling of vector and terrain data is individual, resulting in geometric and topological inconsistencies in simplification processes. The three dimension Douglas Peucker (3D_DP)algorithm can realize gradual discrete point selection through threshold, which is mainly used in DEM synthesis, and its simplified process is very suitable for the dynamic establishment of massive data sets. A new LOD modeling method based on 3D_DP algorithm is proposed to simplify the consistency of river network vector elements and DEM in this paper. The specific steps are as follows: Firstly, the “Bending Adjustment Index (BAI)” is introduced to improve the 3D_DP algorithm, called the improved 3D_DP algorithm; Secondly, the DEM data is extracted into a 3D discrete point dataset, and the river line vector data is also converted into a discrete point dataset, assigned with elevation attributes, and merges with the DEM’s 3D discrete points. The merged point datasets are equenced based on the importance of each point, which are computed by the improved 3D_DP algorithm. The order of deleting points is determined by the sequence and the corresponding model spatial errors are calculated with the deletion of points. Then, the DEM’s 3D discrete points are constructed a Delaunay network with the river line as a mandatory constraint condition. The required triangulation is called in real time with the change of sight distance depending on the simple correspondence between screen projection error and model space error, and the unified LOD model for river line vector and DEM is established. The results show that the river’s overall shape and the terrain’s main features can be reserved under the same simplified factor based on the improved 3D_DP algorithm. The unified LOD model for the river network and DEM is feasible under the importance sequence of merged point datasets by the improved 3D_DP algorithm. Under the proper operation of data blocking, the rendering frame rate can meet practical application requirements.

Empirical Evaluation and Simulation of the Impact of Global Navigation Satellite System Solutions on Uncrewed Aircraft System–Structure from Motion for Shoreline Mapping and Charting

Uncrewed aircraft systems (UASs) and structure-from-motion/multi-view stereo (SfM/MVS) photogrammetry are efficient methods for mapping terrain at local geographic scales. Traditionally, indirect georeferencing using ground control points (GCPs) is used to georeference the UAS image locations before further processing in SfM software. However, this is a tedious practice and unsuitable for surveying remote or inaccessible areas. Direct georeferencing is a plausible alternative that requires no GCPs. It relies on global navigation satellite system (GNSS) technology to georeference the UAS image locations. This research combined field experiments and simulation to investigate GNSS-based post-processed kinematic (PPK) as a means to eliminate or reduce reliance on GCPs for shoreline mapping and charting. The study also conducted a brief comparison of real-time network (RTN) and precise point positioning (PPP) performances for the same purpose. Ancillary experiments evaluated the effects of PPK base station distance and GNSS sample rate on the accuracy of derived 3D point clouds and digital elevation models (DEMs). Vertical root mean square errors (RMSEz), scaled to the 95% confidence interval using an assumption of normally-distributed errors, were desired to be within 0.5 m to satisfy National Oceanic and Atmospheric Administration (NOAA) requirements for nautical charting. Simulations used a Monte Carlo approach and empirical tests to examine the influence of GNSS performance on the quality of derived 3D point clouds. RTN and PPK results consistently yielded RMSEz values within 10 cm, thus satisfying NOAA requirements for nautical charting. PPP did not meet the accuracy requirements but showed promising results that prompt further investigation. PPK experiments using higher GNSS sample rates did not always provide the best accuracies. GNSS performance and model accuracies were enhanced when using base stations located within 30 km of the survey site. Results without using GCPs observed a direct relationship between point cloud accuracy and GNSS performance, with R2 values reaching up to 0.97.

Is it possible to generate accurate 3D point clouds with UAS-LIDAR and UAS-RGB photogrammetry without GCPs? A case study on a beach and rocky cliff

ContextRecently, Unoccupied Aerial Systems (UAS) with photographic or Light Detection and Ranging (LIDAR) sensors have incorporated on-board survey-grade Global Navigation Satellite Systems that allow the direct georeferencing of the resulting datasets without Ground Control Points either in Real-Time (RTK) or Post-Processing Kinematic (PPK) modes. These approaches can be useful in hard-to-reach or hazardous areas. However, the resulting 3D models have not been widely tested, as previous studies tend to evaluate only a few points and conclude that systematic errors can be found.ObjectivesWe test the absolute positional accuracy of point clouds produced using UAS with direct-georeferencing systems.MethodsWe test the accuracy and characteristics of point clouds produced using a UAS-LIDAR (with PPK) and a UAS-RGB (Structure-from-Motion or SfM photogrammetry with RTK and PPK) in a challenging environment: a coastline with a composite beach and cliff. The resulting models of each processing were tested using as a benchmark a point cloud surveyed simultaneously by a Terrestrial Laser Scanner.ResultsThe UAS-LIDAR produced the most accurate point cloud, with homogeneous cover and no noise. The systematic bias previously observed in the UAS-RGB RTK approaches are minimized using oblique images. The accuracy observed across the different surveyed landforms varied significantly.ConclusionsThe UAS-LIDAR and UAS-RGB with PPK produced unbiased point clouds, being the latter the most cost-effective method. For the other direct georeferencing systems/approaches, the acquisition of GCP or the co-registration of the resulting point cloud is still necessary.

Three-Dimensional Hullform Reconstruction from Two-Dimensional Drawings Based on Image Processing Techniques

Abstract Repair shipyards sometimes need three-dimensional geometry for repairs and retrofits. However, they often create models manually from two-dimensional drawings provided by ship owners. In this case, human error leads to inaccuracies, making the process time-consuming and laborious. Therefore, there is a need for research on efficient three-dimensional hullform reconstruction from two-dimensional drawings. This study proposed a method to automatically extract points from two-dimensional lines and visualize them in three dimensions. The proposed method consists of three steps. The first step is a point extraction through image processing, which uses a starting point search algorithm to access overlapping or intersection lines and extracts the points on the lines in the drawing by searching for paths between the starting point and the end point entered by the user. The second step is the transformation of the extracted data, which transforms the points based on pixel coordinate into 3D points through coordinate transformation and scaling by utilizing the stored line data and three-dimensional coordinate information. The last step is to visualize the transformed data as a real three-dimensional model with point visualization. This study demonstrates that the proposed method can be effectively utilized by detecting two-dimensional lines and reconstructing the hullform in three dimensions.

Attention Mechanisms in PointNet++ for Effective Object Classification in 3D Point Clouds

Attention Mechanisms in PointNet++ for Effective Object Classification in 3D Point Clouds

A novel method: YOLO-CE and 3D point cloud-based feature extraction for welding seams of tower bases

Abstract Robotic automated welding of non-standard steel structures presents significant challenges, particularly for electric power tower bases. This study introduces a novel approach that integrates the You Only Look Once—Compact Invert Block and Efficient Local Attention (YOLO-CE) model, an enhanced version of YOLOV8 for 2D image segmentation, with 3D point cloud technology. The YOLO-CE model is used to accurately extract point cloud data from the target area, which is then processed using the MSAC algorithm for efficient plane segmentation. Weld lines are identified through plane equations, allowing for initial weld point cloud extraction. To further refine accuracy, an optimized evaluation equation is developed that accounts for both the distance between the weld point cloud and the fitted plane, and the angle between their normal vectors. This enables precise classification of the weld point cloud. From this classification, key weld feature points are identified, and their exact positions are determined by calculating the distances between these points and their intersections with three planes. The reliability of the proposed method was validated using a robot for precise measurements, with a total error margin of less than 1.5084 mm, demonstrating high accuracy and stability. Post-operation inspections confirmed that the welds were filled and free from defects, meeting all process requirements. The YOLO-CE model achieved a mIoU of 96.38% and a precision of 99.8%, highlighting its effectiveness. This method provides an efficient and precise solution for the automated welding of non-standard steel structural components and has promising application potential.

Integration by design: driving mineral system knowledge using multi-modal, collocated, scale-consistent characterisation

Abstract. Recent decades have seen an exponential rise in the application of machine learning in geoscience. However, fundamental differences distinguish geoscience data from most other data types. Geoscience datasets are typically multi-dimensional, and contain 1D (drill holes), 2D (maps or cross-sections), and 3D volumetric and point data (models/voxels). Geoscience data quality is a product of the data's resolution and the precision of the methods used to acquire them. The dimensionality, resolution, and precision of each layer within a geoscience dataset translate into limitations to the spatiality, scale, and uncertainty of resulting interpretations. Historically, geoscience datasets were overlaid cartographically to incorporate subjective, experience-driven knowledge and variances in scale and resolution. These nuances and limitations that underpin the reliability of automated interpretation are well understood by geoscientists but are rarely appropriately transferred to data science. For true integration of geoscience data, such issues cannot be overlooked without consequence. To apply data analytics to complex geoscience data (e.g. hydrothermal mineral systems) effectively, methodologies that characterise the system quantitatively at a common scale, using collocated analyses, should be sought. This paper provides research and exploration insights from an innovative district-wide, scale-integrated geoscience data project, which analysed 1590 samples from 23 mineral deposits and prospects across the Cloncurry district, Queensland, Australia. Nine different analytical techniques were used, including density, magnetic susceptibility, remanent magnetisation, anisotropy of magnetic susceptibility, radiometrics, conductivity, automated mineralogy based on scanning electron microscopy (SEM), geochemistry, and short-wave infrared (SWIR) hyperspectral data with 561 columns of scale-integrated data (+2151 columns of SWIR data). All data were collected on 2.2 cm × 2.5 cm sample cylinders, a scale at which the confidence in the coupling of data from techniques can be high. These data are integrated by design to eliminate the need to downscale coarser measurements via assumptions, inferences, inversions, and interpolations. This scale-consistent approach is critical to the quantitative characterisation of mineral systems and has numerous applications in mineral exploration, such as linking alteration paragenesis with structural controls and petrophysical zonation. The Cloncurry METAL dataset is made freely available via the AuScope Data Repository: https://doi.org/10.60623/82trleue (Austin et al., 2024).

3D morphometry of Martian craters from HRSC DEMs using a multi-scale semantic segmentation network and morphological analysis

The morphology of impact craters reveals the structure and composition of the Martian surface, especially the subsurface conditions and Martian geological history, which have increasing importance in Mars exploration missions. This work presents a 3D morphometric method for detecting and delineating Martian craters, and a 3D morphological analysis was conducted. Specifically, this work first focused on the segmentation of Martian craters. Based on the segmentation results, clustering of crater instances was then carried out. Finally, with the individual craters that were extracted, a morphological analysis involving the measurements of their diameter, depth, area, RMS height, rim height, circularity, and the statistics thereof was performed. Unlike previous studies, which have mainly used optical images and object detection approaches, this work regards crater extraction as a semantic segmentation task instead of an object detection task to better delineate the precise shape and boundary information. Digital elevation model (DEM) was utilized as primary data to directly obtain 3D information, which was converted into 3D point cloud format and fed to a multi-scale semantic segmentation network. The semantic segmentation results achieved an overall accuracy of 0.932 and mIOU of 0.871 on the test data. We automatically counted 63 craters in Noachis Terra and 40 craters in Terra Cimmeria. The 3D morphological measurements showed that 66% of the impact craters in the first region were larger than 10 km in diameter, while 50% of the impact craters in the second region were larger than 10 km. In both areas, craters could reach a maximum depth of 2000 m. With the proposed method, we can automatically conduct 3D morphological measurements of Martian craters with high efficiency that is improved by 15 times compared with that of manual crater analysis tools. The achieved 3D morphometric results can provide a reference and support for future research on Martian landforms.

3D dataset generation using virtual reality for forest biodiversity

ABSTRACT While forest biodiversity faces a concerning decline, modern technology presents promising avenues for mitigation. However, a critical gap persists in reconciling ecological knowledge with the technical expertise required to use state-of-the-art technologies in 3D data classification. Currently, one main issue is the scarcity of 3D datasets for biodiversity, particularly within the context of machine learning applications. Unlike the straightforward classification of human-made structures, forest environments are uniquely intricate and nuanced due to its inherently complex nature. This study addresses this challenge by introducing a fully automated pipeline for tree stem 3D point cloud segmentation, focussing on a biodiversity indicator: tree-related microhabitats (TreMs). Furthermore, our research advances the field by demonstrating that machine learning models trained with labels generated by our proposed virtual reality (VR) method, Labelling Flora, yield predictions statistically similar to the traditional desktop-based labelling methods. This implies that existing 3D datasets could be augmented using the more rapid approach of VR labelling. Additionally, the findings of this paper demonstrate the potential integration of VR and immersive technology into the 3D labelling workflow, facilitating a quicker and more intuitive labelling process. This could empower users, who are non-familiar with 3D modelling, to contribute their expertise to the segmentation process.

AIFormer: Adaptive Interaction Transformer for 3D Point Cloud Understanding

Recently, significant advancements have been made in 3D point cloud analysis by leveraging transformer architecture in 3D space. However, it remains challenging to effectively implement local and global learning within irregular and sparse structures of 3D point clouds. This paper presents the Adaptive Interaction Transformer (AIFormer), a novel hierarchical transformer architecture designed to enhance 3D point cloud analysis by fusing local and global features through the adaptive interaction of features. Specifically, AIFormer mainly consists of several stacked AIFormer Blocks. Each AIFormer module employs the Local Relation Aggregation Module and the Global Context Aggregation Module, respectively, to extract local details of relationships within the reference point and long-range dependencies between reference points. Then, the local and global features are fused using the Adaptive Interaction Module for adaptive interaction to optimize the point representation. Additionally, the AIFormer Block further designs geometric relation functions and contextual relative semantic encoding to enhance local and global feature extraction capabilities, respectively. Extensive experiments on three popular 3D point cloud datasets verify that AIFormer achieves state-of-the-art or comparable performances. Our comprehensive ablation study further validates the effectiveness and soundness of the AIFormer design.