Accurate Point Cloud Research Articles

Discontinuity Recognition and Information Extraction of High and Steep Cliff Rock Mass Based on Multi-Source Data Fusion

It is fundamental to acquire accurate point cloud information on rock discontinuities efficiently and comprehensively when evaluating the stability of rock masses. Taking a high and steep cliff as an example, we combined 3D laser scanning and UAV photogrammetry technology to collect rock data, and proposed an intelligent identification method for rock discontinuities based on the multi-source fusion of point clouds. First, the 3D-laser-collected point cloud data were used as the basis to fuse with the UAV-photogrammetry-collected data, and the unified coordinate system and improved ICP algorithm were used to obtain the complete 3D point cloud in the study area. Secondly, we used neighborhood information entropy to achieve adaptive neighborhood-scale selection and to obtain the optimal neighborhood radius for the KNN search, to effectively calculate the point cloud normal vector and rock mass orientation information. Finally, the KDE algorithm and DBSCAN algorithm were combined for rock discontinuity clustering to achieve intelligent identification and information extraction of the rock structural plane. The clustering results were imported into the DSE program developed based on Matlab to calculate the discontinuity spacing and continuity of the rock mass structure, and to efficiently obtain the parameters of rock mass occurrence. The research results showed that this method can effectively solve the problem of incomplete-data-acquisition ground 3D laser scanning in complex geological conditions, and UAV photogrammetry prone to blurred images in depressed areas. When the extraction results were compared with the field-measured rock occurrence, the average dip angle error was about 2°, the average dip direction error was 1°, and the recognition results met the accuracy requirements. The research results provide a feasible scheme for the identification and extraction of discontinuities of high and steep rock masses.

DOPNet: Achieving Accurate and Efficient Point Cloud Registration Based on Deep Learning and Multi-Level Features.

Point cloud registration aims to find a rigid spatial transformation to align two given point clouds; it is widely deployed in many areas of computer vision, such as target detection, 3D localization, and so on. In order to achieve the desired results, registration error, robustness, and efficiency should be comprehensively considered. We propose a deep learning-based point cloud registration method, called DOPNet. DOPNet extracts global features of point clouds with a dynamic graph convolutional neural network (DGCNN) and cascading offset-attention modules, and the transformation is predicted by a multilayer perceptron (MLP). To enhance the information interaction between the two branches, the feature interaction module is inserted into the feature extraction pipeline to implement early data association. We compared DOPNet with the traditional method of using the iterative closest point (ICP) algorithm along with four learning-based registration methods on the Modelnet40 data set. In the experiments, the source and target point clouds were generated by sampling the original point cloud twice independently; we also conducted additional experiments with asymmetric objects. Further evaluation experiments were conducted with point cloud models from Stanford University. The results demonstrated that our DOPNet method outperforms these comparative methods in general, achieving more accurate and efficient point cloud registration.

Terrestrial laser scanning assisted dimensional quality assessment for space frame components

Space frame components (SFCs), mainly spherical joints and cylindrical tubes, are characterized by their large number. The dimensional quality assessment (DQA) of SFCs relies on manual measurements, which are subjective and inefficient. Since the surface of SFCs is mainly curved, many DQA items are hardly achieved by manual measurements. As the terrestrial laser scanning (TLS) can record accurate point cloud data of SFCs, it is widely used in the construction industry for DQA. This study proposes an automated method for DQA on SFCs based on TLS, including the component recognition, geometric parameter estimation, and DQA. A feature-based method is applied for component recognition. The geometric parameter estimation algorithms for spherical joints and cylindrical tubes are proposed. The effectiveness of the proposed method is verified by comparing the proposed algorithms with other algorithms and by conducting the validation experiment on 13 spherical joints and 33 cylindrical tubes.

SEMANTIC SEGMENTATION OF INDOOR 3D POINT CLOUDS BY JOINT OPTIMIZATION OF GEOMETRIC FEATURES AND NEURAL NETWORKS

Abstract. Indoor navigation, indoor robotics, and other deep applications of interior space can be realized through semantic segmentation of 3D point clouds. We propose a semantic segmentation method for point clouds that uses geometric features of point clouds and neural networks to address the problem of incomplete and inconsistent segmentation objectives in existing semantic segmentation methods. Using neural networks, semantic labels are extracted from indoor structural information as the first step. The paper proposes a probabilistic model to cross-validate the initial segmentation results with the segmentation results of geometric features to achieve joint optimization of the results for semantic segmentation. Three sets of indoor point clouds data from simple to complex indoor scenes are used to test the accuracy and validity of the segmentation method proposed in this paper. The experimental results demonstrate that the method proposed in this paper can effectively improve the semantic segmentation accuracy of indoor 3D point clouds.

Assessment of Using UAV Imagery over Featureless Surfaces for Topographic Applications

The growth and development of Unmanned Aerial Vehicles (UAVs) as a photogrammetric platform, concurrently with the advances in Computer Vision (CV) and image processing algorithms have resulted using UAV Photogrammetry in several topographic applications. CV software algorithms rely on extracting, describing, and matching tie points from the sequences overlapping images to generate 3D colored point clouds. One of the biggest problems obstructing the automated processing of UAV imagery is the featureless of the covered surface. This paper has provided the ability, results, and accuracy of processing images captured by UAVs over non-textured sandy surfaces by providing four aligning and geo-referencing techniques. These four methods, IG/blind matching, IG/reference matching, DG/blind matching, and DG/reference matching, have been presented and tested for 630 aerial images with 80 % overlap and 80 % side lap covered approximately 1 km2 at altitude 178 m above ground level (AGL). The results showed that the captured images could be used to extract the photogrammetric topographical measurements with reliable accuracy. The four techniques' geometric accuracy has ranged between (0.043 m to 0.076 m) & (0.047 m to 0.074) for generated point clouds and linear exterior orientation (EO) parameters, respectively. The indirect geo-referencing with reference matching (IG/reference) recorded the highest-level accuracy of point clouds with 0.043m RMSE compared to the direct geo-referencing with reference matching (DG/reference) which gave the highest geometric accuracy of the linear EO parameters with 0.047m

The Application of Laser-Scanning-Based BIM Technology in Large Steel Structure Engineering for Environmental Protection

The rise of heteromorphic architecture brings great challenges to engineering design, blanking, construction, completion testing, and maintenance. The 3D laser scanning measurement technology can quickly achieve the “copy” measurement of the target, especially suitable for the digitization of complex structure and the accurate construction of true 3D model and it takes care of other things which are necessary environment point of view. In this paper, we are considering the steel structure inspection and curtain wall blanking in the construction of the Grand Theater in the “Shangqiu Cultural and Art Center” as a case study. First, the laser scanning technology has been introduced to complete the data acquisition of steel structure objects and to screen the other details of the structure. Then, the high-quality point cloud is obtained through multistation splicing, filtering, denoising and smoothing, compression, and simplification. Through field comparison, the point cloud accuracy reached an acceptable level. Through field comparison, it can be seen that the difference between the designed building model combining laser technology and BIM technology is distributed within the range of ±(1–5)mm, and the mean square error is ±3.5 mm, and the deviation of numerical simulation is small, which meets the building requirements. Therefore, this method can effectively detect steel structure, and the BIM model can be updated according to the measured point cloud data. We provide accurate data reference for curtain wall material and installation.

Combined use of Terrestrial Laser Scanning and UAV Photogrammetry in producing the LoD3 of 3D high building model

Both Unmanned Aerial Vehicles (UAVs) and Terrestrial Laser Scanners (TLS) are important techniques for surveying and mapping. UAV equipment is commonly used to collect 2D or 3D data acquisition. Meanwhile, TLS equipment is used for obtaining only 3D data acquisition. However, if both are integrated, they were able to produce more accurate data. Multi-sensor data fusion helps overcome the limitations of a single sensor and enables a complete 3D model for the structure and better object classification. This study focuses on studying the combination of UAV and TLS technologies to collect, process data, and create the complete point cloud between two point clouds of the high building in Ha Long city, Quang Ninh province to establish a 3D model at LoD 3 detail level, with high accuracy. FARO FOCUS3D X130 and DJI Phantom 4 RTK equipments were used to acquire the data in the field. The aerial and ground data were processed using FARO SCENE 2019 and Agisoft PhotoScan software, respectively. The data integration process is done by converting both point clouds into the same coordinate system and then by aligning the same points of both points clouds in Cloud Compare. The result of this study is a 3D model at LoD 3 detail level of the high building based on the point cloud accuracy in centimeter level. The combined use of UAV and TLS technologies has proven to be possible to create a highly accurate 3D model, at the 1:500 scale of urban areas according to current standards.

Three-Dimensional Reconstruction and Protection of Mining Heritage Based on Lidar Remote Sensing and Deep Learning

In the past, the traditional small-scale 3D reconstruction technology developed maturely and mainly relied on manual handheld devices for data acquisition. Due to multiple factors, inaccuracy, and other problems, data will be omitted, and sometimes staff will face risks directly. Nowadays, with the development of computer science and technology, three-dimensional reconstruction technology has been innovated continuously, and it has also made a breakthrough in the application of large-scale reconstruction of dangerous areas. In this paper, the expensive sensors in the past are abandoned, and the cheaper and more efficient lidar is used instead. The laser point cloud and image are combined to describe the mining heritage scene in depth, and the academic achievements are transformed into productivity. The results show that (1) comparing the performance of the algorithm, the overall error of the proposed method is 5.86, and the average time consumption is about 10.23 ms. This filtering algorithm can restore the landscape of mining heritage to a great extent and optimize the problems of noise and outliers. (2) Our designed model has a very low loss value, and the point cloud accuracy is as high as 92.9%. Compared with other model methods, the model has excellent performance. (3) After the completion of the model, the overall satisfaction effect is above 70%, which can well restore the mining heritage style. Finally, the experimental effect of 3D reconstruction is good, which is more conducive to the research of reconstruction protection. There are still some work details and performance problems that need to be optimized and solved.

Research on Underground Coal Mine Map Construction Method Based on LeGO-LOAM Improved Algorithm

The application of intelligent equipment and technologies such as robots and unmanned vehicles is an important part of the construction of intelligent mines, and has become China’s national coal energy development strategy and the consensus of the coal industry. Environment perception and instant positioning is one of the key technologies destined to realize unmanned and autonomous navigation in underground coal mines, and simultaneous location and mapping (SLAM) is an effective method of deploying this key technology. The underground space of a coal mine is long and narrow, the environment is complex and changeable, the structure is complex and irregular, and the lighting is poor. This is a typical unstructured environment, which poses a great challenge to SLAM. This paper summarizes the current research status of underground coal mine map construction based on visual SLAM and Lidar SLAM, and analyzes the defects of the LeGO-LOAM algorithm, such as loopback detection errors or omissions. We use SegMatch to improve the loopback detection module of LeGO-LOAM, use the iterative closest point (ICP) algorithm to optimize the global map, then propose an improved SLAM algorithm, namely LeGO-LOAM-SM, and describe its principle and implementation. The performance of the LeGO-LOAM-SM was also tested using the KITTI dataset 00 sequence and SLAM experimental data collected in two coal mine underground simulation scenarios, and the performance indexes such as the map construction effect, trajectory overlap and length deviation, absolute trajectory error (ATE), and relative pose error (RPE) were analyzed. The results show that the map constructed by LeGO-LOAM-SM is clearer, has a better loopback effect, the estimated trajectory is smoother and more accurate, and the translation and rotation accuracy is improved by approximately 5%. This can construct more accurate point cloud map and low drift position estimation, which verifies the effectiveness and accuracy of the improved algorithm. Finally, to satisfy the navigation requirements, the construction method of a two-dimensional occupancy grid map was studied, and the underground coal mine simulation environment test was carried out. The results show that the constructed raster map can effectively filter out outlier noise such as dynamic obstacles, has a mapping accuracy of 0.01 m, and the required storage space compared with the point cloud map is reduced by three orders of magnitude. The research results enrich the SLAM algorithm and implementation in unstructured environments such as underground coal mines, and help to solve the problems of environment perception, real-time positioning, and the navigation of coal mine robots and unmanned vehicles.

A Matching Optimization Algorithm About Low-Altitude Remote Sensing Images Based on Geometrical Constraint and Convolutional Neural Network

This article presents a novel matching optimization algorithm for low-altitude remote sensing images based on a geometrical constraint and a convolutional neural network (CNN). The proposed method was designed to be effective in enhancing the integrity and accuracy of point clouds generated by stereo matching. To overcome the limitations of stereo matching, we trained a CNN to predict how well image patches match and used it in patch optimization. The main advantage of this approach is that the proposed algorithm can decrease the mismatching and errors caused by noise, deep discontinuity, and weak texture in low-altitude remote sensing images and can reconstruct an integrated and accurate point cloud. Comparative studies and experimental results validate the accuracy of the proposed algorithm when used for dense point generation from low-altitude remote sensing images.

GCN-Based Pavement Crack Detection Using Mobile LiDAR Point Clouds

Mobile Laser Scanning (MLS) system can provide high-density and accurate 3D point clouds that enable rapid pavement crack detection for road maintenance tasks. Supervised learning-based algorithms have been proved pretty effective for handling such a large amount of inhomogeneous and unstructured point clouds. However, these algorithms often rely on a lot of annotated data, which is labor-intensive and time-consuming. This paper presents a semi-supervised point-level approach to overcome this challenge. We propose a graph-widen module to construct a reasonable graph structure for point clouds, increasing the detection performance of graph convolutional networks (GCN). The constructed graph characterizes the local features from a small amount of annotated data, avoiding information loss and dramatically reduces the dependence on annotated data. The MLS point clouds acquired by a commercial RIEGL VMX-450 system are used in this study. The experimental results demonstrate that our method outperforms the state-of-the-art point-level methods in terms of recall, F1 score, and efficiency while achieving comparable accuracy.

A deep-learning-based in-situ surface anomaly detection methodology for laser directed energy deposition via powder feeding

Laser Directed Energy Deposition via Powder Fed (LDED-PF) based Metal Additive Manufacturing suffers from inferior dimensional accuracy mainly due to thermal cycling, localized heat accumulation, inconsistency in the powder stream, deviation in the speed of the motion system and powder focus, etc. In-situ monitoring, along with defect detection algorithms, can be used for avoiding unpredictable build failures while minimizing the time and cost of exhaustive ex-situ characterization techniques. Existing surface quality assessment techniques for LDED-PF mainly focus on intermittent process monitoring, data acquisition, data post-processing, feature extraction, and error identification. The present work investigates the development of a novel in-situ monitoring software platform that can be used for the surface anomaly detection of LDED-PF parts using machine learning techniques. Despite the existing methods in the literature, this technique has shown high robustness and high confidence in defect detection regardless of the geometry, and varied density of point clouds. First, a novel method is developed to calibrate the laser line scanner with respect to the robotic end-effector with the sub 0.5 mm accuracy. Subsequently, 2D surface profiles obtained from the LDED-PF built part surface using the laser scanner are stitched together to create an accurate 3D point cloud representation. Further, the point cloud data is processed, and defect detection is carried out using unsupervised learning and supervised (deep) learning techniques. The overall accuracy and mean IoU of 91.3 % and 83.3 % are achieved, respectively. The study paves the way for the development of automatic tool path generation for the LDED-PF process to build high-quality components.

PKS: A photogrammetric key-frame selection method for visual-inertial systems built on ORB-SLAM3

Key-frame selection methods were developed in the past years to reduce the complexity of frame processing in visual odometry (VO) and visual simultaneous localization and mapping (VSLAM) algorithms. Key-frames help increasing algorithm's performances by sparsifying frames while maintaining its accuracy and robustness. Unlike current selection methods that rely on many heuristic thresholds to decide which key-frame should be selected, this paper proposes a photogrammetric-based key-frame selection method built upon ORB-SLAM3. The proposed algorithm, named Photogrammetric Key-frame Selection (PKS), replaces static heuristic thresholds with photogrammetric principles, ensuring algorithm’s robustness and better point cloud quality. A key-frame is chosen based on adaptive thresholds and the Equilibrium Of Center Of Gravity (ECOG) criteria as well as Inertial Measurement Unit (IMU) observations. To evaluate the proposed PKS method, the European Robotics Challenge (EuRoC) and an in-house datasets are used. Quantitative and qualitative evaluations are made by comparing trajectories, point clouds quality and completeness and Absolute Trajectory Error (ATE) in mono-inertial and stereo-inertial modes. Moreover, for the generated dense point clouds, extensive evaluations, including plane-fitting error, model deformation, model alignment error, and model density and quality, are performed. The results show that the proposed algorithm improves ORB-SLAM3 positioning accuracy by 18% in stereo-inertial mode and 20% in mono-inertial mode without the use of heuristic thresholds, as well as producing a more complete and accurate point cloud up to 50%. The open-source code of the presented method is available at https://github.com/arashazimi0032/PKS.

Investigation of the effect of height difference and geometry of GCP on position accuracy of point cloud in UAV photogrammetry

In this study, the effect of the height difference, geometry, and number of GCPs on the positional accuracy of point cloud was investigated in UAV photogrammetry. It has been determined that the topographic change of the study area, the geometric structure of GCPs, and the distance between GCPs are the most important factors in GCP network design. It was observed that the GCP network design is more important than the number of GCPs in the UAV Photogrammetry method to increase the positional accuracy.

Conservation of Balinese Customary Buildings with BIM Technology Approach

Digitizing architectural heritage information has become an important task during these past years. However, only a few studies have addressed the digitization of historic building models for conservation and restoration purposes. As historical building conservation began to decline, many historical buildings are being abandoned and converted into modern facilities in response to our changing needs. The conservation attempt can be made simple by utilizing the Building Information Modeling (BIM), and High Definition Survey (HDS) approaches. These technologies are considered brand new in managing important information related to 3D models, providing us with semantic information and accurate information in both position and elevation. HDS data is constructed in the form of point clouds generated by the collaboration of survey equipment such as GNSS, Drones, Terrestrial Laser Scanning, and high-performance computers. Point cloud data is registered, filtered, and georeferenced before using BIM modeling. This study resulted from registering point cloud data with RMSE 3mm and customary semantic information. The BIM model based on accurate point cloud data is expected to be a solution and a reference in carrying out the conservation and reconstruction of historical buildings in the future.

Integrating 3D cameras into sterile surgical environments: A comparison of different protective materials regarding scan accuracy

Abstract This work presents a sterile concept for 3D cameras within sterile surgical environments. In the digital operating room (OR), such cameras can serve as a valuable data source for cognitive workflow assistance systems, e.g decision or mechatronic support systems. One recent example are robotic assistants for instrument handling, such as the robotic scrub nurse currently developed in the framework of the SASHA-OR research project1. In this context, we detect laparoscopic instruments and the surgical environment with a 3D camera, whereby hygienic requirements need to be met. Using a Zivid Two sensor, we generated point clouds of the laparoscopic instruments located in an instrument holder and a drop zone. We compared the effect of using different pane types and thicknesses for the sterile camera enclosure and compared the performance with and without protective pane in terms of the point cloud accuracy. When analyzing multiple pane types, polymethyl methacrylate with 0.5 mm thickness (PMMA 0.5) provided the best results. At a scan distance of 560 mm to the surface center, which is required for the complete acquisition of a laparoscopic instrument, PMMA 0.5 achieved the smallest Chamfer distance (CD) values for both the scans with the laparoscopic instruments in the instrument holder (0.23 ± 1.52 mm) and in the drop zone (0.12 ± 0.25 mm).

Mapping Sensitive Vegetation Communities in Mining Eco-space using UAV-LiDAR

Near earth sensing from uncrewed aerial vehicles or UAVs has emerged as a potential approach for fine-scale environmental monitoring. These systems provide a cost-effective and repeatable means to acquire remotely sensed images in unprecedented spatial detail and a high signal-to-noise ratio. It is increasingly possible to obtain both physiochemical and structural insights into the environment using state-of-art light detection and ranging (LiDAR) sensors integrated onto UAVs. Monitoring sensitive environments, such as swamp vegetation in longwall mining areas, is essential yet challenging due to their inherent complexities. Current practices for monitoring these remote and challenging environments are primarily ground-based. This is partly due to an absent framework and challenges of using UAV-based sensor systems in monitoring such sensitive environments. This research addresses the related challenges in developing a LiDAR system, including a workflow for mapping and potentially monitoring highly heterogeneous and complex environments. This involves amalgamating several design components, including hardware integration, calibration of sensors, mission planning, and developing a processing chain to generate usable datasets. It also includes the creation of new methodologies and processing routines to establish a pipeline for efficient data retrieval and generation of usable products. The designed systems and methods were applied to a peat swamp environment to obtain an accurate geo-spatialised LiDAR point cloud. Performance of the LiDAR data was tested against ground-based measurements on various aspects, including visual assessment for generation LiDAR metrices maps, canopy height model, and fine-scale mapping.

EVALUATION OF A COMPACT HELMET-BASED LASER SCANNING SYSTEM FOR ABOVEGROUND AND UNDERGROUND 3D MAPPING

Abstract. As a strategic resource, urban underground space can be used for rail transportation, commercial streets, which has high economic and social benefits, and is of great significance to sustainable city development. Due to denied Global Navigation Satellite System (GNSS) signal, traditional mobile mapping systems have difficulty collecting accurate 3D point clouds in urban underground space. Thus, a helmet-based laser scanning system, named "WHU-Helmet", is integrated in this paper to make up for the shortcomings of the existing traditional mobile mapping systems. "WHU-Helmet" is mainly equipped with four types of sensors: a GNSS receiver (optional), an IMU, a laser scanner, and a global shutter camera. "WHU-Helmet" is not relying on GNSS signal and has the advantages of low cost, small volume and easy operation. Using "WHU-Helmet", a multi-scale Normal Distributions Transform (NDT) based LiDAR-IMU SLAM is implemented to collect underground 3D point cloud in real-time. To validate the performance of "WHU-Helmet" in aboveground and underground 3D mapping, experiments were conducted in a typical urban metro station. The experiments show that the average and RMSE of HLS point errors of "WHU-Helmet" are 0.44 meters and 0.23 meters, respectively, showing great potential of "WHU-Helmet" in the application of aboveground and underground 3D mapping.

A TWO-STAGE APPROACH FOR RARE CLASS SEGMENTATION IN LARGE-SCALE URBAN POINT CLOUDS

Abstract. Although deep learning has greatly improved the semantic segmentation accuracy of point clouds, the segmentation of rare classes in large-scale urban scenes has not been targeted in available methods. This paper proposes a two-stage segmentation framework with automated workflows for imbalanced rare classes based on general semantic segmentation. The proposed approach includes two stages: general semantic segmentation and object-based refined semantic segmentation. Firstly, general segmentation networks are utilized to segment general large objects. Secondly, refined semantic segmentation is conducted by an automated workflow: 3D clustering and bounding box (BBox) generation are applied to the point cloud of rare fine-grained objects during the training, followed by object detection to extract fine-grained objects. Afterwards, as the constraints, the extracted BBoxes further refine the segmentation results. Our approach is evaluated on the Hessigheim High-Resolution 3D Point Cloud (H3D) Benchmark and obtains state-of-the-art 89.35% overall accuracy and outstanding 75.70% mean F1-Score. Furthermore, rare classes Vehicle and Chimney achieve breakthroughs from zero to 63.63% and 52.00% in F1-score, respectively.

A low-cost approach for the estimation of rock joint roughness using photogrammetry

To evaluate the possibility of utilizing commercially available devices for making joint roughness estimates, a digital single-lens reflex (DSLR) camera and a smartphone were used to reconstruct the three-dimensional models of 10 rock-joint specimens. The accuracy and efficiency were considered in investigating the performance of photogrammetry on the point clouds production and joint roughness estimation in the laboratory. Additionally, a handheld laser scanner was used in collecting high-resolution point clouds of the same specimens as reference data for evaluating the photogrammetry measurement accuracy. The interval for image capture was set at 8° with consideration for the data quality, point density, and computation time. The iterative closest point (ICP) algorithm was used to compare the surface topographies of the rock joints produced via photogrammetry and laser scanning. The point clouds generated by photogrammetry agreed with the laser-scanning results, with a small average difference (0.005 mm for the DSLR camera and 0.006 mm for the smartphone). Furthermore, two- and three-dimensional surface parameters were calculated using the three different sources of point clouds, which were in good agreement in the joint roughness estimates when comparing photogrammetry and laser scanning results. Comparative analyses revealed that both DSLR camera photogrammetry and smartphone photogrammetry can produce accurate point clouds of rock joints for roughness assessment; however, point clouds generated by the DSLR camera had higher accuracies than those generated by the smartphone.