High-precision Point Cloud Research Articles

Semantic segmentation of point clouds of ancient buildings based on weak supervision

Semantic segmentation of point clouds of ancient buildings plays an important role in Historical Building Information Modelling (HBIM). As the annotation task of point cloud of ancient architecture is characterised by strong professionalism and large workload, which greatly restricts the application of point cloud semantic segmentation technology in the field of ancient architecture, therefore, this paper launches a research on the semantic segmentation method of point cloud of ancient architecture based on weak supervision. Aiming at the problem of small differences between classes of ancient architectural components, this paper introduces a self-attention mechanism, which can effectively distinguish similar components in the neighbourhood. Moreover, this paper explores the insufficiency of positional encoding in baseline and constructs a high-precision point cloud semantic segmentation network model for ancient buildings—Semantic Query Network based on Dual Local Attention (SQN-DLA). Using only 0.1% of the annotations in our homemade dataset and the Architectural Cultural Heritage (ArCH) dataset, the mean Intersection over Union (mIoU) reaches 66.02% and 58.03%, respectively, which is an improvement of 3.51% and 3.91%, respectively, compared to the baseline.

HPNet: High precision point cloud registration using feature pyramid and hybrid position encoding

Point cloud registration precision degrades when the scene contains symmetric structures or repeated patches. Also, the common operation of continuous downsampling in existing registration methods causes the loss of detailed information and leads to further registration precision degradation. In this paper, we propose a high precision point cloud registration network (HPNet) to learn discriminative features by considering both multi-scale information and position information. We first propose a multi-scale feature extraction module similar to the feature pyramid that allows the extracted features to contain multi-scale information. Then, an information interaction is performed on the features to learn global contextual information by using Transformer with a hybrid position encoding, which takes into account both absolute and relative positions of points. Finally, the features obtained from the information interaction module are directly used to predict the point correspondences. Comprehensive experiments on 3DMatch, 3DLoMatch, ModelNet, and ModelLoNet datasets demonstrate the state-of-the-art performance of the proposed method and the implementation of HPNet in SLAM shows its effectiveness in real application.

High-Precision Map Construction in Degraded Long Tunnel Environments of Urban Subways

In response to the demand for high-precision point cloud mapping of subway trains in long tunnel degradation scenarios in major urban cities, we propose a map construction method based on LiDAR and inertial measurement sensors. This method comprises a tightly coupled frontend odometry system based on error Kalman filters and backend optimization using factor graphs. In the frontend odometry, inertial calculation results serve as predictions for the filter, and residuals between LiDAR points and local map plane point clouds are used for filter updates. The global pose graph is constructed based on inter-frame odometry and other constraint factors, followed by a smoothing optimization for map building. Multiple experiments in subway tunnel scenarios demonstrate that the proposed method achieves robust trajectory estimation in long tunnel scenes, where classical multi-sensor fusion methods fail due to sensor degradation. The proposed method achieves a trajectory consistency of 0.1 m in tunnel scenes, meeting the accuracy requirements for train arrival, parking, and interval operations. Additionally, in an industrial park scenario, the method is compared with ground truth provided by inertial navigation, showing an accumulated error of less than 0.2%, indicating high precision.

Dynamic Spatial–Spectral Feature Optimization-Based Point Cloud Classification

With the development and popularization of LiDAR technology, point clouds are becoming widely used in multiple fields. Point cloud classification plays an important role in segmentation, geometric analysis, and vegetation description. However, existing point cloud classification algorithms have problems such as high computational complexity, a lack of feature optimization, and low classification accuracy. This paper proposes an efficient point cloud classification algorithm based on dynamic spatial–spectral feature optimization. It can eliminate redundant features, optimize features, reduce computational costs, and improve classification accuracy. It achieves feature optimization through three key steps. First, the proposed method extracts spatial, geometric, spectral, and other features from point cloud data. Then, the Gini index and Fisher score are used to calculate the importance and relevance of features, and redundant features are filtered. Finally, feature importance factors are used to dynamically enhance the discriminative power of highly distinguishable features to strengthen their contribution to point cloud classification. Four real-scene datasets from STPLS3D are utilized for experimentation. Compared to the other five algorithms, the proposed algorithm achieves at least a 37.97% improvement in mean intersection over union (mIoU). Meanwhile, the results indicate that the proposed algorithm can achieve high-precision point cloud classification with low computational complexity.

A Method for Industrial Robots to Grasp and Detect Parts of Instrument under 3D Visual Guidance

<p>Guiding industrial robots to complete grasping tasks through machine vision is an important part of achieving autonomous robot operation. This article explores the control method of industrial robots under 3D vision, focusing on the feature that two-dimensional vision can only perform color and pose recognition but lacks depth recognition. Firstly, a high-precision point cloud registration calibration matrix solution method is proposed. Then, an improved recognition model is designed to address the issue of how vision guides robots to grasp and detect. This model integrates feature extraction and object detection modules, and describes the parameters of each module. Finally, the effectiveness of the proposed method is verified in the assembly scene of gas instruments. Finally, experimental results show that, the method proposed in this article can limit the grasping accuracy to within 2 millimeters in guiding robots to grasp detection scenes, achieving the expected effect.</p> <p> </p>

High-precision point cloud registration method based on volume image correlation

With the rapid development of binocular reconstruction, fringe projection profilometry, and time of flight, 3D imaging technology has been widely applied in the field of 3D measurement. However, due to limited measurement range and self-occlusion, point cloud registration methods are often used to obtain larger or more complete 3D contours. Although many scholars have proposed various point cloud registration methods, the accuracy and efficiency of point cloud registration still need to be further improved, especially for point clouds with different density or non-rigid transformation. Image registration technology based on image correlation has been developed for many years and has achieved great success in fields such as computer vision, photomechanics, and photogrammetry. Therefore, a simple and direct idea in this paper is to transform the point cloud registration problem into volume image correlation problem. By this, an efficient image registration method based on fast Fourier transform and an inverse compositional Gaussian Newton optimization method that only needs to calculate the Hessian matrix once can be introduced into the point cloud registration field, which can greatly improve the speed and accuracy of point cloud registration. Comparative experiments have shown that our method has doubled the accuracy and efficiency compared to the iterative closest point (ICP) method, and its practicality has also been verified in impeller reconstruction experiments.

Data-driven strategy for geometric imperfection analysis of assembled large-span ring truss components

Assembled large-span ring truss, constructed using circular steel tubes, is gaining popularity as a new structural form. However, the insufficient availability of design and modeling methods for geometric imperfection of components poses a challenge in studying the cumulative effects of these imperfections on the overall structural performance. Therefore, a data-driven strategy for analyzing geometric imperfections in assembled ring truss structural components (circular steel tubes) is presented in this study. The primary objective is to enable the design of geometric imperfections based on measured data and establish a corresponding equivalent calculation model to incorporate the measured imperfections into the structural analysis model. The core of this strategy lies in utilizing the high-precision point cloud data of components obtained through 3D laser scanning technology as the primary input. The geometric imperfection modes are extracted accurately and rapidly by employing a developed algorithm. Furthermore, a comprehensive framework is proposed for establishing the geometric imperfection calculation model based on the identified modes. A case study was conducted to validate the proposed strategy, revealing its high reliability. Moreover, it is recommended to extend this strategy for analyzing geometric imperfections in similar building structural members.

MuCoGraph: A multi-scale constraint enhanced pose-graph framework for MLS point cloud inconsistency correction

The position consistency of mobile laser scanning (MLS) point clouds is crucial for large-scale applications, and is normally guaranteed by the global navigation satellite system (GNSS) and high-precision inertial measurement unit (IMU) in the data acquisition. However, GNSS-denied environments such as city valleys result in significant position inconsistency for overlapping areas, and it is difficult to automatically locate these inconsistent areas. In this paper, to overcome these problems, we present MuCoGraph, which introduces multi-scale constraints to establish the correct correspondences for revisited areas, and formulates an enhanced pose graph for position inconsistency correction. The georeferenced MLS point cloud is first sliced into segments adaptively, and these segments are then abstracted as graph vertices, which satisfy local geometric consistency and rigid transformation hypotheses. Accurate revisited graph edges are then constructed hierarchically under multi-scale scenery consistency constraints. These revisited edges are initialized based on feature-based correspondence estimation and further unreliable edge pruning. Finally, through combination with virtual co-observations, correspondence-enhanced pose-graph optimization is introduced to globally redistribute the errors and obtain a high-precision point cloud. The proposed method was used to correct the MLS point cloud position inconsistency in three datasets. The average three-dimensional distance of the checkpoints was reduced from 0.362 m, 0.108 m, and 1.027 m to 0.057 m, 0.033 m, and 0.051 m for datasets I, II, and III respectively. In addition, the root-mean-square error of all three datasets was less than 0.04 m after correction. The experiments confirmed that the proposed method can automatically locate and correct the position inconsistency of MLS point clouds, showing good robustness and effectiveness.

PointCNT: A One-Stage Point Cloud Registration Approach Based on Complex Network Theory

Inspired by the parallel visual pathway model of the human neural system, we propose an efficient and high-precision point cloud registration method based on complex network theory (PointCNT). A deep learning network (DNN) design method based on complex network theory is proposed, and a multipath feature extraction network, namely, Complex Kernel Point Convolution Neural Network (ComKP-CNN) for point clouds is designed based on the design method. Self-supervision is introduced to improve the feature extraction ability of the model. A feature embedding module is proposed to explicitly embed the transformation-variant coordinate information and transformation-invariant distance information into features. A feature fusion module is proposed to enable the source and template point clouds to perceive each other’s nonlocal features. Finally, a Multilayer Perceptron (MLP) with prominent fitting characteristics is utilized to estimate the transformation matrix. The experimental results show that the Registration Recall (RR) of PointCNT on ModelNet40 dataset reached 96.4%, significantly surpassing one-stage methods such as Feature-Metric Registration (FMR) and approaching two-stage methods such as Geometric Transformer (GeoTransformer). The computation speed is faster than two-stage methods, and the registration run time is 0.15 s. In addition, ComKP-CNN is universal and can improve the registration accuracy of other point cloud registration methods.

Deep learning for filtering the ground from ALS point clouds: A dataset, evaluations and issues

The capability of partially penetrating vegetation canopy and efficiently collecting high-precision point clouds over large areas makes airborne laser scanning (ALS) a valuable tool for various geospatial applications. However, automated ground filtering (GF), one fundamental and challenging step for most ALS applications, has remained a widely researched yet unsolved problem for decades. The recent breakthroughs in supervised deep learning (DL) techniques, which rely on sufficient and high-quality labeled datasets, provide a new solution to better solve this problem. Unfortunately, public 3D geospatial datasets are scarce, especially for those tailored for the landform-scale GF task. Moreover, whether advanced deep neural networks (DNNs) can be well-scaled to the problem of GF remains an open question. To comprehensively advance the development of effective DL-based GF pipelines, we first publish an ultra-large-scale GF dataset built upon open-access ALS point clouds of four different countries worldwide, which covers over 47 km2 and nine different terrain scenes. Then, multiple attractive advantages of DL techniques in GF are evaluated through extensive experimental comparisons with traditional GF methods on the presented dataset. Furthermore, we reveal several issues faced by generalizing existing advanced 3D DNNs into GF tasks with a series of in-depth experimental analyses. Finally, some promising directions for future research are suggested in response to the identified challenges. Our dataset, named OpenGF, is available at https://github.com/Nathan-UW/OpenGF.

PERFORMANCE OF LOCALIZATION BASED ON VISUAL SLAM AND LIDAR POINT CLOUD REGISTRATION

Abstract. Self-driving car technology has become increasingly popular in recent years. Traditionally, these cars rely on GNSS and INS, but limitations in urban environments can affect their effectiveness. GNSS technology has developed to address this issue, but it still has limitations in certain environments. LiDAR technology has emerged as a solution to obtain high-precision point cloud maps and SLAM technology is now used to integrate these maps with sensor data, such as cameras, to achieve precise positioning. This enables visual SLAM to be performed in environments where GNSS signals are blocked. In this research, we aim to use SLAM technology to obtain posterior environmental information and match it with prior high-precision point cloud maps. The research will start with hardware configuration, actual measurement and analysis of SLAM algorithms, and 3D point cloud matching methods. Results and benefits will be analysed to compare the advantages and disadvantages of point cloud matching algorithms and applicable hardware.

High-precision point cloud registration system of multi-view industrial self-similar workpiece based on super-point space guidance

High-precision point cloud registration system of multi-view industrial self-similar workpiece based on super-point space guidance

Local-to-global structure prior guided high-precision point cloud registration framework based on FPP

As one of the popular point cloud reconstruction technologies, fringe projection profilometry (FPP) has great application prospects in point cloud registration from different viewing angles. While the existing point cloud registration algorithm that focuses on FPP is limited to high-speed fringe projection system. Besides, the deep learning methods based on the global feature retrieval mechanism can hardly mine the multi-modal data features of FPP, and are difficult to meet the accuracy requirements of FPP. To address the above issue, we design a local–global structure prior guided high-precision registration framework that focuses on the FPP data characteristics. Specifically, the consistent clustering matching (CCM) module is proposed firstly to obtain the structure priors of cluster correspondences in the overlap regions by analyzing the FPP multi-modal data. Then the local-cluster interaction network (LCINet) guided by the structure priors is introduced for feature extraction and interaction. Finally, the spatial alignment module based on cluster voting(SACV) is proposed to select cluster correspondences with high confidence to calculate the transformation matrix and align point clouds. Experiments show that our method achieves state-of-the-art performance on the dataset collected by FPP, and our framework can also greatly improve the training efficiency and performance of other deep learning models.

Construction Quality Inspection Method of Substation Fabricated Structure Based on 3D Laser Scanning

The substation construction project is highly professional and comprehensive, requiring multiple disciplines’ cooperation. It is greatly affected by geographical and environmental factors and has the characteristics of high management requirements and difficult quality control in the construction process. Based on the good initial position of the point cloud to be registered, if this requirement cannot be met, the registered point cloud data is prone to the optimal local solution. Thus, this paper introduces three-dimensional laser scanning to detect the construction quality of the substation assembly structure. On this basis, the data registration method realizes the accurate point cloud registration, and the data’s main feature vector is extracted through a linear transformation. Through the spatial transformation of the principal axis direction, the registration of the point cloud number is completed. The problem of the optimal local solution of registration is effectively solved. The point cloud data model is created by actual data acquisition, and the influencing factors of reverse construction of the geometric model are analyzed. The specific implementation scheme is put forward to ensure the accuracy of the geometric model construction. The test verified the reasonable and effective application of 3D laser scanning in the prefabricated building installation. Besides, the effective method of the high-precision point cloud model data output is determined.

3D MODEL OF BUILDING BASED ON MULTI-SOURCE DATA FUSION

Abstract. The process of building a digital city is the construction of a three-dimensional city model. The construction of 3 d data model is the process of multi-source spatial information collection and fusion. The current means of spatial information collection are mainly divided into three-dimensional laser technology and UAV photogrammetry technology. The three-dimensional laser technology is mainly based on ground laser, which has high acquisition accuracy for the bottom part of the building, but it is insufficient for the information collection at the top of the building. UAV photogrammetry has a large area and high efficiency spatial information collection method. At the same time, a large three-dimensional city model of the building can also be constructed. However, due to practical problems such as flight height and ground occlusion, the model constructed by UAV photogrammetry has a perfect top and poor bottom quality. The two methods are combined to build a more perfect point cloud model. Aiming at the above problems, a three-dimensional modeling method of buildings under multi-source data fusion is proposed. Firstly, the point cloud data and image data obtained by 3D laser scanner and UAV are preprocessed, and the two data forms are transformed into high-precision point cloud data. Then two sets of point cloud data are fused to generate the whole point cloud data. Finally, based on the patch generated by the whole point cloud, the point cloud model is reverse modeled. Get the corresponding building entity model, in order to get a more perfect 3D city model.

Research on non-concentric spatial scanning measurement system

For the requirement of 3D coordinate measurement of structures, a novel non-concentric measurement system is proposed. This measurement system consists of a laser rangefinder and a two-dimensional rotating table, in which the rotation center of rotating table does not coincide with the light-emitting point of the laser rangefinder. Additionally, a mathematical model for distance correction is established in order to control systematic errors and meet high accuracy demands. Then, two calibration parameters of this system is defined and corresponding calibration method is introduced. Finally, a non-concentric scanning system is constructed and a comparison experiment with laser scanners is conducted. Experimental results demonstrate that this new system can obtain the same high precision point clouds and monitoring deformation within the measurement distance of 44 m. That means the system can meet the high-precision measurement requirements of structures. Compared with traditional laser scanners, this system is easy to set up and has fewer parameters. As a result, the cost price can be controlled and the calibration process is more convenient so as to be employed in monitoring structure deformation under complex conditions.

Remote inspection method for water conveyance tunnel based on single-beam scanning sonar

Water conveyance tunnels perform an important role in water-diversion projects. With a water conveyance tunnel transferring water, the conflict between urban crowded ecology and agricultural water can be relieved. Research shows that there is a significant need for a more objective and accurate inspection of water conveyance tunnels during operation. Hence, we investigate the use of a single-beam scanning sonar to detect a water conveyance tunnel and generate a three-dimensional point cloud model, which can be used to make objective predictions for surface defects. First, a cobweb-like grid matrix was proposed to grid discrete points to establish the topological relationship of the point cloud. Second, a denoising method fusing acoustic properties and intensity was used to filter the noise in the acoustic point cloud. Third, an optimal echo extraction technology based on a second-order differential model was proposed to obtain a high-precision point cloud of the tunnel. Experiments show that the proposed denoising method and optimal echo extraction method are superior to traditional algorithms commonly used in ground point clouds, with good results achieved at real test sites.

An improved space colonization algorithm with DBSCAN clustering for a single tree skeleton extraction

ABSTRACT The tree skeleton is a powerful tool for reasoning about the structure of forest, because of its ability to describe the topology and geometry of a tree in a simple and compact form. Terrestrial Laser Scanning (TLS) is one of the modern precision mapping techniques, it can provide high-precision point cloud data for digitization studies of forests. Accurate extraction of the stem and branch skeleton from TLS data is of great importance for studying the morphological structure and 3D visualization of trees. Although good progress has been made in the study of extracting the skeleton of 3D trees, there are still some problems, such as difficult to implement calculations on a large number of point clouds. Considering these problems, in this paper, cherry and begonia trees in a plot at Baima campus of Nanjing Forestry University are selected as the research object, and the space colonization algorithm is directly applied to the TLS point cloud data, but we find that the space colonization method would lead to over-extension of the skeletal trunk of the trees. Therefore, improvements are made on the basis of the original algorithm. Pre-processing the point cloud data with noise filtering and identifying the number of bifurcations using the DBSCAN clustering in advance makes the experimental results clearer visually. We select trees of different shapes within the plots for skeleton extraction and parameter validation was performed to illustrate the effectiveness of the improved method. The proposed method extracts the skeleton in the point cloud of 100 reference trees. The comparative measurement experiments with real values were performed, the root mean square error (RMSE) for the branch angle, trunk length and branch length were 3.309 degree, 0.069 m, 0.051 m respectively. These results demonstrate that the proposed method can accurately extract the skeleton points of a single tree.

3D-CSTM: A 3D continuous spatio-temporal mapping method

3D mapping is now essential for urban infrastructure resource monitoring, autonomous driving, and the fulfillment of digital earth. LiDAR-based Simultaneous Localization And Mapping (SLAM) technology has been widely studied because of its efficient application in the reconstruction of the 3D environments that benefitted from its sensor characteristic. However, the sparsity of low cost LiDAR (Light Detection And Ranging) sensor provides great challenge for it. Insufficient or poor distribution of the constraints as well as the motion distortion in one scan of point cloud would deteriorate the pose estimation in SLAM. In this paper, a novel 3D mapping method based only on the LiDAR data is proposed to enhance the performance of SLAM-based 3D point clouds map reconstruction. The proposed system consists of two main modules: localization module for rapid ego-motion estimation within the surrounding environment, and mapping module for accurate point clouds fusion. As for the localization, scan-to-map point cloud registration scheme are adopted where structural features are firstly extracted to provide valid correspondences. What’s more, an effective method is figured out to balance the cost factors for the optimization function of pose estimation in the process. This further handles the degenerated cases for cloud registration. Focusing on improving the quality of the map, a spline motion model is integrated with non-rigid point cloud fusion to facilitate the continuous and nonrepetitive mapping of the environment. Extensive experiments are carried out for the verification of our system including large-scale outdoor and long-corridor scenes. According to the experimental results, our system has achieved 0.78‰closure error and 27.5% improvement in the APE (Absolute Pose Errors), outperforming the state-of-the-art LiDAR-based SLAM (e.g. LeGO-LOAM) frameworks in our test sites. To specifically verify the performance of 3D mapping, the precision of 3D points is quantified through comparing with the high-precision point cloud maps collected by MLS (Mobile Laser Scanning) and TLS (Terrestrial Laser Scanning). The RMS of point-to-plane distances have improved 20% based on the fitted Weibull distribution. In addition, some ablation tests are conducted to reveal the efficacy of different components in our system.

A novel point cloud simplification method with integration of multiple-feature fusion and density uniformity

In recent years, 3D point cloud technology has played an increasingly important role in the field of casting parts detection. Due to the huge amount of high-precision point cloud data, simplification is often necessary before 3D reconstruction to improve computational efficiency. However, most of the existing point cloud simplification algorithms often blur edge details and produce holes in the final 3D reconstruction. To avoid these problems, a novel point cloud simplification method is proposed here. To accurately describe the geometric and texture features, a multi-featured fusion method was first developed to integrate the curvature, local projection distance, normal vector and local color differences. As this approach uses multiple types of geometric features, it accurately distinguishes sharp and smooth edges. Moreover, due to the use of local color difference, the texture features of the point cloud are recognizable. Then, to evaluate the density of the non-uniform point cloud, a globular neighborhood search strategy was used to estimate the density of the point cloud itself. Since the number of points in the spherical neighborhood reflected the sparsity of the point cloud, this method accurately represented the density of a point cloud with an uneven distribution. Finally, the feature fusion and density uniformity were integrated, allowing the use of these preserved points to retain more edge details of the point cloud in the final 3D reconstruction. This approach also avoided holes in the final 3D reconstruction. On the point cloud data of Anchor, Fantisk and Humana, the root mean square error of the proposed method reduces 40%, 32% and 19% than the best algorithm among the K-means clustering, Grid average, and Graph based method, respectively.