Point Cloud Information Research Articles

Lidar Simultaneous Localization and Mapping Algorithm for Dynamic Scenes

To address the issue of significant point cloud ghosting in the construction of high-precision point cloud maps by low-speed intelligent mobile vehicles due to the presence of numerous dynamic obstacles in the environment, which affects the accuracy of map construction, this paper proposes a LiDAR-based Simultaneous Localization and Mapping (SLAM) algorithm tailored for dynamic scenes. The algorithm employs a tightly coupled SLAM framework integrating LiDAR and inertial measurement unit (IMU). In the process of dynamic obstacle removal, the point cloud data is first gridded. To more comprehensively represent the point cloud information, the point cloud within the perception area is linearly discretized by height to obtain the distribution of the point cloud at different height layers, which is then encoded to construct a linear discretized height descriptor for dynamic region extraction. To preserve more static feature points without altering the original point cloud, the Random Sample Consensus (RANSAC) ground fitting algorithm is employed to fit and segment the ground point cloud within the dynamic regions, followed by the removal of dynamic obstacles. Finally, accurate point cloud poses are obtained through static feature matching. The proposed algorithm has been validated using open-source datasets and self-collected campus datasets. The results demonstrate that the algorithm improves dynamic point cloud removal accuracy by 12.3% compared to the ERASOR algorithm and enhances overall mapping and localization accuracy by 8.3% compared to the LIO-SAM algorithm, thereby providing a reliable environmental description for intelligent mobile vehicles.

Three-dimensional colour reconstruction of aviation spiral bevel gear tooth surface through fusion of image and point cloud information

Three-dimensional colour reconstruction of aviation spiral bevel gear tooth surface through fusion of image and point cloud information

A Fast Obstacle Detection Algorithm Based on 3D LiDAR and Multiple Depth Cameras for Unmanned Ground Vehicles

With the advancement of technology, unmanned ground vehicles (UGVs) have shown increasing application value in various tasks, such as food delivery and cleaning. A key capability of UGVs is obstacle detection, which is essential for avoiding collisions during movement. Current mainstream methods use point cloud information from onboard sensors, such as light detection and ranging (LiDAR) and depth cameras, for obstacle perception. However, the substantial volume of point clouds generated by these sensors, coupled with the presence of noise, poses significant challenges for efficient obstacle detection. Therefore, this paper presents a fast obstacle detection algorithm designed to ensure the safe operation of UGVs. Building on multi-sensor point cloud fusion, an efficient ground segmentation algorithm based on multi-plane fitting and plane combination is proposed in order to prevent them from being considered as obstacles. Additionally, instead of point cloud clustering, a vertical projection method is used to count the distribution of the potential obstacle points through converting the point cloud to a 2D polar coordinate system. Points in the fan-shaped area with a density lower than a certain threshold will be considered as noise. To verify the effectiveness of the proposed algorithm, a cleaning UGV equipped with one LiDAR sensor and four depth cameras is used to test the performance of obstacle detection in various environments. Several experiments have demonstrated the effectiveness and real-time capability of the proposed algorithm. The experimental results show that the proposed algorithm achieves an over 90% detection rate within a 20 m sensing area and has an average processing time of just 14.1 ms per frame.

Development of an Autonomous Mobile Robot to Transport Heavy Objects on Narrow Alleys and Steep Slopes

ABSTRACTIn this study, we developed a four‐wheel‐driven autonomous mobile robot to transport heavy objects on narrow alleys and steep slopes. The robot, which weighs 450 kg, is driven by four 650 W hub motors to climb 20° steep slopes and travel through a narrow alley. Two 2D‐LiDARs attached to the front and rear of the robot body are used for highly accurate driving. A method based on the least squares approach for estimating the position and posture of the robot body is proposed by using the point cloud information of the side wall of the narrow alley. Field experiments on test courses demonstrated that the autonomous robot could transport heavy objects over steep slopes and travel in a narrow alley with high accuracy.

GETr: A Geometric Equivariant Transformer for Point Cloud Registration

AbstractAs a fundamental problem in computer vision, 3D point cloud registration (PCR) aims to seek the optimal transformation to align point cloud pairs. Meanwhile, the equivariance lies at the core of matching point clouds at arbitrary pose. In this paper, we propose GETr, a geometric equivariant transformer for PCR. By learning the point‐wise orientations, we decouple the coordinate to the pose of the point clouds, which is the key to achieve equivariance in our framework. Then we utilize attention mechanism to learn the geometric features for superpoints matching, the proposed novel self‐attention mechanism encodes the geometric information of point clouds. Finally, the coarse‐to‐fine manner is used to obtain high‐quality correspondence for registration. Extensive experiments on both indoor and outdoor benchmarks demonstrate that our method outperforms various existing state‐of‐the‐art methods.

TAG-fusion: Two-stage attention guided multi-modal fusion network for semantic segmentation

In the current research, leveraging auxiliary modalities, such as depth information or point cloud information, to improve RGB semantic segmentation has shown significant potential. However, existing methods mainly use convolutional modules for aggregating features from auxiliary modalities, thereby lacking sufficient exploitation of long-range dependencies. Moreover, fusion strategies are typically limited to singular approaches. In this paper, we propose a transformer-based multimodal fusion framework to better utilize auxiliary modalities for enhancing semantic segmentation results. Specifically, we employ a dual-stream architecture for extracting features from RGB and auxiliary modalities, respectively. We incorporate both early fusion and deep feature fusion techniques. At each layer, we introduce mixed attention mechanisms to leverage features from other modalities, guiding and enhancing the current modality's features before propagating them to the subsequent stage of feature extraction. After the extraction of features from different modalities, we employ an enhanced cross-attention mechanism for feature interaction, followed by channel fusion to obtain the final semantic features. Subsequently, we provide separate supervision to the network on the RGB stream, auxiliary stream, and fusion stream to facilitate the learning of representations for different modalities. The experimental results demonstrate that our framework exhibits superior performance across diverse modalities. Specifically, our approach achieves state-of-the-art results on the NYU Depth V2, SUN-RGBD, DELIVER and MFNet datasets.

GRNet:Geometry Restoration for G-PCC Compressed Point Clouds Using Auxiliary Density Signaling.

The lossy Geometry-based Point Cloud Compression (G-PCC) inevitably impairs the geometry information of point clouds, which deteriorates the quality of experience (QoE) in reconstruction and/or misleads decisions in tasks such as classification. To tackle it, this work proposes GRNet for the geometry restoration of G-PCC compressed large-scale point clouds. By analyzing the content characteristics of original and G-PCC compressed point clouds, we attribute the G-PCC distortion to two key factors: point vanishing and point displacement. Visible impairments on a point cloud are usually dominated by an individual factor or superimposed by both factors, which are determined by the density of the original point cloud. To this end, we employ two different models for coordinate reconstruction, termed Coordinate Expansion and Coordinate Refinement, to attack the point vanishing and displacement, respectively. In addition, 4-byte auxiliary density information is signaled in the bitstream to assist the selection of Coordinate Expansion, Coordinate Refinement, or their combination. Before being fed into the coordinate reconstruction module, the G-PCC compressed point cloud is first processed by a Feature Analysis Module for multiscale information fusion, in which kNN-based Transformer is leveraged at each scale to adaptively characterize neighborhood geometric dynamics for effective restoration. Following the common test conditions recommended in the MPEG standardization committee, GRNet significantly improves the G-PCC anchor and remarkably outperforms state-of-the-art methods on a great variety of point clouds (e.g., solid, dense, and sparse samples) both quantitatively and qualitatively. Meanwhile, GRNet runs fairly fast and uses a smaller-size model when compared with existing learning-based approaches, making it attractive to industry practitioners.

Deformation measurement of aero-engine rotating twisted blade based on multi-vision overlapping area feature registration method

Due to the complexity of the spatial surface of the twisted blade of the aero-engine, there are occlusions and shadows between the blades, resulting in visual blind spots. This causes the loss of certain information in the deformation measurement of twisted blade. A deformation measurement method of rotating twisted blades based on multi-vision overlapping area feature registration is proposed. It aims to solve the problem of limited field of view (FOV) in the deformation measurement of twisted blade, and overcome the problems of small measurement range and difficult measurement of traditional contact sensor method in the rotating state of the blade. First, a calibration and image-matching method based on multi-vision digital image correlation method is proposed. Under the premise of ensuring the simultaneous calibration of multi-vision camera stereo calibration, the matching of image detection points on the twisted blade surface is realized. Second, a three-dimensional reconstruction method based on the feature registration of the overlapping area of the FOV is proposed. It is used to solve the problem that the traditional image stitching method is prone to mismatching and to provide complete point cloud information for the deformation field calculation. Finally, the strain of each point is solved by piecewise point-by-point fitting of the local subdomains of the discrete displacement data, which suppresses the amplification of noise and improves the accuracy of blade strain calculation. In this paper, the deformation measurement experiment is carried out for the rotating twisted blade (up to 6000 rpm). The experimental results show that the proposed method can achieve a displacement measurement accuracy of 5 μm and the strain measurement accuracy of 50 με. This method can provide an important guarantee for the accurate evaluation and safe operation of the key performance parameters of the engine.

VPA-Net: A visual perception assistance network for 3d lidar semantic segmentation

The semantic segmentation of 3D point clouds holds paramount importance in visual perception tasks of automatic driving, including obstacle avoidance, decision control, path planning, and map construction, etc. Multi-sensor fusion is a rational and pivotal technique for implementing LiDAR semantic segmentation. However, effectively fusing and utilizing multi-source data remains a challenging task. In this work, we introduce a novel visual perception-assisted point cloud segmentation network, termed VPA-Net. This network architecture employs a dual-branch design to effectively combine spatial information from point clouds and visual cues from images, thereby bolstering the performance of 3D LiDAR semantic segmentation. More specifically, the dual-branch network structure processes and fuses multi-modal data from both point clouds and RGB images. Subsequently, the intermediate features from the two branches are merged via the proposed attention-based feature fusion module. Furthermore, to address the challenge of precise boundary prediction in large-scale point cloud scene segmentation, we introduce a refinement module based on 3D sparse convolution to enhance the spatial information of the LiDAR point clouds. The effectiveness of our method is validated on SemanticKITTI and a more challenging 3D semantic segmentation dataset. Experimental results underscore significant improvement on SemanticKITTI, with our approach surpassing the state-of-the-art method, achieving a 7.1% higher mIoU than SalsaNext.

An Algorithm for Generating Outdoor Floor Plans and 3D Models of Rural Houses Based on Backpack LiDAR.

As the Rural Revitalization Strategy continues to progress, there is an increasing demand for the digitization of rural houses, roads, and roadside trees. Given the characteristics of rural areas, such as narrow roads, high building density, and low-rise buildings, the precise and automated generation of outdoor floor plans and 3D models for rural areas is the core research issue of this paper. The specific research content is as follows: Using the point cloud data of the outer walls of rural houses collected by backpack LiDAR as the data source, this paper proposes an algorithm for drawing outdoor floor plans based on the topological relationship of sliced and rasterized wall point clouds. This algorithm aims to meet the needs of periodically updating large-scale rural house floor plans. By comparing the coordinates of house corner points measured with RTK, it is verified that the floor plans drawn by this algorithm can meet the accuracy requirements of 1:1000 topographic maps. Additionally, based on the generated outdoor floor plans, this paper proposes an algorithm for quickly generating outdoor 3D models of rural houses using the height information of wall point clouds. This algorithm can quickly generate outdoor 3D models of rural houses by longitudinally stretching the floor plans, meeting the requirements for 3D models in spatial analyses such as lighting and inundation. By measuring the distance from the wall point clouds to the 3D models and conducting statistical analysis, results show that the distances are concentrated between -0.1 m and 0.1 m. The 3D model generated by the method proposed in this paper can be used as one of the basic data for real 3D construction.

RD-SLAM: Real-Time Dense SLAM Using Gaussian Splatting

Simultaneous localization and mapping (SLAM) is fundamental for intelligent mobile units to perform diverse tasks. Recent work has shown that integrating neural rendering and SLAM showed promising results in photorealistic environment reconstruction. However, existing methods estimate pose by minimizing the error between rendered and input images, which is time-consuming and cannot be run in real-time, deviating from the original intention of SLAM. In this paper, we propose a dense RGB-D SLAM based on 3D Gaussian splatting (3DGS) while employing generalized iterative closest point (G-ICP) for pose estimation. We actively utilize 3D point cloud information to improve the tracking accuracy and operating speed of the system. At the same time, we propose a dual keyframe selection strategy and its corresponding densification method, which can effectively reconstruct new observation scenes and improve the quality of previously constructed maps. In addition, we introduce regularization loss to address the scale explosion of the 3D Gaussians and over-elongate in the camera viewing direction. Experiments on the Replica, TUM-RGBD, and ScanNet datasets show that our method achieves state-of-the-art tracking accuracy and runtime while being competitive in rendering quality.

LiDAR‐based place recognition for mobile robots in ground/water surface multiple scenes

AbstractLiDAR‐based 3D place recognition is an essential component of simultaneous localization and mapping systems in multi‐scene robotic applications. However, extracting discriminative and generalizable global descriptors of point clouds is still an open issue due to the insufficient use of the information contained in the LiDAR scans in existing approaches. In this paper, we propose a novel spatial‐temporal point cloud encoding network for multiple scenes, dubbed STM‐Net, to fully fuse the multi‐view spatial information and temporal information of LiDAR point clouds. Specifically, we first develop a spatial feature encoding module consisting of the single‐view transformer and multi‐view transformer. The module learns the correlation both within a single view and between two views by utilizing the multi‐layer range images generated by spherical projection and multi‐layer bird's eye view images generated by top‐down projection. Then in the temporal feature encoding module, we exploit the temporal transformer to mine the temporal information in the sequential point clouds, and a NetVLAD layer is applied to aggregate features and generate sub‐descriptors. Furthermore, we use a GeM pooling layer to fuse more information along the time dimension for the final global descriptors. Extensive experiments conducted on unmanned ground/surface vehicles with different LiDAR configurations indicate that our method (1) achieves superior place recognition performance than state‐of‐the‐art algorithms, (2) generalizes well to diverse sceneries, (3) is robust to viewpoint changes, (4) can operate in real‐time, demonstrating the effectiveness and satisfactory capability of the proposed approach and highlighting its promising applications in multi‐scene place recognition tasks.

Detection-first tightly-coupled LiDAR-Visual-Inertial SLAM in dynamic environments

To address the challenges posed by the dynamic environment for Simultaneous Localization and Mapping (SLAM), a detection-first tightly-coupled LiDAR-Visual-Inertial SLAM incorporating lidar, camera, and inertial measurement unit (IMU) is proposed. Firstly, the point cloud clustering with semantic labels are obtained by fusing image and point cloud information. Then, a tracking algorithm is applied to obtain the information of the motion state of the targets. Afterwards, the tracked dynamic targets are utilized to eliminate extraneous feature points. Finally, a factor graph is used to jointly optimize the IMU pre-integration, and tightly couple the laser odometry and visual odometry within the system. To validate the performance of the proposed SLAM framework, both public datasets (KITTI and UrbanNav) and actual scene data are tested. The experimental results show that compared with LeGO-LOAM, LIO-SAM and LVI-SAM for public dataset, the root mean squared error (RMSE) of proposed algorithm is decreased by 44.56 % (4.47 m) and 4.15 % (4.62 m) in high dynamic scenes and normal scenes, respectively. Through actual scene data, the proposed algorithm mitigates the impact of dynamic objects on map building directly.

YOLOTree-Individual Tree Spatial Positioning and Crown Volume Calculation Using UAV-RGB Imagery and LiDAR Data

Individual tree canopy extraction plays an important role in downstream studies such as plant phenotyping, panoptic segmentation and growth monitoring. Canopy volume calculation is an essential part of these studies. However, existing volume calculation methods based on LiDAR or based on UAV-RGB imagery cannot balance accuracy and real-time performance. Thus, we propose a two-step individual tree volumetric modeling method: first, we use RGB remote sensing images to obtain the crown volume information, and then we use spatially aligned point cloud data to obtain the height information to automate the calculation of the crown volume. After introducing the point cloud information, our method outperforms the RGB image-only based method in 62.5% of the volumetric accuracy. The AbsoluteError of tree crown volume is decreased by 8.304. Compared with the traditional 2.5D volume calculation method using cloud point data only, the proposed method is decreased by 93.306. Our method also achieves fast extraction of vegetation over a large area. Moreover, the proposed YOLOTree model is more comprehensive than the existing YOLO series in tree detection, with 0.81% improvement in precision, and ranks second in the whole series for mAP50-95 metrics. We sample and open-source the TreeLD dataset to contribute to research migration.

DopplerPTNet: Object Detection Network with Doppler Velocity Information for FMCW LiDAR Point Cloud

Abstract In the field of autonomous driving, LiDAR plays a crucial role in perception and detection. LiDAR based on Time-of-Flight (ToF) mode can only provide three-dimensional spatial coordinate information of point clouds. In point cloud object detection, the limited feature information of spatial coordinates to some extent restricts the further optimization and improvement of algorithm detection performance. However, LiDAR based on Frequency-Modulated Continuous-Wave (FMCW) mode can not only obtain the three-dimensional spatial coordinates of point clouds, but also directly measure the Doppler velocity information of points, effectively compensating for the limitation of relying solely on spatial coordinate information for object recognition. Therefore, based on the CARLA simulator, we construct the first FMCW LiDAR point cloud object detection simulation dataset, FMCWLidDet. What’s more, a novel 4D object detection algorithm, DopplerPTNet, is proposed based on the direct acquisition of point Doppler velocity information by FMCW LiDAR. The algorithm solves the problem of single spatial coordinate information feature in existing 3D object detection algorithms, which makes it difficult to further improve detection accuracy. The dataset is available at https://github.com/xilight123/FMCW-LiDAR-object-detection-dataset.

Filtering of 3D point clouds using maximum likelihood algorithm

Recently, the 3D point cloud (PC) has become more popular as an innovative object representation. However, there is usually noise and outliers in the raw point cloud. It is essential to eliminate the noise from the point cloud and outlier data while maintaining the features and finer details intact. This paper presents a comprehensive method for filtering and classification point clouds using a maximum likelihood algorithm (ML). TOPCON GLS-2000 3D terrestrial laser scanners (TLS) have been used to collect the 3D PC data set; the scan range is up to 350 m. About 30 m apart from the study area. ScanMaster software has been used to import, view, and filter point cloud information. The position information of the points is linked with the training point cloud and the filtered point cloud to derive the nonlinear model using MATLAB software. To evaluate the quality of the denoising results, two error metrics have been used: the average angle (δ) and distance (Dmean) between the ground truth point and the resulting point. The experimental findings demonstrate that the suggested approach can effectively filter out background noise while improving feature preservation. The filtering and classifying technique is more effective and efficient compared to the selected filtering methods when applied to 3D point clouds containing a large number of points and a variety of natural characteristics.

A Novel Point Cloud Adaptive Filtering Algorithm for LiDAR SLAM in Forest Environments Based on Guidance Information

To address the issue of accuracy in Simultaneous Localization and Mapping (SLAM) for forested areas, a novel point cloud adaptive filtering algorithm is proposed in the paper, based on point cloud data obtained by backpack Light Detection and Ranging (LiDAR). The algorithm employs a K-D tree to construct the spatial position information of the 3D point cloud, deriving a linear model that is the guidance information based on both the original and filtered point cloud data. The parameters of the linear model are determined by minimizing the cost function using an optimization strategy, and a guidance point cloud filter is subsequently constructed based on these parameters. The results demonstrate that, comparing the diameter at breast height (DBH) and tree height before and after filtering with the measured true values, the accuracy of SLAM mapping is significantly improved after filtering. The Mean Absolute Error (MAE) of DBH before and after filtering are 2.20 cm and 1.16 cm; the Root Mean Square Error (RMSE) values are 4.78 cm and 1.40 cm; and the relative RMSE values are 29.30% and 8.59%. For tree height, the MAE before and after filtering are 0.76 m and 0.40 m; the RMSE values are 1.01 m and 0.50 m; the relative RMSE values are 7.33% and 3.65%. The experimental results validate that the proposed adaptive point cloud filtering method based on guided information is an effective point cloud preprocessing method for enhancing the accuracy of SLAM mapping in forested areas.

ModelNet-O: A large-scale synthetic dataset for occlusion-aware point cloud classification

Recently, 3D point cloud classification has made significant progress with the help of many datasets. However, these datasets do not reflect the incomplete nature of real-world point clouds caused by occlusion, which limits the practical application of current methods. To bridge this gap, we propose ModelNet-O, a large-scale synthetic dataset of 123,041 samples that emulates real-world point clouds with self-occlusion caused by scanning from monocular cameras. ModelNet-O is 10 times larger than existing datasets and offers more challenging cases to evaluate the robustness of existing methods. Our observation on ModelNet-O reveals that well-designed sparse structures can preserve structural information of point clouds under occlusion, motivating us to propose a robust point cloud processing method that leverages a critical point sampling (CPS) strategy in a multi-level manner. We term our method PointMLS. Through extensive experiments, we demonstrate that our PointMLS achieves state-of-the-art results on ModelNet-O and competitive results on regular datasets such as ModelNet40 and ScanObjectNN, and we also demonstrate its robustness and effectiveness. Code available: https://github.com/fanglaosi/ModelNet-O_PointMLS.

A method for hierarchical weighted fitting of regular grid DSM with discrete points

Abstract. A Digital Surface Model (DSM) is a crucial spatial geographic information data used to describe the shape of the earth’s surface in Geographic Information Systems (GIS). DSM is the core data used in terrain analysis in GIS. A regular grid DSM is generally generated by interpolating a large number of discrete point clouds. This paper proposes a method of using a hierarchical weighted strategy to fit a regular grid DSM with discrete points. This method uses a pyramid hierarchical strategy to refine the target regular grid from one grid with finer parameters of 3*3, until the nth level (the interval of the grid is equal to the expected interval), and then gradually places the discrete point cloud into the corresponding grid by weighted averaging, and uses the result of this level as the initial value of the next level. This algorithm can avoid the problem of low efficiency in retrieving a large number of discrete point clouds, and the indirect interpolation method not considering the contribution of distant neighboring point clouds. The operation of point cloud data is a stream operation, which does not require consideration of the topological information of point clouds, and has simple operation and no additional memory consumption. It is especially suitable for the production of regular grid DSM with massive point clouds. To verify the effectiveness of this method, the article selected six typical terrain data such as high mountains, mountains, hills, plains, urban areas, and lakes for experiments. The results show that compared with the construct-TIN method for producing DSM, this method has very good processing accuracy and processing efficiency.

A novel joint depth sensor calibration method without fixture for mobile robots’ navigation

AbstractCommercial mobile robots are usually equipped with multiple depth sensors that can measure the point cloud information around the robot's environment. The installation process of these sensors contains assembly error and sensor measurement error, so it is necessary to calibrate each sensor to align the point cloud. In order to obtain the sensor calibration results of commercial robots under normal working conditions, this study proposes a fixture free multi depth sensor joint calibration method that can be deployed on low‐cost embedded computing units, which efficiently aligns the point clouds of each sensor. During the calibration process, the robot is placed in the center of three upright thin plates perpendicular to the ground. 2D LIDAR depicts high‐precision contours of the upright thin plates. In the calibration process of each depth sensor, the roll angle and pitch angle of the sensor point cloud are first calibrated to make it perpendicular to the ground, and then the yaw angle and position of the point cloud are calibrated to fit the high‐precision contour of the upright thin plate. The results show that this method can be deployed on low‐cost embedded computing units, with real‐time and accurate calibration results. The convergence of calibration results can be achieved through up to 5 iterations, and the average running time is less than 120 ms. This research achievement provides a reference for multi‐sensor calibration of commercial robots.