Local Point Cloud Research Articles

E-LOAM: LiDAR Odometry and Mapping With Expanded Local Structural Information

This paper investigates the real time LiDAR odometry and mapping (LOAM) problem in unstructured environments. We propose E-LOAM (LOAM with Expanded Local Structural Information), a paradigm which expands the pre-extracted geometric information with local point cloud information around the geometric feature points. State-of-the-art approaches usually extract pointed geometric features as the only correspondence primitives for point cloud scan-to-scan and scan-to-map registration. We argue that, in unstructured environments, sometimes, the extracted geometric features are too sparse for adequate point cloud registration. Therefore, E-LOAM expands the ‘pointed’ geometric correspondence primitives with the point clouds around them, <i>i.e.,</i> the local point clouds in the voxel around the feature point. The local point clouds, approximated by a multivariate normal distribution, offer additional local structural information, on top of the <i>pointed </i>geometric information. Additionally, to enrich the sparse geometric features, we make use of the intensity information of point clouds, and extract the places with high intensity variations as additional feature points. Experimental results with KITTI dataset show the efficacy of E-LOAM, when compared with state of the arts. We further implement E-LOAM on a real robot platform, and evaluate E-LOAM with in-field tests.

Anatomical Feature Segmentation of Femur Point Cloud Based on Medical Semantics

Feature segmentation is an essential phase for geometric modeling and shape processing in anatomical study of human skeleton and clinical digital treatment of orthopedics. Due to various degrees of freedom of bone surface, the existing segmentation algorithms can hardly meet specific medical need. To address this, a novel segmentation methodology for anatomical features of femur model based on medical semantics is put forward. First, anatomical reference objects (ARO) are created to represent typical characteristics of femur anatomy by 3D point fitting in combination with medical priori knowledge. Then, local point clouds between adjacent anatomies are selected according to the AROs to extract boundary feature point (BFP)s. Finally, the complete model of femur is divided into anatomical regions by executing the enhanced watershed algorithm guided with BFPs. Experimental results show that the proposed method has the advantages of automatic segmentation of femoral head, neck and other complex areas, and the segmentation results have better medical semantics. In addition, the slight modification of segmentation results can be achieved by adjusting a few threshold parameter values, which improves the convenience of modification for ordinary users.

FG-Net: A Fast and Accurate Framework for Large-Scale LiDAR Point Cloud Understanding.

This work presents FG-Net, a general deep learning framework for large-scale point cloud understanding without voxelizations, which achieves accurate and real-time performance with a single NVIDIA GTX 1080 8G GPU and an i7 CPU. First, a novel noise and outlier filtering method is designed to facilitate the subsequent high-level understanding tasks. For effective understanding purpose, we propose a novel plug-and-play module consisting of correlated feature mining and deformable convolution-based geometric-aware modeling, in which the local feature relationships and point cloud geometric structures can be fully extracted and exploited. For the efficiency issue, we put forward a new composite inverse density sampling (IDS)-based and learning-based operation and a feature pyramid-based residual learning strategy to save the computational cost and memory consumption, respectively. Compared with current methods which are only validated on limited datasets, we have done extensive experiments on eight real-world challenging benchmarks, which demonstrates that our approaches outperform state-of-the-art (SOTA) approaches in terms of accuracy, speed, and memory efficiency. Moreover, weakly supervised transfer learning is also conducted to demonstrate the generalization capacity of our method.

Research on 3D curved weld seam trajectory position and orientation detection method

The "teaching-reproduction" mode and off-line programming mode are widely used in traditional welding robots. For curved welding workpieces, the operators need to visually inspect the position and orientation of the weld seam during processing to select the teaching points or the starting point for programming. The welding accuracy depends on the operators’ experience. Re-teaching or re-programming is required after adjusting the welding position or orientation, resulting in poor welding accuracy and efficiency. To enhance the adaptability of the robot to various welding conditions, we propose a 3D curved weld seam trajectory position and orientation detection method. A local welding workpiece point cloud is obtained by laser vision sensor scanning. The point cloud is smoothed by using the moving least squares (MLS) method. The surface variation of each point is calculated, and the surface variation threshold is set to extract the weld seam feature points. The weld seam orientation is calculated by solving the basic vectors of the spatial curve according to the parametric equation of the weld seam trajectory. The rotation angles of the robot are calculated by the transformation matrix between the weld seam local coordinate system and the robot base coordinate system. According to the experiments, the proposed method can adapt to various orientations of the curved welding workpiece and improve the robot's adaptability to orientations, therefore, the accuracy and efficiency of welding will be enhanced.

A Novel Scheme about Skeleton Optimization Designed for ISTTWN Algorithm

The ISTTWN algorithm overcame the defect of separating the production process of skeleton points and skeleton lines in tree branch point cloud skeleton extraction and improved the accuracy of the extracted initial skeletons, but the skeletons need further optimization. In the existing skeleton optimization, it is difficult to see the stump adjustment, and most of the bifurcation optimization and skeleton smoothness adopt fitting. Based on the characteristics of the initial skeletons extracted by the ISTTWN algorithm, this research optimizes the skeleton from four aspects. An algorithm for the stump adjustment for reconstructing the stump based on the layer and hierarchical relationship and an algorithm for the bifurcation optimization based on the local branch point cloud and cosine correlation are proposed, and an existing pruning method and a skeleton smoothing method are used. The results show that the skeleton optimization method proposed or used in this research has a high computational efficiency in general and can ultimately retain the necessary skeleton lines. In a visual analysis, the optimized skeleton is obviously much more natural and more in line with the actual topology of trees. In the quantitative analysis, the completeness, accuracy and effectiveness reached 97.82%, 95.72% and 89.47%, respectively. In this study, in addition to the existing tree parameters extracted by the skeleton or generalized cylinder model, the generated skeleton is used to extract the branch attributes. The R2 of the deflection angle of the branch tip, distance from branch tip and branch length are about 0.897, 0.986 and 0.988, respectively, which illustrates that their models are very good. This research can further expand the application of the skeleton.

Robust Map Merging Method for Collaborative LiDAR-based SLAM Using GPS Sensor

Typical lidar-based mappings need to cover the entire environment alone, which often leads to long build cycles and large computational resource requirements. Collaborative simultaneous localization and mapping have been applied to explore large and complex scenarios. However, merging local maps from multiple robots remains a challenge. It is crucial for Collaborative SLAM to identify overlapping areas and obtain the transformation matrix for map merging. To this end, this paper proposes a map merging method for collaborative lidar-based SLAM based on GPS measurements and improved Iterative Closest Point (ICP). We add a GPS sensor to the traditional robot configuration and use a simplified model to convert latitude and longitude data into plane coordinate data. According to Euclidean geometry, we determine the overlapping regions scanned by robots and calculate the initial pose transformation matrix between different robots. Based on the traditional ICP algorithm, KD-tree and normal vectors are used to register point clouds in overlapping regions accurately. The transformation matrix obtained by precise registration is used to rotate and translate the source local point cloud, so as to obtain the global point cloud map. In the real-world experiment, the initial transformation matrix is successfully obtained based on GPS measurements, which provides a good initial pose for the ICP algorithm. Compared with the ICP algorithm, the time consumption and RMSE of the improved ICP algorithm are reduced by 26.3% and 7.94%, respectively. Experimental results demonstrate that the proposed method can achieve accurate and efficient map fusion for multi-robot collaborative SLAM.

A Multi-Channel Descriptor for LiDAR-Based Loop Closure Detection and Its Application

Simultaneous localization and mapping (SLAM) algorithm is a prerequisite for unmanned ground vehicle (UGV) localization, path planning, and navigation, which includes two essential components: frontend odometry and backend optimization. Frontend odometry tends to amplify the cumulative error continuously, leading to ghosting and drifting on the mapping results. However, loop closure detection (LCD) can be used to address this technical issue by significantly eliminating the cumulative error. The existing LCD methods decide whether a loop exists by constructing local or global descriptors and calculating the similarity between descriptors, which attaches great importance to the design of discriminative descriptors and effective similarity measurement mechanisms. In this paper, we first propose novel multi-channel descriptors (CMCD) to alleviate the lack of point cloud single information in the discriminative power of scene description. The distance, height, and intensity information of the point cloud is encoded into three independent channels of the shadow-casting region (bin) and then compressed it into a two-dimensional global descriptor. Next, an ORB-based dynamic threshold feature extraction algorithm (DTORB) is designed using objective 2D descriptors to describe the distributions of global and local point clouds. Then, a DTORB-based similarity measurement method is designed using the rotation-invariance and visualization characteristic of descriptor features to overcome the subjective tendency of the constant threshold ORB algorithm in descriptor feature extraction. Finally, verification is performed over KITTI odometry sequences and the campus datasets of Jilin University collected by us. The experimental results demonstrate the superior performance of our method to the state-of-the-art approaches.

Research on Intelligent Robot Point Cloud Grasping in Internet of Things.

The development of Internet of Things (IoT) technology has enabled intelligent robots to have more sensing and decision-making capabilities, broadening the application areas of robots. Grasping operation is one of the basic tasks of intelligent robots, and vision-based robot grasping technology can enable robots to perform dexterous grasping. Compared with 2D images, 3D point clouds based on objects can generate more reasonable and stable grasping poses. In this paper, we propose a new algorithm structure based on the PointNet network to process object point cloud information. First, we use the T-Net network to align the point cloud to ensure its rotation invariance; then we use a multilayer perceptron to extract point cloud characteristics and use the symmetric function to get global features, while adding the point cloud characteristics attention mechanism to make the network more focused on the object local point cloud. Finally, a grasp quality evaluation network is proposed to evaluate the quality of the generated candidate grasp positions, and the grasp with the highest score is obtained. A grasping dataset is generated based on the YCB dataset to train the proposed network, which achieves excellent classification accuracy. The actual grasping experiments are carried out using the Baxter robot and compared with the existing methods; the proposed method achieves good grasping effect.

Multistage Adaptive Point-Growth Network for Dense Point Cloud Completion

The point cloud data from actual measurements are often sparse and incomplete, making it difficult to apply them directly to visual processing and 3D reconstruction. The point cloud completion task can predict missing parts based on a sparse and incomplete point cloud model. However, the disordered and unstructured characteristics of point clouds make it difficult for neural networks to obtain detailed spatial structures and topological relationships, resulting in a challenging point cloud completion task. Existing point cloud completion methods can only predict the rough geometry of the point cloud, but cannot accurately predict the local details. To address the shortcomings of existing point cloud complementation methods, this paper describes a novel network for adaptive point cloud growth, MAPGNet, which generates a sparse skeletal point cloud using the skeletal features in the composite encoder, and then adaptively grows the local point cloud in the spherical neighborhood of each point using the growth features to complement the details of the point cloud in two steps. In this paper, the Offset Transformer module is added in the process of complementation to enhance the contextual connection between point clouds. As a result, MAPGNet improves the quality of the generated point clouds and recovers more local detail information. Comparing our algorithm with other state-of-the-art algorithms in different datasets, experimental results show that our algorithm has advantages in dense point cloud completion.

Fast Registration of Point Cloud Based on Custom Semantic Extraction.

With the increase in the amount of 3D point cloud data and the wide application of point cloud registration in various fields, the question of whether it is possible to quickly extract the key points of registration and perform accurate coarse registration has become a question to be urgently answered. In this paper, we proposed a novel semantic segmentation algorithm that enables the extracted feature point cloud to have a clustering effect for fast registration. First of all, an adaptive technique was proposed to determine the domain radius of a local point. Secondly, the feature intensity of the point is scored through the regional fluctuation coefficient and stationary coefficient calculated by the normal vector, and the high feature region to be registered is preliminarily determined. In the end, FPFH is used to describe the geometric features of the extracted semantic feature point cloud, so as to realize the coarse registration from the local point cloud to the overall point cloud. The results show that the point cloud can be roughly segmented based on the uniqueness of semantic features. The use of a semantic feature point cloud can make the point cloud have a very fast response speed based on the accuracy of coarse registration, almost equal to that of using the original point cloud, which is conducive to the rapid determination of the initial attitude.

Local offset point cloud transformer based implicit surface reconstruction

AbstractImplicit neural representations, such as MLP, can well recover the topology of watertight object. However, MLP fails to recover geometric details of watertight object and complicated topology due to dealing with point cloud in a point‐wise manner. In this paper, we propose a point cloud transformer called local offset point cloud transformer (LOPCT) as a feature fusion module. Before using MLP to learn the implicit function, the input point cloud is first fed into the local offset transformer, which adaptively learns the dependency of the local point cloud and obtains the enhanced features of each point. The feature‐enhanced point cloud is then fed into the MLP to recover the geometric details and sharp features of watertight object and complex topology. Extensive reconstruction experiments of watertight object and complex topology demonstrate that our method achieves comparable or better results than others in terms of recovering sharp features and geometric details. In addition, experiments on watertight objects demonstrate the robustness of our method in terms of average result.

A novel method for point cloud completion: Adaptive region shape fusion network

Point cloud data obtained by 3D scanning equipment is often incomplete. In recent years, to complete the missing point clouds has become an important task in computer visualization research. This paper adopts a coarse-to-fine completion strategy and build a novel adaptive region shape fused network (ARSF-Net), which contains three core modules, namely region shape encoding module (RSE), adaptive feature selection-aggregation module (ASA), and encoding-attention transformer module (EAT). The RSE module adaptively aggregates the latent shape information contained in local features according to the feature strength. In ASA module, we first treat point coordinates and shape features as parent nodes and design a hybrid correlation method to adaptively group parent nodes. Then, each set of parent nodes generates a child node. Finally, we splice the features and points in the parent node and child node separately to double the number. For EAT module, we learn features from the encoding stage and use a coordinate-based embedding transformer to generate uniform high-resolution point clouds. Compared with previous methods, we pay special attention to the difference among the latent shape information contained in the local point clouds, thus making the local feature extraction more interpretable. At the same time, to generate valid detail features from the original ones, we abundantly consider the correlation among the original ones, and directly combine the original features with the generated ones. Our experiments on different datasets verify the good performance of ARSF-Net in the point cloud completion task.

Imitative Collaboration: A mirror-neuron inspired mixed reality collaboration method with remote hands and local replicas

Mixed reality can overlay and display 3D digital content in the real world, convey abstract concepts to users, and promote the understanding of complex tasks. However, the abstract graphics overlaid on the physical space may cause a certain cognitive load for local users and reduce the efficiency of collaboration. To improve the efficiency of remote collaboration, we conducted an elicitation study on assembly tasks, explored the user needs for collaboration, and defined the design goals of our remote collaboration method. Inspired by the mirror-neuron mechanism, we present an imitative collaboration method that allows local users to imitate the interaction behavior of remote users to complete tasks. We also propose a series of interaction methods for remote users to select, copy, and interact with the local point clouds to facilitate the expression of collaboration intentions. Finally, the results of a user study evaluating our imitative collaboration method on assembly tasks are reported, confirming that our method improves collaboration efficiency while reducing the cognitive load of local users.

A deep architecture for log-Euclidean Fisher vector end-to-end learning with application to 3D point cloud classification

Point clouds are a widely used form of 3D data, which can be produced by depth sensors, such as RGB-D cameras. The classification of common elements of 3D point clouds remains an open research problem.We propose a new deep network approach for the end-to-end training of log-Euclidean Fisher vectors (LE-FVs), applied to the classification of 3D point clouds. Our method uses a log-Euclidean (LE) metric in order to extend the concept of Fisher vectors (FVs) to LE-FV encoding. The LE-FV was computed on covariance matrices of local 3D point cloud descriptors, representing multiple features. Our architecture is composed of two blocks. The first one aims to map the covariance matrices representing the 3D point cloud descriptors to the Euclidean space. The second block allows for joint and simultaneous learning of LE-FV Gaussian Mixture Model (GMM) parameters, LE-FV dimensionality reduction, and multi-label classification.Our LE-FV deep learning model is more accurate than the FV deep learning architecture. Additionally, the introduction of joint learning of 3D point cloud features in the log-Euclidean space, including LE-FV GMM parameters, LE-FV dimensionality reduction, and multi-label classification greatly improves the accuracy of classification. Our method has also been compared with the most popular methods in the literature for 3D point cloud classification, and it achieved good performance. The quantitative evidence will be shown through different experiments.

SAGA-net

In this paper, we propose a real-time shape-assisted graph attention neural network to perform local pointcloud repairment. The orderless pointclouds require an effective shape encoder to distill local and global geometric feature descriptors. Previous work has attempted to convert pointcloud representation into a voxelized shape or perform grid-transformations. While these approaches can subsequently allow common convolution operations on the structured data, they either pose additional computational cost or disrupt the local geometric information. We present SAGA-Net, an efficient graph attention neural network framework with a prior shape inquiry protocol that effectively extracts local geometric information, locates the descriptor for the missing region and accurately reconstructs the local region in a real-time manner. Our framework has been validated on a benchmark dataset, ShapeNet. We demonstrate that our framework can repair each partial pointcloud with accuracy surpassing other frameworks in most object categories, and improve the computational efficiency by orders of magnitude in terms of time cost.

RAILWAY LIDAR SEMANTIC SEGMENTATION WITH AXIALLY SYMMETRICAL CONVOLUTIONAL LEARNING

Abstract. This paper presents a new deep-learning-based method for 3D Point Cloud Semantic Segmentation specifically designed for processing real-world LIDAR railway scenes. The new approach relies on the use of spatial local point cloud transformations for convolutional learning. These transformations allow an increased robustness to varying point cloud densities while preserving metric information and a sufficient descriptive ability. The resulting performances are illustrated with results on railway data from two distinct LIDAR point cloud datasets acquired in industrial settings. The quality of the extraction of useful information for maintenance operations and topological analysis is pointed together with a noticeable robustness to point cloud variations in distribution and point redundancy.

3D Global Localization in the Underground Mine Environment Using Mobile LiDAR Mapping and Point Cloud Registration.

This study proposes a 3D global localization method that implements mobile LiDAR mapping and point cloud registration to recognize the locations of objects in an underground mine. An initial global point cloud map was built for an entire underground mine area using mobile LiDAR; a local LiDAR scan (local point cloud) was generated at the point where underground positioning was required. We calculated fast point feature histogram (FPFH) descriptors for the global and local point clouds to extract point features. The match areas between the global and the local point clouds were searched and aligned using random sample consensus (RANSAC) and iterative closest point (ICP) registration. The object’s location on the global coordinate system was measured using the LiDAR sensor trajectory. Field experiments were performed at the Gwan-in underground mine using three mobile LiDAR systems. The local point cloud dataset formed for the six areas of the underground mine precisely matched the global point cloud, with a low average error of approximately 0.13 m, regardless of the type of mobile LiDAR system used. In addition, the LiDAR senor trajectory was aligned on the global coordinate system to confirm the change in the dynamic object’s position over time.

Group-in-Group Relation-Based Transformer for 3D Point Cloud Learning

Deep point cloud neural networks have achieved promising performance in remote sensing applications, and the prevalence of Transformer in natural language processing and computer vision is in stark contrast to underexplored point-based methods. In this paper, we propose an effective transformer-based network for point cloud learning. To better learn global and local information, we propose a group-in-group relation-based transformer architecture to learn the relationships between point groups to model global information and between points within each group to model local semantic information. To further enhance the local feature representation, we propose a Radius Feature Abstraction (RFA) module to extract radius-based density features characterizing the sparsity of local point clouds. Extensive evaluation on public benchmark datasets demonstrate the effectiveness and competitive performance of our proposed method on point cloud classification and part segmentation.

VIPDA: A Visually Driven Point Cloud Denoising Algorithm Based on Anisotropic Point Cloud Filtering

Point clouds (PCs) provide fundamental tools for digital representation of 3D surfaces, which have a growing interest in recent applications, such as e-health or autonomous means of transport. However, the estimation of 3D coordinates on the surface as well as the signal defined on the surface points (vertices) is affected by noise. The presence of perturbations can jeopardize the application of PCs in real scenarios. Here, we propose a novel visually driven point cloud denoising algorithm (VIPDA) inspired by visually driven filtering approaches. VIPDA leverages recent results on local harmonic angular filters extending image processing tools to the PC domain. In more detail, the VIPDA method applies a harmonic angular analysis of the PC shape so as to associate each vertex of the PC to suit a set of neighbors and to drive the denoising in accordance with the local PC variability. The performance of VIPDA is assessed by numerical simulations on synthetic and real data corrupted by Gaussian noise. We also compare our results with state-of-the-art methods, and we verify that VIPDA outperforms the others in terms of the signal-to-noise ratio (SNR). We demonstrate that our method has strong potential in denoising the point clouds by leveraging a visually driven approach to the analysis of 3D surfaces.

Restoration of Individual Tree Missing Point Cloud Based on Local Features of Point Cloud

LiDAR (Light Detection And Ranging) technology is an important means to obtain three-dimensional information of trees and vegetation. However, due to the influence of scanning mode, environmental occlusion and mutual occlusion between tree canopies and other factors, a tree point cloud often has different degrees of data loss, which affects the high-precision quantitative extraction of vegetation parameters. Aiming at the problem of a tree laser point cloud being missing, an individual tree incomplete point cloud restoration method based on local features of the point cloud is proposed. The L1-Median algorithm is used to extract key points of the tree skeleton, then the dominant direction of skeleton key points and local point cloud density are calculated, and the point cloud near the missing area is moved based on these features to gradually complete the incomplete point cloud compensation. The experimental results show that the above repair method can effectively repair the incomplete point cloud with good robustness and can adapt to the individual tree point cloud with different geometric structures and correct the branch topological connection errors.