Set Of Point Clouds Research Articles

A new point cloud simplification algorithm based on V-P container constraint and normal vector angle information entropy

Abstract Most point cloud simplification algorithms use k-order neighborhood parameters, which are set by human experience; thus, the accuracy of point feature information is not high, and each point is repeatedly calculated simultaneously. The proposed method avoids this problem. The first ordinal point of the original point cloud file was used as the starting point, and the same spatial domain was then described. The design method filters out points located in the same spatial domain and stores them in the same V-P container. The normal vector angle information entropy was calculated for each point in each container. Points with information entropy values that met the threshold requirements were extracted and stored as simplified points and new seed points. In the second operation, a point from the seed point set was selected as the starting point for the operation. The same process was repeated as the first operation. After the operation, the point from the seed point set was deleted. This process was repeated until the seed point set was empty and the algorithm ended. The simplified point set thus obtained was the simplified result. Five experimental datasets were selected and compared using the five advanced methods. The results indicate that the proposed method maintains a simplification rate of over 82% and reduces the maximum error, average error, and Hausdorff distance by 0.1099, 0.074, and 0.0062 (the highest values among the five datasets), respectively. This method has superior performance for single object and multi object point cloud sets, particularly as a reference for the study of simplified algorithms for more complex, multi object and ultra-large point cloud sets obtained using terrestrial laser scanning and mobile laser scanning.

Granular3D: Delving into multi-granularity 3D scene graph prediction

This paper addresses the significant challenges in 3D Semantic Scene Graph (3DSSG) prediction, essential for understanding complex 3D environments. Traditional approaches, primarily using PointNet and Graph Convolutional Networks, struggle with effectively extracting multi-grained features from intricate 3D scenes, largely due to a focus on global scene processing and single-scale feature extraction. To overcome these limitations, we introduce Granular3D, a novel approach that shifts the focus towards multi-granularity analysis by predicting relation triplets from specific sub-scenes. One key is the Adaptive Instance Enveloping Method (AIEM), which establishes an approximate envelope structure around irregular instances, providing shape-adaptive local point cloud sampling, thereby comprehensively covering the contextual environments of instances. Moreover, Granular3D incorporates a Hierarchical Dual-Stage Network (HDSN), which differentiates and processes features of instances and their pairs at varying scales, leading to a targeted prediction of instance categories and their relationships. To advance the perception of sub-scene in HDSN, we design a Gather Point Transformer structure (GaPT) that enables the combinatorial interaction of local information from multiple point cloud sets, achieving a more comprehensive local contextual feature extraction. Extensive evaluations on the challenging 3DSSG benchmark demonstrate that our methods provide substantial improvements, establishing a new state-of-the-art in 3DSSG prediction, boosting the top-50 triplet accuracy by ＋2.8%.

Point cloud registration method for indoor depth sensor acquisition system based on dual graph computation with irregular shape factors

AbstractThe registration performance determines the widespread indoor application of 3D models acquired by depth sensors. Many advanced registration methods lack comprehensive feature aggregation and poor generalization capabilities, which improves the mismatching ratio. Here, a dual graph network is proposed by incorporating irregular shape factors to make point cloud features more expressive. At first, we transform point cloud sets into the stellar graph within the local neighbourhood of each point. The deep feature and shape factor of each point are combined in the directional‐connected irregular projection space. Subsequently, the combined features are modelled as the second graph. By the attention mechanism computation, feature information is continuously aggregated with intra‐graph and inter‐graph. Finally, a loss function is utilized to confirm point correspondence and perform the registration through singular value decomposition. Extensive experiments validate that the proposed point cloud registration method achieves state‐of‐the‐art performance.

A new point cloud simplification method for reducing visual distortion

Existing point cloud simplification methods often focus on geometric fidelity while paying little attention to the visual distortion they cause. As a result, the simplified point cloud cannot meet the visualization requirements well. This study proposes a new simplification method that significantly reduces the visual distortion in the simplified point cloud. The proposed method begins by establishing a visual information loss function of the simplified point cloud and minimizing it, resulting in a set of simplified point clouds. Then, a quantification criterion for visual distortion in the simplified point cloud is established. By the criterion, point clouds are projected onto geometric and color feature domains that are closely related to visual effects, and the simplified point cloud with a small visual difference from the original point cloud is generated. The effectiveness of the proposed method has been verified using benchmark datasets and a scanned substation's massive point cloud.

TESTING OF CLOSE-RANGE PHOTOGRAMMETRY AND LASER SCANNING FOR EASY DOCUMENTATION OF HISTORICAL OBJECTS AND BUILDINGS PARTS

In contemporary times, technology has dramatically simplified the process of documenting cultural heritage objects. This paper aims to illustrate modern methodologies for documenting historical monuments by applying advanced technologies, as these case studies show. Two technologies were used: laser scanning and digital photogrammetry. The first subject of our study was a historically significant 16th-century building located in Jachymov, situated in the North-West region of the Czech Republic. The object was documented by laser scanning (BLK360) and using a standard digital camera Canon Powershot SX230 HS. The laser scanning data were post-processed using Cyclone Register 360 Plus, while photogrammetric point clouds were generated with Agisoft Metashape. The second objects were a building and a sculpture in the historical UNESCO city of Telč; here laser scanning instruments, digital photogrammetry and a new one low-cost ViDoc documentation system were use. Our study is focused on analyzing the sets of point clouds obtained from the data processing, conducted using CloudCompare to draw insights and conclusions. It turns out that the rapid development of techniques and technology makes it possible to use low-cost devices for sufficiently accurate and fast documentation; however, the mentioned ViDOC device can only be used for very close and small objects up to about 5 meters. For the documentation of sculptures, classic close-range photogrammetry is especially appropriate; for the documentation of buildings, laser scanners are suitable; the problem here is with hidden spaces and it is often necessary to use either a platform or a combination with photogrammetry using a drone, especially for the upper parts of the object.

SEMANTIC ENRICHMENT OF 3D POINT CLOUDS USING 2D IMAGE SEGMENTATION

Abstract. 3D point cloud segmentation is computationally intensive due to the lack of inherent structural information and the unstructured nature of the point cloud data, which hinders the identification and connection of neighboring points. Understanding the structure of the point cloud data plays a crucial role in obtaining a meaningful and accurate representation of the underlying 3D environment. In this paper, we propose an algorithm that builds on existing state-of-the-art techniques of 2D image segmentation and point cloud registration to enrich point clouds with semantic information. DeepLab2 with ResNet50 as backbone architecture trained on the COCO dataset is used for indoor scene semantic segmentation into several classes like wall, floor, ceiling, doors, and windows. Semantic information from 2D images is propagated along with other input data, i.e., RGB images, depth images, and sensor information to generate 3D point clouds with semantic information. Iterative Closest Point (ICP) algorithm is used for the pair-wise registration of consecutive point clouds and finally, optimization is applied using the pose graph optimization on the whole set of point clouds to generate the combined point cloud of the whole scene. 3D point cloud of the whole scene contains pseudo-color information which denotes the semantic class to which each point belongs. The proposed methodology use an off-the-shelf 2D semantic segmentation deep learning model to semantically segment 3D point clouds collected using handheld mobile LiDAR sensor. We demonstrate a comparison of the accuracy achieved compared to a manually segmented point cloud on an in-house dataset as well as a 2D3DS benchmark dataset.

Domain Adaptive Sampling for Cross-Domain Point Cloud Recognition

Point cloud recognition has recently gained increasing research interest due to the huge potential in real-world applications such as autonomous driving, robotics, etc. However, the point clouds of similar objects often exhibit notable geometric variations due to the difference in capturing devices or environmental changes. This leads to significant performance degradation when the learned point cloud recognition model is applied to a new scenario, which is also known as the domain adaptation issue. In this work, we propose a new unsupervised domain adaptation approach for point cloud recognition via domain adaptive sampling (DAS). In particular, we propose a two-level sampling strategy of point level and instance level to improve the cross-domain recognition ability of the model. First, we propose a domain adaptive point sampling (DAPS) strategy to enhance the domain-invariant representation of point clouds by progressively focusing on representative points in each point cloud based on geometric consistency. Then, we further propose an instance-level domain adaptive cloud sampling (DACS) strategy to learn target-specific information based on a self-paced learning paradigm, where we select a set of pseudo-labeled target point clouds to train our designed light-weighted adapters without modifying the learned domain-invariant representation. We validate our domain adaptive sampling approach on the benchmark datasets PointDA-10 and GraspNetPC-10, where our method achieves new state-of-the-art performance.

A point cloud registration method based on multiple-local-feature matching

A dual-view point cloud registration method has been proposed to enhance the accuracy of point-cloud registration by utilizing multiple local feature descriptors. The method focuses on studying the eigenvalues of the point cloud set at various scales and the impact of descriptors that consist of corresponding curvature radii in reducing misregistration in fast point feature histogram (FPFH). The optimal scale size and number were determined through experimental analysis. SIFT-3D key points of point clouds were extracted to calculate their FPFH and complete initial correspondence matching. Next, a multi-neighborhood local feature descriptor was designed to filter correspondence. A large number of incorrect corresponding point pairs were removed after the initial FPFH matching, and the remaining correct point pairs were used for rough registration. Finally, the improved ICP algorithm was used to reduce registration errors and achieve more accurate dual-view registration. According to the experiments of the public point cloud data set, the dual-view point cloud registration algorithm implemented by the method has good accuracy, high robustness, and more reliable point cloud registration results, which provides a theoretical basis for 3D point-cloud reconstruction.

Precise plane registration with multiple geometric feature matching and weighted least squares estimation

Point cloud registration is a fundamental problem for 3D laser scanning technology, which is extensively applied in geographic entity modelling such as 3D reconstruction of urban roads and buildings. Registration accuracy is one of the main focuses for these applications. However, noisy points, limited overlap, varying data sources, and differing measuring accuracy may cause changes between point cloud sets and reduce registration accuracy. This paper introduces an automatic plane registration method for urban roads and building scenes, which does not need manual on-site deployment. Firstly, plane primitives are extracted using voxel-based filtering region growth. Next, corresponding planes for the extracted primitives are identified by leveraging saliency features and constructing adjacency matrices. Finally, plane registration is achieved using a weighted plane coordinate conversion model. Through real-world scene experimentation, an overall accuracy of 10 cm and a segmental registration accuracy of 5–6 cm is achieved with our method, outperforming both feature point-based and global point cloud registration approaches in terms of efficiency and accuracy.

Skeleton-guided generation of synthetic noisy point clouds from as-built BIM to improve indoor scene understanding

The limited amount of high-quality training data available in indoor understanding with deep learning is a major problem. A possible solution to this problem is to use synthetic data to improve network training. In this study, a fully automatic method to generate synthetic noisy point clouds from as-built building information modeling (BIM) models is presented and it assesses the potential of these synthetic point clouds to improve deep neural network training. Based on a skeleton-guided strategy, all hypothetical scanning sites are located along the central axis of the buildings, which are obtained through equidistant sampling. Then, the synthetic labeled point cloud is generated station-by-station, and data augmentation is achieved using a random combination of data from different stations. The proposed approach involves generating over 44 sets of synthetic noisy point clouds based on BIM models. The performance of state-of-the-art (SOTA) deep learning methods in understanding indoor scenes enhanced by the synthetic point clouds is thoroughly assessed, and the effectiveness of various combinations of real and synthetic datasets is investigated. The experimental results demonstrate that leveraging synthetic point clouds generated from BIM models leads to a remarkable 5%–10% improvement in 3D semantic segmentation accuracy. The research signifies the value of synthetic point clouds as an effective tool for improving deep neural network training. All simulation datasets are publicly available, including original BIM models, full synthetic point clouds, and point clouds after IHPR processing, accessible via the BIMSyn Dataset link. In future research, an exploration of how synthetic point clouds will be further improved by considering specific characteristics of objects such as color, material reflectance, and illumination.

Optimized GICP registration algorithm based on principal component analysis for point cloud edge extraction

For iterative closest point (ICP) algorithm, the initial position and the number of iterations are needed in registration. At the same time, the ICP algorithm is easy to fall into local convergence and convergence speed is slow. By constructing K-D tree to search neighborhood points and artificially set threshold, plane fitting is carried out, the on-time point cloud to be deployed is separated from the complex background, and statistical analysis is used to calculate the distance between the point cloud and the neighborhood point to quickly remove the invalid point cloud. The surface equation is set to calculate the tangent plane of point cloud normal vector and each normal vector, and the local coordinate system is constructed. The angle between adjacent vectors and the local coordinate system is calculated to determine the feature point set of edge contour. According to the covariance matrix of the feature points set, the principal feature component is obtained, the principal axis direction of the two sets of point clouds is found, and the rotation matrix and the displacement vector are obtained. Finally, GICP precise registration of point cloud is carried out according to initial pose parameters and rigid body transformation matrix obtained by maximum likelihood estimation method. The results show that the optimized algorithm can effectively avoid local convergence. Compared with the traditional ICP algorithm, when the algorithm achieves the same registration accuracy in the public dataset experiment, the registration speed is on average 44.82% faster and the overlap rate is on average 15.26% higher. In the real dataset experiment, the registration speed is on average 59.04% faster, the registration accuracy is on average 30.24% higher and the overlap rate is on average 10.61% higher. This shows that the optimization algorithm is superior to the traditional ICP algorithm in registration accuracy and convergence speed.

Lossless Point Cloud Geometry and Attribute Compression Using a Learned Conditional Probability Model

In recent years, we have witnessed the presence of point cloud data in many aspects of our life, from immersive media, autonomous driving to healthcare, although at the cost of a tremendous amount of data. In this paper, we present an efficient lossless point cloud compression method that uses sparse tensor-based deep neural networks to learn point cloud geometry and color probability distributions. Our method represents a point cloud with both occupancy feature and three attribute features at different bit depths in a unified sparse representation. This allows us to efficiently exploit feature-wise and point-wise dependencies within point clouds using a sparse tensor-based neural network and thus build an accurate auto-regressive context model for an arithmetic coder. To the best of our knowledge, this is the first learning-based lossless point cloud geometry and attribute compression approach. Compared with the-state-of-the-art lossless point cloud compression method from Moving Picture Experts Group (MPEG), our method achieves 22.6% reduction in total bitrate on a diverse set of test point clouds while having 49.0% and 18.3% rate reduction on geometry and color attribute component, respectively.

OLFF-Net: Robust Registration of 3D Point Cloud based on Overlapped Local Feature Fusion

Recent advance in high-accuracy sensors has made point cloud become the main data format to characterize the three-dimensional world. Since the sensor can only scan and capture the 3D data within a limited field of view, an alignment algorithm is needed to generate the complete 3D scene. Point cloud registration is the solution for alignment problem that aims to estimate the transformation matrix between two frames of different point cloud sets. In this paper, we propose a neural network called OLFF-Net to achieve robust registration of 3D point clouds based on overlapped local feature fusion, which focuses on extracting rotational-invariant local features while providing enough information to achieve accurate alignment. Extensive experiments on representative datasets indicate that the framework can largely outperform competing methods with an average improvement of 16.82% in the metrics over the compared methods. More importantly, it shows significant generalization capability and can be widely applied to point cloud data with multiple complex structures.

Partition-Based Point Cloud Completion Network with Density Refinement.

In this paper, we propose a novel method for point cloud complementation called PADPNet. Our approach uses a combination of global and local information to infer missing elements in the point cloud. We achieve this by dividing the input point cloud into uniform local regions, called perceptual fields, which are abstractly understood as special convolution kernels. The set of point clouds in each local region is represented as a feature vector and transformed into N uniform perceptual fields as the input to our transformer model. We also designed a geometric density-aware block to better exploit the inductive bias of the point cloud's 3D geometric structure. Our method preserves sharp edges and detailed structures that are often lost in voxel-based or point-based approaches. Experimental results demonstrate that our approach outperforms other methods in reducing the ambiguity of output results. Our proposed method has important applications in 3D computer vision and can efficiently recover complete 3D object shapes from missing point clouds.

Sparsity-Based Human Activity Recognition With PointNet Using a Portable FMCW Radar

Radar-based solutions have attracted great attention in human activity recognition (HAR) for their advantages in accuracy, robustness, and privacy protection. The conventional approaches transform radar signals into feature maps and then directly process them as visual images. While effective, these image-based methods may not be the best solutions in terms of representation efficiency to encode the relevant information for classification. This paper proposes a novel HAR method combining sparse theory and PointNet network, with both operations in the time-Doppler and range-Doppler domains. First, sparsity-based feature extraction is introduced to use a limited number of sparse solutions to characterize human activities in the form of time-Doppler sparse point clouds (TDSP) or dynamic range-Doppler sparse point clouds (DRDSP). This new representation is validated by comparing the reconstructed and original signals. Then, PointNet networks are adopted to summarize multi-domain features and predict human activity labels by a sparse set of input point clouds. Comprehensive experiments were conducted to demonstrate that the proposed method can yield a higher representation efficiency, classification accuracy, and better generalization capability than existing ones.

Reduction of LiDAR Point Cloud Maps for Localization of Resource-Constrained Robotic Systems

This article presents a method for mobile robotic systems to reduce the size of 3-D point cloud maps while achieving small localization errors even though the reduced maps are used for localization. Localization with dense depth maps especially in large environments could be computationally expensive, so not affordable for robotic systems with limited computational resources. Since such point cloud data collected using a multichannel 3-D LiDAR have representational redundancy, some amount of the data could be removed for compactness. However, existing methods demand annotations and descriptors defined by human experts or lose important spatial features for accurate localization. We propose a fully-automatic reduction method for reliable navigation of mobile systems with low computing power and limited storage. Given a full 3-D point cloud map, our method generates a graph whose vertices and edges represent robot poses and the similarity between the data collected from a pair of robot poses, respectively. Each robot pose is associated with a point cloud set. We find a dominating set of the graph, which is a subset of robot poses such that the robot can localize reliably only with the point clouds collected from those poses. Experimental results with indoor environments show that our system reduces the amount of map data up to 93% while not exceeding acceptable ranges of average translation and rotation errors, which are 0.25 m and 0.5 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> , respectively.

Multiway Non-Rigid Point Cloud Registration via Learned Functional Map Synchronization.

We present SyNoRiM, a novel way to jointly register multiple non-rigid shapes by synchronizing the maps that relate learned functions defined on the point clouds. Even though the ability to process non-rigid shapes is critical in various applications ranging from computer animation to 3D digitization, the literature still lacks a robust and flexible framework to match and align a collection of real, noisy scans observed under occlusions. Given a set of such point clouds, our method first computes the pairwise correspondences parameterized via functional maps. We simultaneously learn potentially non-orthogonal basis functions to effectively regularize the deformations, while handling the occlusions in an elegant way. To maximally benefit from the multi-way information provided by the inferred pairwise deformation fields, we synchronize the pairwise functional maps into a cycle-consistent whole thanks to our novel and principled optimization formulation. We demonstrate via extensive experiments that our method achieves a state-of-the-art performance in registration accuracy, while being flexible and efficient as we handle both non-rigid and multi-body cases in a unified framework and avoid the costly optimization over point-wise permutations by the use of basis function maps.

Surface Approximation by Means of Gaussian Process Latent Variable Models and Line Element Geometry

The close relation between spatial kinematics and line geometry has been proven to be fruitful in surface detection and reconstruction. However, methods based on this approach are limited to simple geometric shapes that can be formulated as a linear subspace of line or line element space. The core of this approach is a principal component formulation to find a best-fit approximant to a possibly noisy or impartial surface given as an unordered set of points or point cloud. We expand on this by introducing the Gaussian process latent variable model, a probabilistic non-linear non-parametric dimensionality reduction approach following the Bayesian paradigm. This allows us to find structure in a lower dimensional latent space for the surfaces of interest. We show how this can be applied in surface approximation and unsupervised segmentation to the surfaces mentioned above and demonstrate its benefits on surfaces that deviate from these. Experiments are conducted on synthetic and real-world objects.

Aggregation Methods Using Bathymetry Sources of Differing Subjective Reliabilities for Navigation Mapping

When planning for ship navigation or compiling data for a bathymetry map, the navigator or mapper uses many different sources of bathymetry information and navigation hazards. The quality of these sources is inconsistent in general, however, making it challenging to provide a coherent picture for planning. Here, we describe an approach for consistent planning/mapping that uses a combination of soft computing and Bayesian estimation. The case study used to exercise this system involves NOAA Electronic Nautical Charts for an area in the Chesapeake Bay. We first interpolate each set of irregularly spaced soundings to gridded versions of each point-cloud set. Each of these intermediate grids is then aggregated into a fused bathymetric realization using order weighted averaging (OWA) to provide the weights for each source based on their subjective reliabilities. The OWA allows for fusion informed by the user’s subjective risk allowed in the reconstruction of the seafloor surface and provides quantitative methods to generate, use, and record subjective reliability weights. Each sounding point that went into the bathymetry estimate is then categorized as “no-go,” “caution,” or “go” status. Reliability estimates are reused for weighted Bayesian categorization of each output grid cell to compute the navigable surface.

An Adaptive Invariant EKF for Map-Aided Localization Using 3D Point Cloud

In map-aided localization using 3D point cloud sets, the poses estimated by 3D Registration Algorithms (3DRAs) are typically fused with other sensor data via the Extended Kalman Filter (EKF) to obtain reliable and smooth results. However, the challenges of this combined method are: 1) The linearization process of EKF may cause errors and singularities due to the state defined by a 6D pose. 2) The results of 3DRA as the measurements of EKF may cause errors in residual calculation. 3) The approach relies heavily on 3DRA to overcome the effects of dynamic scenes. This paper proposes an adaptive localization framework based on Invariant Extended Kalman Filter (Invariant EKF), in which the Lie Group is introduced to define the state. In this framework, the points of the raw point cloud set are the measurements of the filter, and the 3DRA is only employed for data association between the raw 3D point cloud set and the 3D point cloud map. Then, a Concentric Ring Model (CRM) is proposed to reduce the influence of dynamic objects, which can adaptively estimate the covariance of each observed point via Gaussian Process Regression (GPR). Besides, the CRM considers Sensor Measurement Noise (SMN) and Sensor Vibration Noise (SVN). The performance of the proposed framework is evaluated on the KITTI dataset and our dataset. The experimental results show that the proposed method is superior to other state-of-the-art methods, and the CRM can achieve more accurate measurement than ever before, especially in high-dynamic scenes.