Large-scale 3D Point Clouds Research Articles

Investigate Indistinguishable Points in Semantic Segmentation of 3D Point Cloud

This paper investigates the indistinguishable points (difficult to predict label) in semantic segmentation for large-scale 3D point clouds. The indistinguishable points consist of those located in complex boundary, points with similar local textures but different categories, and points in isolate small hard areas, which largely harm the performance of 3D semantic segmentation. To address this challenge, we propose a novel Indistinguishable Area Focalization Network (IAF-Net), which select indistinguishable points adaptively by utilizing the hierarchical semantic features and enhance fine-grained features for points especially those indistinguishable points. We also introduce multi-stage loss to improve the feature representation in a progressive way. Moreover, in order to analyze the segmentation performances of indistinguishable areas, we propose a new evaluation metric called Indistinguishable Points Based Metric (IPBM). Our IAF-Net achieves the state-of-the-art performance on several popular 3D point datasets e.g. S3DIS and ScanNet, and clearly outperform other methods on IPBM. Our code will be available at https://github.com/MingyeXu/IAF-Net.

Virtual Point Removal for Large-Scale 3D Point Clouds with Multiple Glass Planes.

Large-scale 3D point clouds (LS3DPCs) captured by terrestrial LiDAR scanners often include virtual points which are generated by glass reflection. The virtual points may degrade the performance of various computer vision techniques when applied to LS3DPCs. In this paper, we propose a virtual point removal algorithm for LS3DPCs with multiple glass planes. We first estimate multiple glass regions by modeling the reliability with respect to each glass plane, respectively, such that the regions are assigned high reliability when they have multiple echo pulses for each emitted laser pulse. Then we detect each point whether it is a virtual point or not. For a given point, we recursively traverse all the possible trajectories of reflection, and select the optimal trajectory which provides a point with a similar geometric feature to a given point at the symmetric location. We evaluate the performance of the proposed algorithm on various LS3DPC models with diverse numbers of glass planes. Experimental results show that the proposed algorithm estimates multiple glass regions faithfully and detects the virtual points successfully. Moreover, we also show that the proposed algorithm yields a much better performance of reflection artifact removal compared with the existing method qualitatively and quantitatively.

3D Point Cloud-Based Indoor Mobile Robot in 6-DoF Pose Localization Using a Wi-Fi-Aided Localization System

Six-DoF (Six-degree-of-freedom) pose localization based on 3D point clouds is a challenging task for LBSs (localization-based services). This paper proposes a robust and efficient method that uses multimodal information (vision and Wi-Fi signal information) to estimate the 6-DoF pose of an RGBD camera on a robot with respect to complex 3D textured models of the indoor environment that can contain more than 650,000,000 points. Our developed method narrows the search scope, which delimits boundaries initially using the Wi-Fi location system and applies an environment-adaptive approach to determine the radius of the search sphere based on the signal stability of the Wi-Fi location system. In addition, we propose an algorithm for estimating a novel correspondence between local points with a 3D submap by combining 3D points and surface normals to acquire absolute poses from noisy and outlier-contaminated matching point sets for RGBD sensors in dynamic indoor scenes. Then, a novel two-level spatial verification strategy is used to estimate an accurate pose, which includes the use of a RANSAC (Random Sample Consensus) algorithm for identification and a direct least-square method to acquire the pose from the inliers. The proposed method has been implemented and tested extensively in various indoor scenes. The experimental results demonstrate that the Wi-Fi-aided localization system can efficiently localize a mobile robot in a variety of large-scale 3D point cloud datasets to realize stable time consumption and similar performance to state-of-the-art methods.

Learning Semantic Segmentation of Large-Scale Point Clouds With Random Sampling.

We study the problem of efficient semantic segmentation of large-scale 3D point clouds. By relying on expensive sampling techniques or computationally heavy pre/post-processing steps, most existing approaches are only able to be trained and operate over small-scale point clouds. In this paper, we introduce RandLA-Net, an efficient and lightweight neural architecture to directly infer per-point semantics for large-scale point clouds. The key to our approach is to use random point sampling instead of more complex point selection approaches. Although remarkably computation and memory efficient, random sampling can discard key features by chance. To overcome this, we introduce a novel local feature aggregation module to progressively increase the receptive field for each 3D point, thereby effectively preserving geometric details. Comparative experiments show that our RandLA-Net can process 1 million points in a single pass up to 200× faster than existing approaches. Moreover, extensive experiments on five large-scale point cloud datasets, including Semantic3D, SemanticKITTI, Toronto3D, NPM3D and S3DIS, demonstrate the state-of-the-art semantic segmentation performance of our RandLA-Net.

DCARN: Deep Context Aware Recurrent Neural Network for Semantic Segmentation of Large Scale Unstructured 3D Point Cloud

Semantic segmentation of large unstructured 3D point clouds is important problem for 3D object recognition which in turn is essential to solving more complex tasks such as scene understanding. The problem is highly challenging owing to large scale of data, varying point density and localization errors of 3D points. Nevertheless, with recent successes of deep neural network architectures to solve complex 2D perceptual problems, several researchers have shown interest to translate the developed 2D networks to 3D point cloud segmentation by a prior voxelization step for an explicit neighborhood representation. However, such a 3D grid representation loses the fine details and inherent structure due to quantization artifacts. For this purpose, this paper proposes an approach to performing semantic segmentation of 3D point clouds by exploiting the idea of super-point based graph construction. The proposed architecture is composed of two cascaded modules including a light-weight representation learning module which uses unsupervised geometric grouping to partition the large-scale unstructured 3D point cloud and a deep context aware sequential network based on long short memory units and graph convolutions with embedding residual learning for semantic segmentation. The proposed model is evaluated on two standard benchmark datasets and achieves competitive performance with the existing state-of-the-art datasets. The code and the obtained results have been made public at https://github.com/saba155/DCARN .

SHREC 2020: 3D point cloud semantic segmentation for street scenes

Scene understanding of large-scale 3D point clouds of an outer space is still a challenging task. Compared with simulated 3D point clouds, the raw data from LiDAR scanners consist of tremendous points returned from all possible reflective objects and they are usually non-uniformly distributed. Therefore, its cost-effective to develop a solution for learning from raw large-scale 3D point clouds. In this track, we provide large-scale 3D point clouds of street scenes for the semantic segmentation task. The data set consists of 80 samples with 60 for training and 20 for testing. Each sample with over 2 million points represents a street scene and includes a couple of objects. There are five meaningful classes: building, car, ground, pole and vegetation. We aim at localizing and segmenting semantic objects from these large-scale 3D point clouds. Four groups contributed their results with different methods. The results show that learning-based methods are the trend and one of them achieves the best performance on both Overall Accuracy and mean Intersection over Union. Next to the learning-based methods, the combination of hand-crafted detectors are also reliable and rank second among comparison algorithms.

3D Large-Scale Point Cloud Semantic Segmentation Using Optimal Feature Description Vector Network: OFDV-Net

Efficient semantic segmentation of large-scale 3D point clouds is a fundamental and essential capability for real-time intelligent systems, such as autonomous driving and augmented reality. The high dimension feature vector and the complex network structure are two major constraints to utilize the large-scale point cloud. This paper proposes an optimal feature description vector network (OFDV-Net) for 3D point cloud semantic segmentation. First, a multiscale point cloud feature extraction structure is constructed to generate an initial feature description vector (IFDV). Then, IFDV is selected by a feature selection unit to obtain the optimal feature description vector (OFDV). The OFDV encapsulates the best 3D features set of the points and can be used as the input of the deep neural network for training and testing. Finally, the OFDV-Net was applied to the standard public outdoor large-scale point cloud datasets Semantic3D and NPM3D, and the overall segmentation accuracy of 88.3% and 87.7% were obtained, respectively; moreover, the OFDV-Net requires less training time, which indicates that the algorithm can obtain high-precision semantic segmentation results on an outdoor large-scale point cloud while reducing model training time.

3D object recognition method with multiple feature extraction from LiDAR point clouds

During autonomous driving, fast and accurate object recognition supports environment perception for local path planning of unmanned ground vehicles. Feature extraction and object recognition from large-scale 3D point clouds incur massive computational and time costs. To implement fast environment perception, this paper proposes a 3D recognition system with multiple feature extraction from light detection and ranging point clouds modified by parallel computing. Effective object feature extraction is a necessary step prior to executing an object recognition procedure. In the proposed system, multiple geometry features of a point cloud that resides in corresponding voxels are computed concurrently. In addition, a scale filter is employed to convert feature vectors from uncertain count voxels to a normalized object feature matrix, which is convenient for object-recognizing classifiers. After generating the object feature matrices of all voxels, an initialized multilayer neural network (NN) model is trained offline through a large number of iterations. Using the trained NN model, real-time object recognition is realized using parallel computing technology to accelerate computation.

3D Point Cloud Analysis and Classification in Large-Scale Scene Based on Deep Learning

We present a deep learning framework for efficient large-scale 3D point cloud analysis and classification using the designed feature description matrix (FDM). As the 3D points are unordered in the large-scale scene, and no topology structure can be employed directly for classification and recognition, it is difficult to apply deep neural network directly on 3D point clouds as points cannot be arranged in a fixed order as 2D image pixels. We design a new pipeline for 3D data processing by combining the traditional features extraction method and deep learning method. Our FDM encapsulates the 3D features of the point and can be used as the input of the deep neural network for training and testing. The experiments demonstrate that our method can acquire higher classification accuracy compared with our previous work and other state-of-art works.

Joint classification and contour extraction of large 3D point clouds

We present an effective and efficient method for point-wise semantic classification and extraction of object contours of large-scale 3D point clouds. What makes point cloud interpretation challenging is the sheer size of several millions of points per scan and the non-grid, sparse, and uneven distribution of points. Standard image processing tools like texture filters, for example, cannot handle such data efficiently, which calls for dedicated point cloud labeling methods. It turns out that one of the major drivers for efficient computation and handling of strong variations in point density, is a careful formulation of per-point neighborhoods at multiple scales. This allows, both, to define an expressive feature set and to extract topologically meaningful object contours.Semantic classification and contour extraction are interlaced problems. Point-wise semantic classification enables extracting a meaningful candidate set of contour points while contours help generating a rich feature representation that benefits point-wise classification. These methods are tailored to have fast run time and small memory footprint for processing large-scale, unstructured, and inhomogeneous point clouds, while still achieving high classification accuracy. We evaluate our methods on the semantic3d.net benchmark for terrestrial laser scans with >109 points.

Neighboring constraint-based pairwise point cloud registration algorithm

Three-dimensional point cloud registration is important in reverse engineering. In this paper, we propose a registration method for large-scale 3D point clouds, which is based on neighborhood constraints of geometrical features. The method consists of initial and exact registration steps.In the process of initial registration, we define a new functon that measures feature similarity by calculating the distance function, and in the process of exact registration, we introduce the angle information that improve the accuracy of iterative closest point algorithm. Compared with the traditional feature-based and iterative closest point algorithms, our method significantly reduced the registration time by 11.9 % and has only 1 % of the registration error of the traditional feature-based algorithm. The proposed algorithm can be used to create efficient 3D models for virtual plant reconstruction and computer-aided design, and the registration results can provide a reference for virtual plant reconstruction and growth.

Large-Scale 3D Point Cloud Compression Using Adaptive Radial Distance Prediction in Hybrid Coordinate Domains

An adaptive range image coding algorithm for the geometry compression of large-scale 3D point clouds (LS3DPCs) is proposed in this work. A terrestrial laser scanner generates an LS3DPC by measuring the radial distances of objects in a real world scene, which can be mapped into a range image. In general, the range image exhibits different characteristics from an ordinary luminance or color image, and thus the conventional image coding techniques are not suitable for the range image coding. We propose a hybrid range image coding algorithm, which predicts the radial distance of each pixel using previously encoded neighbors adaptively in one of three coordinate domains: range image domain, height image domain, and 3D domain. We first partition an input range image into blocks of various sizes. For each block, we apply multiple prediction modes in the three domains and compute their rate-distortion costs. Then, we perform the prediction of all pixels using the optimal mode and encode the resulting prediction residuals. Experimental results show that the proposed algorithm provides significantly better compression performance on various range images than the conventional image or video coding techniques.

Real-time terrain storage generation from multiple sensors towards mobile robot operation interface.

A mobile robot mounted with multiple sensors is used to rapidly collect 3D point clouds and video images so as to allow accurate terrain modeling. In this study, we develop a real-time terrain storage generation and representation system including a nonground point database (PDB), ground mesh database (MDB), and texture database (TDB). A voxel-based flag map is proposed for incrementally registering large-scale point clouds in a terrain model in real time. We quantize the 3D point clouds into 3D grids of the flag map as a comparative table in order to remove the redundant points. We integrate the large-scale 3D point clouds into a nonground PDB and a node-based terrain mesh using the CPU. Subsequently, we program a graphics processing unit (GPU) to generate the TDB by mapping the triangles in the terrain mesh onto the captured video images. Finally, we produce a nonground voxel map and a ground textured mesh as a terrain reconstruction result. Our proposed methods were tested in an outdoor environment. Our results show that the proposed system was able to rapidly generate terrain storage and provide high resolution terrain representation for mobile mapping services and a graphical user interface between remote operators and mobile robots.

A 3D Simplification Method based on Dual Point Sampling

For efficiently processing integration, registration, representation and recognition of large-scale 3D point clouds stored in computer disks or other hardware, it is important to simplify their sheer volume, discarding redundant information and meanwhile preserving the most important information as much as possible. This paper proposes an effective point cloud simplification method which is based on data points sampling. The dual sampling scheme considers both the local details and the overall shape. The local details analysis approach is based on graph-based segmentation, while for the overall shape analysis, the approach voxelizes the model and samples points in terms of the entropy, based on the shape index of vertices. Compared to other simplification methods, this approach reduces the number of vertices in a 3D model described by a point cloud and better preserves local details. We present a number of results to show that the method performs well both visually and quantitatively.