Unstructured 3D Point Clouds Research Articles

DeepPoint3D: Learning discriminative local descriptors using deep metric learning on 3D point clouds

Learning local descriptors is an important problem in computer vision. While there are many techniques for learning local patch descriptors for 2D images, recently efforts have been made for learning local descriptors for 3D points. The recent progress towards solving this problem in 3D leverages the strong feature representation capability of image based convolutional neural networks by utilizing RGB-D or multi-view representations. However, in this paper, we propose to learn 3D local descriptors by directly processing unstructured 3D point clouds without needing any intermediate representation. The method constitutes a deep network for learning permutation invariant representation of 3D points. To learn the local descriptors, we use a multi-margin contrastive loss which discriminates between similar and dissimilar points on a surface while also leveraging the extent of dissimilarity among the negative samples at the time of training. With comprehensive evaluation against strong baselines, we show that the proposed method outperforms state-of-the-art methods for matching points in 3D point clouds. Further, we demonstrate the effectiveness of the proposed method on various applications achieving state-of-the-art results.

DPRNet: Deep 3D Point Based Residual Network for Semantic Segmentation and Classification of 3D Point Clouds

Point clouds are an important type of geometric data obtained from a variety of 3D sensors. They do not have an explicit neighborhood structure and therefore several researchers often perform a voxelization step to obtain structured 3D neighborhood. This, however, comes with certain disadvantages, e.g., it makes the data unnecessarily voluminous, enforces additional computation effort and can potentially introduce quantization errors that may not only hinder in extracting implicit 3D shape information but also in capturing the essential data invariances for the required segmentation and recognition task. In this context, this paper addresses the highly challenging problem of semantic segmentation and 3D object recognition using raw unstructured 3D point cloud data. Specifically, the deep network architecture has been proposed which consists of a cascaded combination of 3D point-based residual networks for simultaneous semantic scene segmentation and object classification. It exploits the 3D point-based convolutions for representational learning from raw unstructured 3D point cloud data. The proposed architecture has a simple design, easier implementation, and the performance which is better than the existing state-of-the architectures particularly for semantic scene segmentation over three public datasets. The implementation and evaluation are made public here https://github.com/saira05/DPRNet.

Semantic Geometric Modelling of Unstructured Indoor Point Cloud

A method capable of automatically reconstructing 3D building models with semantic information from the unstructured 3D point cloud of indoor scenes is presented in this paper. This method has three main steps: 3D segmentation using a new hybrid algorithm, room layout reconstruction, and wall-surface object reconstruction by using an enriched approach. Unlike existing methods, this method aims to detect, cluster, and model complex structures without having prior scanner or trajectory information. In addition, this method enables the accurate detection of wall-surface “defacements”, such as windows, doors, and virtual openings. In addition to the detection of wall-surface apertures, the detection of closed objects, such as doors, is also possible. Hence, for the first time, the whole 3D modelling process of the indoor scene from a backpack laser scanner (BLS) dataset was achieved and is recorded for the first time. This novel method was validated using both synthetic data and real data acquired by a developed BLS system for indoor scenes. Evaluating our approach on synthetic datasets achieved a precision of around 94% and a recall of around 97%, while for BLS datasets our approach achieved a precision of around 95% and a recall of around 89%. The results reveal this novel method to be robust and accurate for 3D indoor modelling.

Robust normal estimation in unstructured 3D point clouds by selective normal space exploration

We present a fast and practical approach for estimating robust normal vectors in unorganized point clouds. Our proposed technique is robust to noise and outliers and can preserve sharp features in the input model while being significantly faster than the current state-of-the-art alternatives. The key idea to this is a novel strategy for the exploration of the normal space: First, an initial candidate normal vector, optimal under a robust least median norm, is selected from a discrete subregion of this space, chosen conservatively to include the correct normal; then, the final robust normal is computed, using a simple, robust procedure that iteratively refines the candidate normal initially selected. This strategy allows us to reduce the computation time significantly with respect to other methods based on sampling consensus and yet produces very reliable normals even in the presence of noise and outliers as well as along sharp features. The validity of our approach is confirmed by an extensive testing on both synthetic and real-world data and by a comparison against the most relevant state-of-the-art approaches.

Convex Variational Methods on Graphs for Multiclass Segmentation of High-Dimensional Data and Point Clouds

Graph-based variational methods have recently shown to be highly competitive for various classification problems of high-dimensional data, but are inherently difficult to handle from an optimization perspective. This paper proposes a convex relaxation for a certain set of graph-based multiclass data segmentation models involving a graph total variation term, region homogeneity terms, supervised information and certain constraints or penalty terms acting on the class sizes. Particular applications include semi-supervised classification of high-dimensional data and unsupervised segmentation of unstructured 3D point clouds. Theoretical analysis shows that the convex relaxation closely approximates the original NP-hard problems, and these observations are also confirmed experimentally. An efficient duality-based algorithm is developed that handles all constraints on the labeling function implicitly. Experiments on semi-supervised classification indicate consistently higher accuracies than related non-convex approaches and considerably so when the training data are not uniformly distributed among the data set. The accuracies are also highly competitive against a wide range of other established methods on three benchmark data sets. Experiments on 3D point clouds acquire by a LaDAR in outdoor scenes and demonstrate that the scenes can accurately be segmented into object classes such as vegetation, the ground plane and human-made structures.

Efficient retrieval of arbitrary objects from long-term robot observations

We present a novel method for efficient querying and retrieval of arbitrarily shaped objects from large amounts of unstructured 3D point cloud data. Our approach first performs a convex segmentation of the data after which local features are extracted and stored in a feature dictionary. We show that the representation allows efficient and reliable querying of the data. To handle arbitrarily shaped objects, we propose a scheme which allows incremental matching of segments based on similarity to the query object. Further, we adjust the feature metric based on the quality of the query results to improve results in a second round of querying. We perform extensive qualitative and quantitative experiments on two datasets for both segmentation and retrieval, validating the results using ground truth data. Comparison with other state of the art methods further enforces the validity of the proposed method. Finally, we also investigate how the density and distribution of the local features within the point clouds influence the quality of the results.

Viewpoint-independent 3D object segmentation for randomly stacked objects using optical object detection

This work proposes a novel approach to segmenting randomly stacked objects in unstructured 3D point clouds, which are acquired by a random-speckle 3D imaging system for the purpose of automated object detection and reconstruction. An innovative algorithm is proposed; it is based on a novel concept of 3D watershed segmentation and the strategies for resolving over-segmentation and under-segmentation problems. Acquired 3D point clouds are first transformed into a corresponding orthogonally projected depth map along the optical imaging axis of the 3D sensor. A 3D watershed algorithm based on the process of distance transformation is then performed to detect the boundary, called the edge dam, between stacked objects and thereby to segment point clouds individually belonging to two stacked objects. Most importantly, an object-matching algorithm is developed to solve the over- and under-segmentation problems that may arise during the watershed segmentation. The feasibility and effectiveness of the method are confirmed experimentally. The results reveal that the proposed method is a fast and effective scheme for the detection and reconstruction of a 3D object in a random stack of such objects. In the experiments, the precision of the segmentation exceeds 95% and the recall exceeds 80%.

Extracting roads from dense point clouds in large scale urban environment

This paper describes a method for extracting roads from a large scale unstructured 3D point cloud of an urban environment consisting of many superimposed scans taken at different times. Given a road map and a point cloud, our system automatically separates road surfaces from the rest of the point cloud. Starting with an approximate map of the road network given in the form of 2D intersection locations connected by polylines, we first produce a 3D representation of the map by optimizing Cardinal splines to minimize the distances to points of the cloud under continuity constraints. We then divide the road network into independent patches, making it feasible to process a large point cloud with a small in-memory working set. For each patch, we fit a 2D active contour to an attractor function with peaks at small vertical discontinuities to predict the locations of curbs. Finally, we output a set of labeled points, where points lying within the active contour are tagged as “road” and the others are not. During experiments with a LIDAR point set containing almost a billion points spread over six square kilometers of a city center, our method provides 86% correctness and 94% completeness.

An improved segmentation approach for planar surfaces from unstructured 3D point clouds

AbstractThe extraction of object features from massive unstructured point clouds with different local densities, especially in the presence of random noisy points, is not a trivial task even if that feature is a planar surface. Segmentation is the most important step in the feature extraction process. In practice, most segmentation approaches use geometrical information to segment the 3D point cloud. The features generally include the position of each point (X, Y and Z), locally estimated surface normals and residuals of best fitting surfaces; however, these features could be affected by noisy points and in consequence directly affect the segmentation results. Therefore, massive unstructured and noisy point clouds also lead to bad segmentation (over‐segmentation, under‐segmentation or no segmentation). While the RANSAC (random sample consensus) algorithm is effective in the presence of noise and outliers, it has two significant disadvantages, namely, its efficiency and the fact that the plane detected by RANSAC may not necessarily belong to the same object surface; that is, spurious surfaces may appear, especially in the case of parallel‐gradual planar surfaces such as stairs. The innovative idea proposed in this paper is a modification for the RANSAC algorithm called Seq‐NV‐RANSAC. This algorithm checks the normal vector (NV) between the existing point clouds and the hypothesised RANSAC plane, which is created by three random points, under an intuitive threshold value. After extracting the first plane, this process is repeated sequentially (Seq) and automatically, until no planar surfaces can be extracted from the remaining points under the existing threshold value. This prevents the extraction of spurious surfaces, brings an improvement in quality to the computed attributes and increases the degree of automation of surface extraction. Thus the best fit is achieved for the real existing surfaces.

Approaches for geometrical and semantic modelling of huge unstructured 3D point clouds

Approaches for geometrical and semantic modelling of huge unstructured 3D point clouds