Set Of 3D Points Research Articles

Human Behavior Recognition Using Range-Velocity-Time Points

Radar-based sensors do not require optimal lighting and atmospheric conditions and nonocclusion, making them a promising solution for human behavior analysis in complex environments. Existing radar-based models generally retrieve features from either the time-velocity domain or the time-range domain. Such two-dimensional representations cannot fully depict dynamic human motion features. In this paper, we propose a temporal range-Doppler PointNet-based method to analyze human behavior. We transform human echoes to 3D point sets and then feed them into the hierarchical PointNet model for classification. The proposed point network can learn structural features from the micromotion trajectory more effectively than directly processing the raw point cloud. To further improve our model’s robustness in practical applications, we design an outlier detection module for detecting anomalies such as in multitarget scenarios. The results of experiments on motion capture databases and range-Doppler radar measurements demonstrate that our method realizes outstanding performance in terms of the classification accuracy, noise robustness, and anomaly detection accuracy.

Robust point cloud registration based on topological graph and Cauchy weighted [formula omitted]-norm

Point Cloud Registration (PCR) is a fundamental and important issue in photogrammetry and computer vision. Its goal is to find rigid transformations that register multiple 3D point sets. This paper proposes a robust and efficient PCR method based on topological graph and Cauchy weighted lq-norm. Our method does not require initializations and is highly robust to outliers and partial overlaps. It contains two major steps: (1) correspondence-based coarse registration, which is called Weighted lq Coarse Registration (WlqCR). In the WlqCR method, we represent feature point sets as topological graphs and transform the point matching problem to an edge matching problem. We build a mathematical model for edge correspondence maximization. We also present an edge voting strategy to distinguish potential correct matches from mismatches. Then, we define a concept called edge vector, which has a property that it is invariant to translations. Based on this property, six Degrees of Freedoms (DoF) PCR problem can be simplified into two three DoF subproblems, i.e., rotation estimation and translation estimation. (2) fine registration based on Weighted lq Iterative Closest Point (WlqICP). We propose a new ICP method called WlqICP, which is much more robust to partial overlaps compared with traditional ICP. In both rotation estimation and WlqICP, we use a new Cauchy weighted lq-norm (0<q<1) instead of l2-norm for object function construction, which has a high degree of robustness. Extensive experiments on both simulated and real data demonstrate the power of the proposed method, i.e., our method is more robust (is able to tolerate up to 99% of outliers) and much faster than compared state-of-the-art methods (WlqCR is almost two orders of magnitude faster than RANdom SAmple Consesus (RANSAC) and its variants under 95% of outliers). The source code will be made publicly available in http://www.escience.cn/people/lijiayuan/index.html.

Local range image descriptor for general point cloud registration

The general point cloud registration problem is addressed in this paper. Different from classical descriptors for alignment of RGB-D image, the present work proposes a descriptor for general point clouds, where no lattice structure is defined. The paper considers a point cloud as a set of 3D points without connectivity or normal vectors. In order to construct the descriptor a local reference frame is defined using PCA in the neighborhood of a keypoint and this basis is used to define a local range image upon which the descriptor is constructed. A histogram based approach is used to achieve invariance to rotation. In order to filter out ambiguity and reduce false correspondences a subset of corresponding points was constructed as an instance of the maximum clique problem. The reliability of the proposed descriptor is validated using ROC curve in comparison with other works from literature. Experiments show that the proposed descriptor have high matching precision and that the precision is improved by using the false correspondence filtering process. Furthermore, the false correspondence filtering process proposed in this paper is successfully applied to improve the precision of other descriptors from literature.

An efficient EM-ICP algorithm for non-linear registration of large 3D point sets

In this paper, we present a new method for non-linear pairwise registration of 3D point sets. In this method, we consider the points of the first set as the draws of a Gaussian mixture model whose centres are the displaced points of the second set. Next we perform a maximum a posteriori estimation of the parameters (which include the unknown transformation) of this model using the expectation–maximisation (EM) algorithm. Compared to other methods using the same “EM-ICP” framework, we propose four key modifications leading to an efficient algorithm allowing for fast registration of large 3D point sets: (1) truncation of the cost function; (2) symmetrisation of the point-to-point correspondences; (3) specification of priors on these correspondences using differential geometry; (4) efficient encoding of deformations using the RKHS theory and the Fourier analysis. We evaluate the added value of these modifications and compare our method to the state-of-the-art CPD algorithm on real and simulated data.

A New Robust and Fast Iterative Closest Point Algorithm for 3D Ship Hull Plate Bending Machines

Liu, X.; Hu, Y., and Xiao, C.-S., 2019. A new robust and fast iterative closest point algorithm for 3D Ship Hull Plate Bending Machines. In: Gong, D.; Zhu, H., and Liu, R. (eds.), Selected Topics in Coastal Research: Engineering, Industry, Economy, and Sustainable Development. Journal of Coastal Research, Special Issue No. 94, pp. 264–268. Coconut Creek (Florida), ISSN 0749-0208.The registration accuracy has a direct impact on the processing accuracy of sheet metal in 3D Ship Hull Plate Bending Machines. This study introduces a new distance measure, Root-Mean-Squared iterative closest point (RMS-ICP) algorithm to significantly improve the robustness of the algorithm, and reduce the influence of outliers. At the same time, the 3D point sets registration algorithm is optimized. In addition, the RMS-ICP has been optimized in robustness, registration accuracy and convergence speed in comparison with the original ICP and other improved ICP algorithms. The experiment results support that the performance of the RMS-ICP algorithm is better than other methods. Overall, the RMS-ICP algorithm is very effective in dealing with the point sets registration problem in 3D Ship Hull Plate Bending Machine.

Visualizing the Stability of 2D Point Sets from Dimensionality Reduction Techniques

AbstractWe usek‐order Voronoi diagrams to assess the stability ofk‐neighbourhoods in ensembles of 2D point sets, and apply it to analyse the robustness of a dimensionality reduction technique to variations in its input configurations. To measure the stability ofk‐neighbourhoods over the ensemble, we use cells in thek‐order Voronoi diagrams, and consider the smallest coverings of corresponding points in all point sets to identify coherent point subsets with similar neighbourhood relations. We further introduce a pairwise similarity measure for point sets, which is used to select a subset of representative ensemble members via the PageRank algorithm as an indicator of an individual member's value. The stability information is embedded into thek‐order Voronoi diagrams of the representative ensemble members to emphasize coherent point subsets and simultaneously indicate how stable they lie together in all point sets. We use the proposed technique for visualizing the robustness of t‐distributed stochastic neighbour embedding and multi‐dimensional scaling applied to high‐dimensional data in neural network layers and multi‐parameter cloud simulations.

A correntropy-based affine iterative closest point algorithm for robust point set registration

The iterative closest point (ICP) algorithm has the advantages of high accuracy and fast speed for point set registration, but it performs poorly when the point set has a large number of noisy outliers. To solve this problem, we propose a new affine registration algorithm based on correntropy which works well in the affine registration of point sets with outliers. Firstly, we substitute the traditional measure of least squares with a maximum correntropy criterion to build a new registration model, which can avoid the influence of outliers. To maximize the objective function, we then propose a robust affine ICP algorithm. At each iteration of this new algorithm, we set up the index mapping of two point sets according to the known transformation, and then compute the closed-form solution of the new transformation according to the known index mapping. Similar to the traditional ICP algorithm, our algorithm converges to a local maximum monotonously for any given initial value. Finally, the robustness and high efficiency of affine ICP algorithm based on correntropy are demonstrated by 2D and 3D point set registration experiments.

Scale‐aware camera localization in 3D LiDAR maps with a monocular visual odometry

AbstractLocalization information is essential for mobile robot systems in navigation tasks. Many visual‐based approaches focus on localizing a robot within prior maps acquired with cameras. It is critical where the Global Positioning System signal is unreliable. In contrast to conventional methods that localize a camera in an image‐based map, we propose a novel approach that localizes a monocular camera within a given three‐dimensional (3D) light detection and ranging (LiDAR) map. We employ visual odometry to reconstruct a semidense set of 3D points from the monocular camera images. These points are continuously matched against the 3D prior LiDAR map by a modified feature‐based point cloud registration method to track a full six‐degree‐of‐freedom camera pose. Since the monocular camera suffers from the scale‐drift problem due to the lack of depth information, the proposed method solves it by adopting updatable scale estimation. Experiments carried out on a publicly large‐scale data set demonstrate that the camera and LiDAR multimodal data matching problem is solved, and the localization accuracy of our method is comparable to state‐of‐the‐art approaches.

Generic Primitive Detection in Point Clouds Using Novel Minimal Quadric Fits.

We present a novel and effective method for detecting 3D primitives in cluttered, unorganized point clouds, without axillary segmentation or type specification. We consider the quadric surfaces for encapsulating the basic building blocks of our environments - planes, spheres, ellipsoids, cones or cylinders, in a unified fashion. Moreover, quadrics allow us to model higher degree of freedom shapes, such as hyperboloids or paraboloids that could be used in non-rigid settings. We begin by contributing two novel quadric fits targeting 3D point sets that are endowed with tangent space information. Based upon the idea of aligning the quadric gradients with the surface normals, our first formulation is exact and requires as low as four oriented points. The second fit approximates the first, and reduces the computational effort. We theoretically analyze these fits with rigor, and give algebraic and geometric arguments. Next, by re-parameterizing the solution, we devise a new local Hough voting scheme on the null-space coefficients that is combined with RANSAC, reducing the complexity from O(N4) to O(N3) (three points). To the best of our knowledge, this is the first method capable of performing a generic cross-type multi-object primitive detection in difficult scenes without segmentation. Our extensive qualitative and quantitative results show that our method is efficient and flexible, as well as being accurate.

Graph-Based Static 3D Point Clouds Geometry Coding

Recently, 3D visual representation models such as light fields and point clouds are becoming popular due to their capability to represent the real world in a more complete and immersive way, paving the road for new and more advanced visual experiences. The point cloud representation model is able to efficiently represent the surface of objects/scenes by means of a set of 3D points and associated attributes and is increasingly being used from autonomous cars to augmented reality. Emerging imaging sensors have made it easier to perform richer and denser point cloud acquisitions, notably with millions of points, making it impossible to store and transmit these very high amounts of data without appropriate coding. This bottleneck has raised the need for efficient point cloud coding solutions in order to offer more immersive visual experiences and better quality of experience to the users. In this context, this paper proposes an efficient lossy coding solution for the geometry of static point clouds. The proposed coding solution uses an octree-based approach for a base layer and a graph-based transform approach for the enhancement layer where an Inter-layer residual is coded. The performance assessment shows very significant compression gains regarding the state-of-the-art, especially for the most relevant lower and medium rates.

An Accelerated Procrustean Markov Process Model With Coherent Constraint for Non-Rigid Structure From Motion

Non-Rigid Structure from Motion (NRSfM) is the task of reconstructing the 3D point set of a non-rigid object from an ensemble of images with 2D correspondences, which has been a long-lasting challenging research topic. Compared to the state-of-the-art methods for NRSfM, the Procrustean Markov Process (PMP) model has obtained a relatively good performance. However, the estimation error and the convergence time of the PMP model will increase simultaneously when noise is present. To address this problem, in this paper, a coherent constraint is constructed to suppress the noise in the initialization step of the PMP algorithm. Moreover, an Accelerated Expectation Maximization (AEM) algorithm is devised to optimize the PMP estimation model. Experimental results on several widely used sequences demonstrate that our proposed algorithm achieves state-of-the-art performance, as well as its effectiveness and feasibility.

Object-Based Crack Detection and Attribute Extraction From Laser-Scanning 3D Profile Data

Cracks in 3D pavement data often show poor continuity, low contrast and different depths, which bring great challenges to related application. Recently, crack attributes, e.g. depth and width have attracted attention of highway agencies for maintenance decision-makings, but few studies have been conducted on crack attributes. This paper presents object-based image analysis (OBIA) method for crack detection and attribute extraction from laser-scanning 3D profile data with elevation accuracy about 0.25 mm. Firstly, a high-pass filter designed for pavement components in our previous research was applied to remove the fluctuation posture in 3D data, and then the smallest of-constant false-alarm rate algorithm was used to acquire lower point sets, including crack seeds and lower textures. Secondly, the objects were represented by above obtained 3D point sets and OBIA, especially, the depth statistics, shape and topological features of objects were described. Moreover, to enhance crack objects and remove texture objects gradually, multi-scale object selections and merges were conducted according to the local statistical characteristics differences of objects. Thirdly, the objects' orientation attributes were combined with tensor voting to connect and infer final crack objects, and then the object-level crack depth attributes could be extracted. The experimental results demonstrated that proposed method achieved average buffered Hausdorff scores of 94.39, Recall of 0.92 and F -value of 0.91 for crack detection on 30 real measured 3D asphalt pavement data. Furthermore, crack depth attributes can be extracted at different scales according requirements, the obtained location and depth attributes provide more comprehensive information for pavement maintenances.

2D-3D Point Set Registration Based on Global Rotation Search.

Simultaneously determining the relative pose and correspondence between a set of 3D points and its 2D projection is a fundamental problem in computer vision, and the problem becomes more difficult when the point sets are contaminated by noise and outliers. Traditionally, this problem is solved by local optimization methods, which usually start from an initial guess of the pose and alternately optimize the pose and the correspondence. In this paper, we formulate the problem as optimizing the pose of the 3D points in the SE(3) space to make its 2D projection best align with the 2D point set, which is measured by the cardinality of the inlier set on the 2D projection plane. We propose four geometric bounds for the position of the projection of a 3D point on the 2D projection plane and solve the 2D-3D point set registration problem by combining a global optimal rotation search and a grid search of translation. Compared with existing global optimization approaches, the proposed method utilizes a different problem formulation and more efficiently searches the translation space, which improves the registration speed. Experiments with synthetic and real data showed that the proposed approach significantly outperformed state-of-the-art local and global methods.

Determining the Similarity Between Two Arbitrary 2-D Shapes and Its Application to Biological Objects

Generalized shape models of objects are necessary to match and identify an object in an image. To acquire these kind of models special methods are necessary that allow to learn the similarity pair-wise similarity between shapes. They mainly concern is the establishment of point correspondences between two shapes and the detection of outlier. Known algorithm assume that the aligned shapes are quite similar in a way. But special problems arise if we have to align shapes that are very different, for example aligning concave to convex shapes. In such cases it is indispensable to take into account the order of the pointsets and to enforce legal sets of correspondences; otherwise the calculated distances are incorrect. We present our novel shape alignment algorithm which can also handle such cases. The algorithm establishes symmetric and legal one-to-one point correspondences between arbitrary shapes, represented as ordered sets of 2D-points and returns a distance measure which runs between 0 and 1.

Enhancement of sparse 3D reconstruction using a modified match propagation based on particle swarm optimization

Sparse 3D reconstruction, based on interest points detection and matching, does not allow to obtain a suitable 3D surface reconstruction because of its incapacity to recover a cloud of well distributed 3D points on the surface of objects/scenes. In this work, we present a new approach to retrieve a 3D point cloud that leads to a 3D surface model of quality and in a suitable time. First of all, our method uses the structure from motion approach to retrieve a set of 3D points (which correspond to matched interest points). After that, we proposed an algorithm, based on the match propagation and the use of particle swarm optimization (PSO), which significantly increases the number of matches and to have a regular distribution of these matches. It takes as input the obtained matches, their corresponding 3D points and the camera parameters. Afterwards, at each time, a match of best ZNCC value is selected and a set of these neighboring points is defined. The point corresponding to a neighboring point and its 3D coordinates are recovered by the minimization of a nonlinear cost function by the use of PSO algorithm respecting the constraint of photo-consistency. Experimental results show the feasibility and efficiency of the proposed approach.

Stereo Camera Head-Eye Calibration Based on Minimum Variance Approach Using Surface Normal Vectors.

This paper presents a stereo camera-based head-eye calibration method that aims to find the globally optimal transformation between a robot’s head and its eye. This method is highly intuitive and simple, so it can be used in a vision system for humanoid robots without any complex procedures. To achieve this, we introduce an extended minimum variance approach for head-eye calibration using surface normal vectors instead of 3D point sets. The presented method considers both positional and orientational error variances between visual measurements and kinematic data in head-eye calibration. Experiments using both synthetic and real data show the accuracy and efficiency of the proposed method.

Incremental Labelling of Voronoi Vertices for Shape Reconstruction

AbstractWe present an incremental Voronoi vertex labelling algorithm for approximating contours, medial axes and dominant points (high curvature points) from 2D point sets. Though there exist many number of algorithms for reconstructing curves, medial axes or dominant points, a unified framework capable of approximating all the three in one place from points is missing in the literature. Our algorithm estimates the normals at each sample point through poles (farthest Voronoi vertices of a sample point) and uses the estimated normals and the corresponding tangents to determine the spatial locations (inner or outer) of the Voronoi vertices with respect to the original curve. The vertex classification helps to construct a piece‐wise linear approximation to the object boundary. We provide a theoretical analysis of the algorithm for points non‐uniformly (ε‐sampling) sampled from simple, closed, concave and smooth curves. The proposed framework has been thoroughly evaluated for its usefulness using various test data. Results indicate that even sparsely and non‐uniformly sampled curves with outliers or collection of curves are faithfully reconstructed by the proposed algorithm.

Globally-Optimal Inlier Set Maximisation for Camera Pose and Correspondence Estimation

Estimating the 6-DoF pose of a camera from a single image relative to a 3D point-set is an important task for many computer vision applications. Perspective-n-point solvers are routinely used for camera pose estimation, but are contingent on the provision of good quality 2D-3D correspondences. However, finding cross-modality correspondences between 2D image points and a 3D point-set is non-trivial, particularly when only geometric information is known. Existing approaches to the simultaneous pose and correspondence problem use local optimisation, and are therefore unlikely to find the optimal solution without a good pose initialisation, or introduce restrictive assumptions. Since a large proportion of outliers and many local optima are common for this problem, we instead propose a robust and globally-optimal inlier set maximisation approach that jointly estimates the optimal camera pose and correspondences. Our approach employs branch-and-bound to search the 6D space of camera poses, guaranteeing global optimality without requiring a pose prior. The geometry of SE(3) is used to find novel upper and lower bounds on the number of inliers and local optimisation is integrated to accelerate convergence. The algorithm outperforms existing approaches on challenging synthetic and real datasets, reliably finding the global optimum, with a GPU implementation greatly reducing runtime.

Accurate and Robust Non-rigid Point Set Registration using Student\u2019s-t Mixture Model with Prior Probability Modeling

A new accurate and robust non-rigid point set registration method, named DSMM, is proposed for non-rigid point set registration in the presence of significant amounts of missing correspondences and outliers. The key idea of this algorithm is to consider the relationship between the point sets as random variables and model the prior probabilities via Dirichlet distribution. We assign the various prior probabilities of each point to its correspondences in the Student’s-t mixture model. We later incorporate the local spatial representation of the point sets by representing the posterior probabilities in a linear smoothing filter and get closed-form mixture proportions, leading to a computationally efficient registration algorithm comparing to other Student’s-t mixture model based methods. Finally, by introducing the hidden random variables in the Bayesian framework, we propose a general mixture model family for generalizing the mixture-model-based point set registration, where the existing methods can be considered as members of the proposed family. We evaluate DSMM and other state-of-the-art finite mixture models based point set registration algorithms on both artificial point set and various 2D and 3D point sets, where DSMM demonstrates its statistical accuracy and robustness, outperforming the competing algorithms.

Weighted regularized statistical shape space projection for breast 3D model reconstruction.

The use of 3D imaging has increased as a practical and useful tool for plastic and aesthetic surgery planning. Specifically, the possibility of representing the patient breast anatomy in a 3D shape and simulate aesthetic or plastic procedures is a great tool for communication between surgeon and patient during surgery planning. For the purpose of obtaining the specific 3D model of the breast of a patient, model-based reconstruction methods can be used. In particular, 3D morphable models (3DMM) are a robust and widely used method to perform 3D reconstruction. However, if additional prior information (i.e., known landmarks) is combined with the 3DMM statistical model, shape constraints can be imposed to improve the 3DMM fitting accuracy. In this paper, we present a framework to fit a 3DMM of the breast to two possible inputs: 2D photos and 3D point clouds (scans). Our method consists in a Weighted Regularized (WR) projection into the shape space. The contribution of each point in the 3DMM shape is weighted allowing to assign more relevance to those points that we want to impose as constraints. Our method is applied at multiple stages of the 3D reconstruction process. Firstly, it can be used to obtain a 3DMM initialization from a sparse set of 3D points. Additionally, we embed our method in the 3DMM fitting process in which more reliable or already known 3D points or regions of points, can be weighted in order to preserve their shape information. The proposed method has been tested in two different input settings: scans and 2D pictures assessing both reconstruction frameworks with very positive results.