Non-rigid Point Set Registration Research Articles

Nonrigid point set registration based on Laplace mixture model with local constraints

PurposeThis paper aims to investigate a probabilistic mixture model for the nonrigid point set registration problem in the computer vision tasks. The equations to estimate the mixture model parameters and the constraint items are derived simultaneously in the proposed strategy.Design/methodology/approachThe problem of point set registration is expressed as Laplace mixture model (LMM) instead of Gaussian mixture model. Three constraint items, namely, distance, the transformation and the correspondence, are introduced to improve the accuracy. The expectation-maximization (EM) algorithm is used to optimize the objection function and the transformation matrix and correspondence matrix are given concurrently.FindingsAlthough amounts of the researchers study the nonrigid registration problem, the LMM is not considered for most of them. The nonrigid registration problem is considered in the LMM with the constraint items in this paper. Three experiments are performed to verify the effectiveness and robustness and demonstrate the validity.Originality/valueThe novel method to solve the nonrigid point set registration problem in the presence of the constraint items with EM algorithm is put forward in this work.

EasyFont

Generating personal handwriting fonts with large amounts of characters is a boring and time-consuming task. For example, the official standard GB18030-2000 for commercial font products consists of 27,533 Chinese characters. Consistently and correctly writing out such huge amounts of characters is usually an impossible mission for ordinary people. To solve this problem, we propose a system, EasyFont , to automatically synthesize personal handwriting for all (e.g., Chinese) characters in the font library by learning style from a small number (as few as 1%) of carefully-selected samples written by an ordinary person. Major technical contributions of our system are twofold. First, we design an effective stroke extraction algorithm that constructs best-suited reference data from a trained font skeleton manifold and then establishes correspondence between target and reference characters via a non-rigid point set registration approach. Second, we develop a set of novel techniques to learn and recover users’ overall handwriting styles and detailed handwriting behaviors. Experiments including Turing tests with 97 participants demonstrate that the proposed system generates high-quality synthesis results, which are indistinguishable from original handwritings. Using our system, for the first time, the practical handwriting font library in a user’s personal style with arbitrarily large numbers of Chinese characters can be generated automatically. It can also be observed from our experiments that recently-popularized deep learning based end-to-end methods are not able to properly handle this task, which implies the necessity of expert knowledge and handcrafted rules for many applications.

Nonrigid Point Set Registration With Robust Transformation Learning Under Manifold Regularization.

This paper solves the problem of nonrigid point set registration by designing a robust transformation learning scheme. The principle is to iteratively establish point correspondences and learn the nonrigid transformation between two given sets of points. In particular, the local feature descriptors are used to search the correspondences and some unknown outliers will be inevitably introduced. To precisely learn the underlying transformation from noisy correspondences, we cast the point set registration into a semisupervised learning problem, where a set of indicator variables is adopted to help distinguish outliers in a mixture model. To exploit the intrinsic structure of a point set, we constrain the transformation with manifold regularization which plays a role of prior knowledge. Moreover, the transformation is modeled in the reproducing kernel Hilbert space, and a sparsity-induced approximation is utilized to boost efficiency. We apply the proposed method to learning motion flows between image pairs of similar scenes for visual homing, which is a specific type of mobile robot navigation. Extensive experiments on several publicly available data sets reveal the superiority of the proposed method over state-of-the-art competitors, particularly in the context of the degenerated data.

Adaptive Non-Rigid Point Set Registration Based on Variational Bayesian

For the existence of outliers in non-rigid point set registration, a method based on Bayesian student's t mixture model(SMM) is proposed. Under the framework of variational Bayesian, the point set registration problem is converted to maximize the variational lower bound of log-likelihood, where the transformation parameters are found through variational inference. By prior model, the constraint over spatial regularization is incorporated into the Bayesian SMM, which can adaptively be determined for different data sets. Compared with Gaussian distribution, the student's t distribution is more robust to outliers. The experimental comparative analysis of simulated points and real images verify the effectiveness of the proposed method on the non-rigid point set registration with outliers.

Non‐rigid point set registration by high‐dimensional representation

Non-rigid point set registration is a key component in many computer vision and pattern recognition tasks. In this study, the authors propose a robust non-rigid point set registration method based on high-dimensional representation. Their central idea is to map the point sets into a high-dimensional space by integrating the relative structure information into the coordinates of points. On the one hand, the point set registration is formulated as the estimation of a mixture of densities in high-dimensional space. On the other hand, the relative distances are used to compute the local features which assign the membership probabilities of the mixture model. The proposed model captures discriminative relative information and enables to preserve both global and local structures of the point set during matching. Extensive experiments on both synthesised and real data demonstrate that the proposed method outperforms the state-of-the-art methods under various types of distortions, especially for the deformation and rotation degradations.

Correction: Robust CPD Algorithm for Non-Rigid Point Set Registration Based on Structure Information.

[This corrects the article DOI: 10.1371/journal.pone.0148483.].

Multifeature energy optimization framework and parameter adjustment-based nonrigid point set registration

Nonrigid point set registration is a key technology in the field of remote sensing image registration, which is widely used in military and civil fields such as natural disaster damage assessment, agricultural and urban land-use planning, environmental quality monitoring, and ground target identification. We present a multifeature energy optimization framework and parameter adjustment-based nonrigid point set registration that has three contributions: (1) an energy optimization framework is designed to freely combine multiple features for estimating correspondences between two point sets, (2) the thin-plate spline and Gauss radial basis function transformation models can be optionally implemented for solving two-dimensional, three-dimensional, or higher-dimensional registration problems, and (3) the free parameters can be adjusted by the proposed three parameter adjustment approaches to yield higher registration accuracy in varied registration patterns. We test the performances of our method in contour point sets, sequence images, remote sensing images, medical images, and real images and compare with 10 state-of-the-art methods, where our method shows favorable performances in most scenarios.

Non-rigid point set registration via global and local constraints

Non-rigid point set registration is often encountered in meical image processing, pattern recognition, and computer vision. This paper presents a new method for non-rigid point set registration that can be used to recover the underlying coherent spatial mapping (CSM). Firstly, putative correspondences between two point sets are established by using feature descriptors. Secondly, each point is expressed as a weighted sum of several nearest neighbors and the same relation holds after the transformation. Then, this local geometrical constraint is combined with the global model, and the transformation problem is solved by minimizing an error function. These two steps of recovering point correspondences and transformation are performed iteratively to obtained a promising result. Extensive experiments on various synthetic and real data demonstrate that the proposed approach is robust and outperforms the state-of-the-art methods.

Accurate and Robust Non-rigid Point Set Registration using Student\u2019s-t Mixture Model with Prior Probability Modeling

A new accurate and robust non-rigid point set registration method, named DSMM, is proposed for non-rigid point set registration in the presence of significant amounts of missing correspondences and outliers. The key idea of this algorithm is to consider the relationship between the point sets as random variables and model the prior probabilities via Dirichlet distribution. We assign the various prior probabilities of each point to its correspondences in the Student’s-t mixture model. We later incorporate the local spatial representation of the point sets by representing the posterior probabilities in a linear smoothing filter and get closed-form mixture proportions, leading to a computationally efficient registration algorithm comparing to other Student’s-t mixture model based methods. Finally, by introducing the hidden random variables in the Bayesian framework, we propose a general mixture model family for generalizing the mixture-model-based point set registration, where the existing methods can be considered as members of the proposed family. We evaluate DSMM and other state-of-the-art finite mixture models based point set registration algorithms on both artificial point set and various 2D and 3D point sets, where DSMM demonstrates its statistical accuracy and robustness, outperforming the competing algorithms.

Unsupervised Articulated Skeleton Extraction From Point Set Sequences Captured by a Single Depth Camera

How to robustly and accurately extract articulated skeletons from point set sequences captured by a single consumer-grade depth camera still remains to be an unresolved challenge to date. To address this issue, we propose a novel, unsupervised approach consisting of three contributions (steps): (i) a non-rigid point set registration algorithm to first build one-to-one point correspondences among the frames of a sequence; (ii) a skeletal structure extraction algorithm to generate a skeleton with reasonable numbers of joints and bones; (iii) a skeleton joints estimation algorithm to achieve accurate joints. At the end, our method can produce a quality articulated skeleton from a single 3D point sequence corrupted with noise and outliers. The experimental results show that our approach soundly outperforms state of the art techniques, in terms of both visual quality and accuracy.

Non-rigid point set registration using dual-feature finite mixture model and global-local structural preservation

We present a dual-feature based point set registration method with global-local structural preservation. A finite mixture model which is able to deal with two features is first constructed. The mixture structure descriptor (MSD) is then obtained by smoothly combining the local structure descriptor (LSD) with the original coordinates through an annealing scheme. Substituting the MSD into the constructed model a fuzzy corresponding matrix is acquired. Next, the energy function which contains three main terms is formulated in the reproducing kernel Hilbert space. The first term is based on the L2 estimation (L2E) criterion, and the other two terms play a complementary role to improve the robustness and accuracy for transformation estimation at both global and local scales. The performances of our method in synthetic data, sequence data and real data against nine state-of-the-art methods are tested where our method shows favorable performance when compared with other methods.

Non-Rigid Point Set Registration via Adaptive Weighted Objective Function

Non-rigid point set registration is a fundamental problem in many fields related to computer vision, medical image processing, and pattern recognition. In this paper, we develop a new point set registration method by using an adaptive weighted objective function, which formulates the alignment of two point sets as a mixture model estimation problem. The correspondences and the transformation are jointly recovered by using the expectation–maximization algorithm to obtain the promising results. First, the correspondences are established using local feature descriptors, and the adaptation parameters for the mixture model are computed from these correspondences. Then, the underlying transformation is recovered by minimizing the adaptive weighted objective function deduced from the mixture model. We demonstrate the advantages of the proposed method on various types of synthetic and real data and compare the results against those obtained using the state-of-the-art methods. The experimental results show that the proposed method is robust and outperforms the other registration approaches.

A robust non-rigid point set registration method based on inhomogeneous Gaussian mixture models

In this paper, we propose a novel robust non-rigid point set registration method adopting a new probability model called inhomogeneous Gaussian mixture models (IGMM), where we regard one point set as the centroids of a Gaussian mixture model and the other point set as the data. The IGMM is defined by applying local features and Gaussian mixture models. Considering the local relationship among neighboring points is stable, a neighborhood structural descriptor, named as local shape context, is first presented. On the basis of local descriptors, we can obtain a measure of compatibility between local features in the point sets. Then, the similarity of the local structure of point neighborhoods can be calculated on the basis of the matching scores. Each Gaussian mixture component is assigned a different weight depending on the feature similarity, which differs from the traditional Gaussian mixture model where each Gaussian mixture component has the same weight. The proposed IGMM makes point pairs with more similar features have bigger probability to formulate a match, while in algorithms based on GMMs, all point pairs have the same probability to construct correspondence points. Finally, we support our claims through regularization theory and formulate registration as a likelihood maximization problem, which is solved by updating transformation parameters and outlier ratios using the expectation maximization algorithm. Extensive comparison and evaluation experiments on synthetic point-sets datasets demonstrate that the proposed approach is robust and achieves superior performance in the presence of non-rigid deformation, noise, outliers and occlusion. In addition, a number of experiments on real images reveal that our proposed algorithm is more applicable than state-of-the-art algorithms.

Fuzzy correspondences guided Gaussian mixture model for point set registration

Recovering correspondences and estimating transformations are challenging to solve in the presence of outliers and other degradations for non-rigid point set registration. In this paper, we propose a new methodology based on fuzzy correspondences guided Gaussian Mixture Model (GMM) to solve the registration problem between two or more feature point sets. We first construct a mixture model to represent the moving point set, where inliers are formulated as a mixture of Gaussian, and outliers are formulated as an additional uniform distribution. Then we use the context-aware shape descriptor to assign the points and obtain the fuzzy correspondences. On the one hand, the soft-assignment is used to classify the weight of inliers and outliers. On the other hand, by using the fuzzy correspondences, the Gaussian elements in the mixture model can be estimated to increase the registration accuracy efficiently. In this way, the optimal transformation between two point sets can be expressed by representation theorem and solved by EM algorithm iteratively in a high-dimensional feature space (i.e., reproducing kernel Hilbert space, RKHS). Extensive experiments on synthesized and real datasets demonstrate the proposed method performs favorably against the state-of-the-art methods in most tested scenarios.

Nonrigid Point Set Registration by Preserving Local Connectivity.

This paper is concerned with the nonrigid point set registration problem and a probability-based registration algorithm with local connectivity preservation is proposed. A unified formulation for point set registration problem is introduced and the derived energy function is composed of three parts, distance measurement item, transformation constraint item, and correspondence constraint item. In order to preserve the local structure of point set, the definitions of -connected neighbors and connectivity matrix are given and the local connectivity constraint is constructed as a weighted least square error item. The point set registration problem is formulated in the expectation-maximization algorithm scheme and the optimal spatial transformation and correspondence matrix are estimated simultaneously. The effectiveness of the proposed method is verified by applying the method to synthetic point sets and real scenarios of hand shapes and surface-mount technology components.

Non-Rigid Point Set Registration Based Masticatory Muscle Deformation Measurement

Non-Rigid Point Set Registration Based Masticatory Muscle Deformation Measurement

Automatically tracking neurons in a moving and deforming brain.

Advances in optical neuroimaging techniques now allow neural activity to be recorded with cellular resolution in awake and behaving animals. Brain motion in these recordings pose a unique challenge. The location of individual neurons must be tracked in 3D over time to accurately extract single neuron activity traces. Recordings from small invertebrates like C. elegans are especially challenging because they undergo very large brain motion and deformation during animal movement. Here we present an automated computer vision pipeline to reliably track populations of neurons with single neuron resolution in the brain of a freely moving C. elegans undergoing large motion and deformation. 3D volumetric fluorescent images of the animal’s brain are straightened, aligned and registered, and the locations of neurons in the images are found via segmentation. Each neuron is then assigned an identity using a new time-independent machine-learning approach we call Neuron Registration Vector Encoding. In this approach, non-rigid point-set registration is used to match each segmented neuron in each volume with a set of reference volumes taken from throughout the recording. The way each neuron matches with the references defines a feature vector which is clustered to assign an identity to each neuron in each volume. Finally, thin-plate spline interpolation is used to correct errors in segmentation and check consistency of assigned identities. The Neuron Registration Vector Encoding approach proposed here is uniquely well suited for tracking neurons in brains undergoing large deformations. When applied to whole-brain calcium imaging recordings in freely moving C. elegans, this analysis pipeline located 156 neurons for the duration of an 8 minute recording and consistently found more neurons more quickly than manual or semi-automated approaches.

Non-Rigid Point Set Registration with Robust Transformation Estimation under Manifold Regularization

In this paper, we propose a robust transformation estimation method based on manifold regularization for non-rigid point set registration. The method iteratively recovers the point correspondence and estimates the spatial transformation between two point sets. The correspondence is established based on existing local feature descriptors which typically results in a number of outliers. To achieve an accurate estimate of the transformation from such putative point correspondence, we formulate the registration problem by a mixture model with a set of latent variables introduced to identify outliers, and a prior involving manifold regularization is imposed on the transformation to capture the underlying intrinsic geometry of the input data. The non-rigid transformation is specified in a reproducing kernel Hilbert space and a sparse approximation is adopted to achieve a fast implementation. Extensive experiments on both 2D and 3D data demonstrate that our method can yield superior results compared to other state-of-the-arts, especially in case of badly degraded data.

Robust Non-Rigid Point Set Registration Using Spatially Constrained Gaussian Fields.

Estimating transformations from degraded point sets is necessary for many computer vision and pattern recognition applications. In this paper, we propose a robust non-rigid point set registration method based on spatially constrained context-aware Gaussian fields. We first construct a context-aware representation (e.g., shape context) for assignment initialization. Then, we use a graph Laplacian regularized Gaussian fields to estimate the underlying transformation from the likely correspondences. On the one hand, the intrinsic manifold is considered and used to preserve the geometrical structure, and a priori knowledge of the point set is extracted. On the other hand, by using the deterministic annealing, the presented method is extended to a projected high-dimensional feature space, i.e., reproducing kernel Hilbert space through a kernel trick to solve the transformation, in which the local structure is propagated by the coarse-to-fine scaling strategy. In this way, the proposed method gradually recovers much more correct correspondences, and then estimates the transformation parameters accurately and robustly when facing degradations. Experimental results on 2D and 3D synthetic and real data (point sets) demonstrate that the proposed method reaches better performance than the state-of-the-art algorithms.

Robust non-rigid point set registration via building tree dynamically

The non-rigid registration methods, such as coherent point drift (CPD) method can deal with similar point sets, but it is difficult for them to achieve the non-rigid registration of point sets with large deformations. To overcome the problem, a novel approach via building dynamic tree is proposed in this paper. First of all, the similarity between the model and subject point sets is evaluated by the affine iterative closest point (ICP) algorithm with bidirectional distance, and the models and their similar subjects are connected. Secondly, the non-rigid registration is conducted on every two similar point sets. The subjects with accurate registration results are added to the model sets and wrong pairs are cut off based on a bidirectional distance. These steps are repeated and a dynamic tree is built up. In this way, a large deformation between two images is decomposed into a series of small deformations and the elimination of the wrong pairs in the dynamic tree guarantees the registration results are precise and satisfactory. Experimental results on several image datasets demonstrate that our method improves the accuracy of the point set registration results with large shape difference compared with existing approaches.