Point Set Matching Research Articles

AN AUTOMATIC OPTICAL AND SAR IMAGE REGISTRATION METHOD USING ITERATIVE MULTI-LEVEL AND REFINEMENT MODEL

Abstract. Automatic image registration is a vital yet challenging task, particularly for multi-sensor remote sensing images. Given the diversity of the data, it is unlikely that a single registration algorithm or a single image feature will work satisfactorily for all applications. Focusing on this issue, the mainly contribution of this paper is to propose an automatic optical-to-SAR image registration method using –level and refinement model: Firstly, a multi-level strategy of coarse-to-fine registration is presented, the visual saliency features is used to acquire coarse registration, and then specific area and line features are used to refine the registration result, after that, sub-pixel matching is applied using KNN Graph. Secondly, an iterative strategy that involves adaptive parameter adjustment for re-extracting and re-matching features is presented. Considering the fact that almost all feature-based registration methods rely on feature extraction results, the iterative strategy improve the robustness of feature matching. And all parameters can be automatically and adaptively adjusted in the iterative procedure. Thirdly, a uniform level set segmentation model for optical and SAR images is presented to segment conjugate features, and Voronoi diagram is introduced into Spectral Point Matching (VSPM) to further enhance the matching accuracy between two sets of matching points. Experimental results show that the proposed method can effectively and robustly generate sufficient, reliable point pairs and provide accurate registration.

AN AUTOMATIC OPTICAL AND SAR IMAGE REGISTRATION METHOD USING ITERATIVE MULTI-LEVEL AND REFINEMENT MODEL

Automatic image registration is a vital yet challenging task, particularly for multi-sensor remote sensing images. Given the diversity of the data, it is unlikely that a single registration algorithm or a single image feature will work satisfactorily for all applications. Focusing on this issue, the mainly contribution of this paper is to propose an automatic optical-to-SAR image registration method using –level and refinement model: Firstly, a multi-level strategy of coarse-to-fine registration is presented, the visual saliency features is used to acquire coarse registration, and then specific area and line features are used to refine the registration result, after that, sub-pixel matching is applied using KNN Graph. Secondly, an iterative strategy that involves adaptive parameter adjustment for re-extracting and re-matching features is presented. Considering the fact that almost all feature-based registration methods rely on feature extraction results, the iterative strategy improve the robustness of feature matching. And all parameters can be automatically and adaptively adjusted in the iterative procedure. Thirdly, a uniform level set segmentation model for optical and SAR images is presented to segment conjugate features, and Voronoi diagram is introduced into Spectral Point Matching (VSPM) to further enhance the matching accuracy between two sets of matching points. Experimental results show that the proposed method can effectively and robustly generate sufficient, reliable point pairs and provide accurate registration.

Probabilistic Model for Robust Affine and Non-Rigid Point Set Matching.

In this work, we propose a combinative strategy based on regression and clustering for solving point set matching problems under a Bayesian framework, in which the regression estimates the transformation from the model to the sceneand the clustering establishes the correspondence between two point sets. The point set matching model is illustrated by a hierarchical directed graph, and the matching uncertainties are approximated by a coarse-to-fine variational inference algorithm. Furthermore, two Gaussian mixtures are proposed for the estimation of heteroscedastic noise and spurious outliers, and an isotropic or anisotropic covariance can be imposed on each mixture in terms of the transformed model points. The experimental results show that the proposed approach achieves comparable performance to state-of-the-art matching or registration algorithms in terms of both robustness and accuracy.

Robust Point Set Matching for Partial Face Recognition.

Over the past three decades, a number of face recognition methods have been proposed in computer vision, and most of them use holistic face images for person identification. In many real-world scenarios especially some unconstrained environments, human faces might be occluded by other objects, and it is difficult to obtain fully holistic face images for recognition. To address this, we propose a new partial face recognition approach to recognize persons of interest from their partial faces. Given a pair of gallery image and probe face patch, we first detect keypoints and extract their local textural features. Then, we propose a robust point set matching method to discriminatively match these two extracted local feature sets, where both the textural information and geometrical information of local features are explicitly used for matching simultaneously. Finally, the similarity of two faces is converted as the distance between these two aligned feature sets. Experimental results on four public face data sets show the effectiveness of the proposed approach.

Three-Layer Convex Network for Domain Adaptation in Multitemporal VHR Images

In this letter, we propose a novel three-layer convex network termed as 3CN for domain adaptation in multitemporal very high resolution (VHR) remote sensing images. 3CN is composed of three main layers: 1) mapping source training samples to the target domain via a special single-layer feedforward neural network called extreme learning machine (ELM); 2) target image classification via ELM too; and 3) spatial regularization via the random-walker algorithm, which models the target image as a lattice graph and then minimizes an energy functional. This network is convex because all three layers have closed-form solutions. In the preprocessing step, we use scale-invariant feature transform to extract a set of matching key points called inliers from source and target images. Then, these inliers are used by layer 1 of 3CN to spectrally map the source training samples to the target domain. Next, in layer 2, we use the mapped training set to classify the target image. In layer 3, we exploit the spatial contextual information in the target image to reduce noise and generate an improved classification map. In the final step, we iteratively fine-tune the network to increase its discrimination ability and reduce the shift between the target and source domains. In the experiments, we report and discuss the results of the proposed method on two data sets of VHR image pairs acquired by IKONOS-2 and GeoEye-1.

New Binary Hausdorff Symmetry measure based seeded region growing for retinal vessel segmentation

Automated retinal vessel segmentation plays an important role in computer-aided diagnosis of serious diseases such as glaucoma and diabetic retinopathy. This paper contributes, (1) new Binary Hausdorff Symmetry (BHS) measure based automatic seed selection, and (2) new edge distance seeded region growing (EDSRG) algorithm for retinal vessel segmentation. The proposed BHS measure directly provides a binary symmetry decision at each pixel without the computation of continuous symmetry map and image thresholding. In a multiscale mask, the BHS measure is computed using the distance sets of opposite direction angle bins with sub-pixel resolution. The computation of the BHS measure from the Hausdorff distance sets involves point set matching based geometrical interpretation of symmetry. Then, we design a new edge distance seeded region growing (EDSRG) algorithm with the acquired seeds. The performance evaluation in terms of sensitivity, specificity and accuracy is done on the publicly available DRIVE, STARE and HRF databases. The proposed method is found to achieve state-of-the-art vessel segmentation accuracy in three retinal databases; DRIVE-sensitivity (0.7337), specificity (0.9752), accuracy (0.9539); STARE-sensitivity (0.8403), specificity (0.9547), accuracy (0.9424); and HRF-sensitivity (0.8159), specificity (0.9525), accuracy (0.9420).

Reconstruction system of individual virtual model based on user interaction

Online fashion shopping platforms presenting 2D images or adjustable fitting models cannot meet individual fitting demands, especially special body types. In this paper, we present a novel virtual reconstruction system based on multiview stereo (MVS) which can rapidly output 3D point cloud and reconstruct body shape. Concretely, overlapped photos were taken with interaction from different views to cover the whole body. The larger set of disordered photos then were serialized as sequence images and a sparse set of matched key points were detected based on image scale-invariant feature transform (SIFT). Multi-View stereopsis based on image features was used to outline dense cloud points. The proposed algorithm can automatically reconstruct individual body and output dense 3D points without expensive apparatus and special scanning environment. It also can be interfaced with major measurement software for individual body measurement, which is useful to consumers for just-in-time measurements and online fashion shopping.

Forward Affine Point Set Matching Under Variational Bayesian Framework

In this work, the affine point set matching is formulated under a variational Bayesian framework and the model points are projected forward into the scene space by a linear transformation. A directed acyclic graph is presented to represent the relationship between the parameters, latent variables, model and scene point sets and an iterative approximate algorithm is proposed for the estimation of the posterior distributions over parameters. Furthermore, the anisotropic covariance is assumed on the transition variable and one Gaussian component is provided for the inference of outlier points. Experimental results demonstrate that the proposed algorithm achieves good performance in terms of both robustness and accuracy.

Robust Point Sets Matching by Fusing Feature and Spatial Information Using Nonuniform Gaussian Mixture Models.

Most of the traditional methods that handle the point sets matching between two images are based on local feature descriptors and the succedent mismatch eliminating strategies, which usually suffers from the sparsity of the initial match set because some correct ambiguous associations are easily filtered out by the ratio test of SIFT matching due to their second ranking in feature similarity. In this paper, we propose a nonuniform Gaussian mixture model (NGMM) for point sets matching between a pair of images which combines feature with position information of the local feature points extracted from the image pair to achieve point sets matching in a GMM framework. The proposed point set matching using an NGMM is able to change the correspondence assignments throughout the matching process and has the potential to match up even ambiguous matches correctly. The proposed NGMM framework can be either used to directly find matches between two point sets obtained from two images or applied to remove outliers in a match set. When finding matches, NGMM tries to learn a nonrigid transformation between the two point sets and provide a probability for every found match to measure the reliability of the match. Then, a probability threshold can be used to get the final robust match set. When removing outliers, NGMM requires that the vector field formed by the correct matches to be coherent and the matches contradicting the coherent vector field will be regarded as mismatches to be removed. A number of comparison and evaluation experiments reveal the good performance of the proposed NGMM framework in both finding matches and discarding mismatches.

Improved Grid Map Layout by Point Set Matching

Associating the regions of a geographic subdivision with the cells of a grid is a basic operation that is used in various types of maps, like spatially ordered treemaps and Origin-Destination maps (OD maps). In these cases the regular shapes of the grid cells allow easy representation of extra information about the regions. The main challenge is to find an association that allows a user to find a region in the grid quickly. We call the representation of a set of regions as a grid a grid map.We introduce a new approach to solve the association problem for grid maps by formulating it as a point set matching problem: Given two sets [Formula: see text] (the centroids of the regions) and [Formula: see text] (the grid centres) of [Formula: see text] points in the plane, compute an optimal one-to-one matching between [Formula: see text] and [Formula: see text]. We identify three optimisation criteria that are important for grid map layout: maximise the number of adjacencies in the grid that are also adjacencies of the regions, minimise the sum of the distances between matched points, and maximise the number of pairs of points in [Formula: see text] for which the matching preserves the directional relation (SW, NW, etc.). We consider matchings that minimise the [Formula: see text]-distance (Manhattan-distance), the ranked [Formula: see text]-distance, and the [Formula: see text]-distance, since one can expect that minimising distances implicitly helps to fulfill the other criteria.We present algorithms to compute such matchings and perform an experimental comparison that also includes a previous method to compute a grid map. The experiments show that our more global, matching-based algorithm outperforms previous, more local approaches with respect to all three optimisation criteria.

Robust rigid coherent point drift algorithm based on outlier suppression and its application in image matching

The typical probability based point pattern matching method is coherent point drift (CPD) algorithm, which treats one point set as centroids of a Gaussian mixture model, and then fits it to the other. It uses the expectation maximization framework, where the point correspondences and transformation parameters are updated alternately. However, the anti-outlier performance of CPD is not robust enough as outliers have always been involved in the operation until the CPD converges. Hence, an automatic outlier suppression (AOS) mechanism is proposed. First, outliers are judged by a matching probability matrix. Then, transformation parameters are fitted using accurate matching point sets. Finally, the Gaussian centroids are forced to move coherently by this transformation model. AOS-CPD can efficiently improve the anti-outlier performance of rigid CPD. Furthermore, CPD is applied to image matching. A new local changing information descriptor-relative phase histogram (RPH) is designed and RPH-AOS-CPD is proposed to embed RPH measurement into AOS-CPD as a constraint condition. RPH-AOS-CPD makes full use of grayscale information besides having an excellent anti-outlier performance. The experimental results based on both synthetic and real data indicate that compared with other algorithms, AOS-CPD is more robust to outliers and RPH-AOS-CPD offers a good practicability and accuracy in image matching applications.

Variational Bayesian Approximation for Affine Point Set Matching

In this paper, we propose a variational approach for the affine point set matching problems under the Bayesian probabilistic framework. A directed acyclic graphis provided for the representation of the joint probability over affine transformation, random variables and the point sets. Based on the directed graph, a variational iterative algorithm is derived to approximate the posteriors of the random variables and the anisotropic Gaussian mixtures are used for the estimation of the spurious outliers instead of the frequently-used uniform distribution. Experimental results demonstrate that our method achieves good performance in terms of both robustness and accuracy and is comparable to other state-of-the-art point registration algorithms especially in the case of complicated outliers.

Extraction of ground surface elevation from ZY-3 winter stereo imagery over deciduous forested areas

Forest height is an important indicator for forest biomass. Ground surface elevation is essential to derive forest height from spaceborne data including interferometric synthetic aperture radar or stereo imagery. Considering the good performance of stereo images in characterizing the vertical structure of forest with non-closed canopies, the main issue addressed in this study is whether stereo imagery acquired in winter can view the ground surface under dense deciduous forest. To make full use of information provided by different observation geometries, the three sets of matching points from different view were combined. Then the vertical distribution of matching points from stereo images was referenced to the vegetation vertical structure and the ground surface elevation from airborne laser scanner (ALS) data. The vertical distribution of matching points from stereo images over typical deciduous forest stands including sparse, disturbed and dense forests were examined. Most matching points were located on the ground surface while some points came from branches and trunks in all the three forest stands. This phenomenon was also observed from a transect of a digital surface model from ZY-3. Twelve elevation indices, including minimum, maximum, mean elevations and an additional nine percentiles of cumulative probability (E10 to E90) from the matching points of ZY-3 over 30m×30m cell were compared with the ground surface elevation from ALS data. The results showed that E30 gave the best measurement of ground surface elevation with R2>0.99 and RMSE=2.54m.

Extracting Corresponding Point Based on Texture Synthesis for Nearly Flat Textureless Object Surface

Since the image feature points are always gathered at the range with significant intensity change, such as textured portions or edges of an image, which can be detected by the state-of-the-art intensity based point-detectors, there is nearly no point in the areas of low textured detected by classical interest-point detectors. In this paper we describe a novel algorithm based on affine transform and graph cut for interest point detecting and matching from wide baseline image pairs with weakly textured object. The detection and matching mechanism can be separated into three steps: firstly, the information on the large textureless areas will be enhanced by adding textures through the proposed texture synthesis algorithm TSIQ. Secondly, the initial interest-point set is detected by classical interest-point detectors. Finally, graph cuts are used to find the globally optimal set of matching points on stereo pairs. The efficacy of the proposed algorithm is verified by three kinds of experiments, that is, the influence of point detecting from synthetic texture with different texture sample, the stability under the different geometric transformations, and the performance to improve the quasi-dense matching algorithm, respectively.

Image Registration Technique Based on a Fast CPD Algorithm

The Coherent Point Drift (CPD) algorithm which based on Gauss Mixture Model is a robust point set registration algorithm. However, the selection of robustness weight which used to describe the noise may directly affect the point set registration efficiency. For resolving the problem, this paper presents a CPD registration algorithm which based on distance threshold constraint. Before the point set registration, the inaccurate template point set by resampling become the initial point set of point set matching, in order to eliminate some points that the distance to target point set is too close and too far in the inaccurate template point set, and set the weights of robustness as . In the simulation experiments, we make two group experiments: the first group is the registration of the inaccurate template point set and the accurate target point set, while the second group is the registration of the accurate template point set and the accurate target point set. The results of comparison show that our method can solve the problem of selection for the weight. And it improves the speed and precision of the original CPD registration.

Discriminative and generative vocabulary tree: With application to vein image authentication and recognition

Finger vein identification is a new biometric identification technology. While many existing works approach the problem by using shape matching which is the generative method, in this paper, we introduce a joint discriminative and generative algorithm for the task. Our method considers both the discriminative appearance of local image patches as well as their generative spatial layout. The method is based on the popular vocabulary tree model, where we utilize the hidden leaf node layer to calculate a generative confidence to weight the discriminative vote from the leaf node. The training process remains the same as building a conventional vocabulary tree, while the prediction process utilizes a proposed point set matching method to support non-parametric patch layout matching. In this way, the entire model retains the efficiency of the vocabulary tree model, which is much lighter than other similar models such as the constellation model (Fergus et al., 2003). The overall estimation follows the Bayesian theory. Experimental results show that our proposed joint model outperformed the purely generative or discriminative counterpart, and can offer competitive performance than existing methods for both the vein authentication and recognition tasks.

Parameter Optimization for the Extraction of Matching Points Between High-Resolution Multisensor Images in Urban Areas

The objective of this paper is to extract a suitable number of evenly distributed matched points, given the characteristics of the site and the sensors involved. The intent is to increase the accuracy of automatic image-to-image registration for high-resolution multisensor data. The initial set of matching points is extracted using a scale-invariant feature transform (SIFT)-based method, which is further used to evaluate the initial geometric relationship between the features of the reference and sensed images. The precise matching points are extracted considering location differences and local properties of features. The values of the parameters used in the precise matching are optimized using an objective function that considers both the distribution of the matching points and the reliability of the transformation model. In case studies, the proposed algorithm extracts an appropriate number of well-distributed matching points and achieves a higher correct-match rate than the SIFT method. The registration results for all sensors are acceptably accurate, with a root-mean-square error of less than 1.5 m.

A Riemannian framework for matching point clouds represented by the Schrödinger distance transform.

In this paper, we cast the problem of point cloud matching as a shape matching problem by transforming each of the given point clouds into a shape representation called the Schrödinger distance transform (SDT) representation. This is achieved by solving a static Schrödinger equation instead of the corresponding static Hamilton-Jacobi equation in this setting. The SDT representation is an analytic expression and following the theoretical physics literature, can be normalized to have unit L2 norm-making it a square-root density, which is identified with a point on a unit Hilbert sphere, whose intrinsic geometry is fully known. The Fisher-Rao metric, a natural metric for the space of densities leads to analytic expressions for the geodesic distance between points on this sphere. In this paper, we use the well known Riemannian framework never before used for point cloud matching, and present a novel matching algorithm. We pose point set matching under rigid and non-rigid transformations in this framework and solve for the transformations using standard nonlinear optimization techniques. Finally, to evaluate the performance of our algorithm-dubbed SDTM-we present several synthetic and real data examples along with extensive comparisons to state-of-the-art techniques. The experiments show that our algorithm outperforms state-of-the-art point set registration algorithms on many quantitative metrics.

Classification of normal and diseased liver shapes based on Spherical Harmonics coefficients.

Liver-shape analysis and quantification is still an open research subject. Quantitative assessment of the liver is of clinical importance in various procedures such as diagnosis, treatment planning, and monitoring. Liver-shape classification is of clinical importance for corresponding intra-subject and inter-subject studies. In this research, we propose a novel technique for the liver-shape classification based on Spherical Harmonics (SH) coefficients. The proposed liver-shape classification algorithm consists of the following steps: (a) Preprocessing, including mesh generation and simplification, point-set matching, and surface to template alignment; (b) Liver-shape parameterization, including surface normalization, SH expansion followed by parameter space registration; (c) Feature selection and classification, including frequency based feature selection, feature space reduction by Principal Component Analysis (PCA), and classification. The above multi-step approach is novel in the sense that registration and feature selection for liver-shape classification is proposed and implemented and validated for the normal and diseases liver in the SH domain. Various groups of SH features after applying conventional PCA and/or ordered by p-value PCA are employed in two classifiers including Support Vector Machine (SVM) and k-Nearest Neighbor (k-NN) in the presence of 101 liver data sets. Results show that the proposed specific features combined with classifiers outperform existing liver-shape classification techniques that employ liver surface information in the spatial domain. In the available data sets, the proposed method can successful classify normal and diseased livers with a correct classification rate of above 90 %. The performed result in average is higher than conventional liver-shape classification method. Several standard metrics such as Leave-one-out cross-validation and Receiver Operating Characteristic (ROC) analysis are employed in the experiments and confirm the effectiveness of the proposed liver-shape classification with respect to conventional techniques.

A High Performance Edge Point Feature Match Method of SAR Images

摘要: 针对合成孔径雷达（Synthetic aperture radar，SAR）图像特征匹配中特征提取的不稳定性和相似度优化搜索的复杂性问题，提出了一种精确高效稳健的SAR图像边缘点集匹配方法. 首先，分析了仿射变换模型在遥感图像匹配中的适应性，并对仿射变换模型进行了参数分解；其次，提出了基于方向模板的SAR图像边缘检测算子，并利用SAR图像边缘的梯度和方向特征，建立了基于像素迁移的多源SAR边缘点集相似性匹配准则，以及图像匹配的联合相似度-联合特征均方和（Square summation joint feature，SSJF）；然后，利用改进的遗传算法（Genetic algorithm，GA）来进行相似度的全局极值优化搜索，获取变换模型参数和边缘点集的对应关系；最后，从理论上分析了本文方法的性能，并利用多幅SAR图像的匹配实验以及与原有方法的对比分析，对本文方法的性能进行了验证. 关键词: 合成孔径雷达图像匹配 / 仿射变换模型 / 参数分解 / 像素迁移 / 联合相似测度 / 遗传算法