Feature-based Image Registration Method Research Articles

An Image Registration Method Using Deep Residual Network Features for Multisource High-Resolution Remote Sensing Images

Accurate registration for multisource high-resolution remote sensing images is an essential step for various remote sensing applications. Due to the complexity of the feature and texture information of high-resolution remote sensing images, especially for images covering earthquake disasters, feature-based image registration methods need a more helpful feature descriptor to improve the accuracy. However, traditional image registration methods that only use local features at low levels have difficulty representing the features of the matching points. To improve the accuracy of matching features for multisource high-resolution remote sensing images, an image registration method based on a deep residual network (ResNet) and scale-invariant feature transform (SIFT) was proposed. It used the fusion of SIFT features and ResNet features on the basis of the traditional algorithm to achieve image registration. The proposed method consists of two parts: model construction and training and image registration using a combination of SIFT and ResNet34 features. First, a registration sample set constructed from high-resolution satellite remote sensing images was used to fine-tune the network to obtain the ResNet model. Then, for the image to be registered, the Shi_Tomas algorithm and the combination of SIFT and ResNet features were used for feature extraction to complete the image registration. Considering the difference in image sizes and scenes, five pairs of images were used to conduct experiments to verify the effectiveness of the method in different practical applications. The experimental results showed that the proposed method can achieve higher accuracies and more tie points than traditional feature-based methods.

A Context-aware Locality Measure for Inlier Pool Enrichment in Stepwise Image Registration.

We present a feature-based image registration method, the stepwise image registration (SIR), with a closed-form solution. Our SIR creates an inlier pool and a candidate pool as the initialization, and then gradually enriches the inlier pool and refines the transformation. In each step, the enriched correspondence exclusively tunes the transformation coefficient within the confirmed inlier pairs, instead of updating the mapping using the complete putative set. In turn, the refined transformation prunes inconsistent mismatches to alleviate the incoming matching ambiguity. The context-aware locality measure (CALM) is designed for dissimilarity measure. The capability of the CALM can be enhanced by the progressive inlier pool enrichment. Finally, a retrieval process is performed based on the finest CALM and alignment, by which the inlier pool is maximized. Extensive experiments of enrichment evaluation, feature matching, image registration, and image retrieval demonstrate the favorable performance of our SIR against state-of-the-art methods. The code and datasets are available at https://github.com/sucv/SIR.

A Reliable Distribution Quality Measure in Image Registration

The accuracy of feature-based image registration methods significantly depends on the number of extracted key-points and the quality of their distribution. Therefore, it is important to examine the quality of distribution of key-points by using a suitable measure. However, in literature, examining distribution of key-points could not get appropriate attention. Very few metrics have been reported that discussed about the distribution quality of the feature points. These metrics consider all detected key-points inliers (correctly matched key-points). Therefore, they may provide reasonable distribution quality measure in the absence of outliers. However, in the presence of outliers, these measures are not compliance with accuracy and may provide misleading quantities. In this paper, we propose a distribution quality metric that can provide a reliable measure for distribution and overcomes the limitations in the existing measures. The proposed measure uses area and shape of Delaunay triangles and incorporates the goodness of the key-points. Experimental results show that the proposed measure can evaluate distribution quality accurately even in the presence of outliers. It is also well-compliance with the registration accuracy.

Evaluation of Distance Measures for Feature based Image Registration using AlexNet

Image registration is a classic problem of computer vision with several applications across areas like defence, remote sensing, medicine etc. Feature based image registration methods traditionally used hand-crafted feature extraction algorithms, which detect key points in an image and describe them using a region around the point. Such features are matched using a threshold either on distances or ratio of distances computed between the feature descriptors. Evolution of deep learning, in particular convolution neural networks, has enabled researchers to address several problems of vision such as recognition, tracking, localization etc. Outputs of convolution layers or fully connected layers of CNN which has been trained for applications like visual recognition are proved to be effective when used as features in other applications such as retrieval. In this work, a deep CNN, AlexNet, is used in the place of handcrafted features for feature extraction in the first stage of image registration. However, there is a need to identify a suitable distance measure and a matching method for effective results. Several distance metrics have been evaluated in the framework of nearest neighbour and nearest neighbour ratio matching methods using benchmark dataset. Evaluation is done by comparing matching and registration performance using metrics computed from ground truth. Keywords: Distance measures; deep learning; feature detection; feature descriptor; image matching

Image registration via low-rank factorization and maximum rank resolving

The feature-based image registration method has a better performance in terms of robustness to the intensity variance, but its accuracy of the feature-based image registration still could be improved. This paper utilizes the low-rank factorization and maximum rank resolving to improve the accuracy of image registration. In detail, the proposed method extracts coarse geometrical transform parameters based on the feature point pairs between images, then constructs low-rank model to optimize the geometrical transform parameters and estimate the inliers. Finally, an iterative optimization strategy is introduced to acquire the optimized transform parameters by maximum rank resolving. Experimental results illustrate that the proposed approach presents a good performance in terms with the root residual mean squares error and the entropy of image difference.

MEDICAL IMAGE REGISTRATION BASED ON IMPROVED FUZZY C-MEANS CLUSTERING

The mutual information (MI) technology and the iterative closest point (ICP) algorithm, as intensity-based and feature-based image registration methods respectively, are commonly put into use in medical image registration. But some naturally existing things which restrict the further development need to be faced and be solved. On one hand, they remain heavy calculation costs and low registration efficiencies. On the other hand, since they seriously depend on whether the initial rotation and translation registration parameters can be exactly selected, they often trap in the local optima and even fail to register images. In this paper, we compute the centroids of the reference and floating images by using the image moments to obtain the initial translation values, and use improved fuzzy C-means clustering (IFCM) to classify the image coordinates. Before clustering, this proposed method first centralizes the medical image coordinates, creates the two-row coordinate matrix to construct the two-dimensional (2D) sample set partitioned into two classes, and computes the slope of a straight line fitted to the two classes, finally derives the rotation angle from solving the arc tangent of the slope and obtains the initial rotation values. The experimental results show that, this proposed method has a fairly simple implementation, a low computational load, a fast registration and good registration accuracy. Also, it can efficiently avoid trapping in the local optima and meets both mono-modality and multi-modality image registrations.

Hierarchical Multi-modal Image Registration by Learning Common Feature Representations.

Mutual information (MI) has been widely used for registering images with different modalities. Since most inter-modality registration methods simply estimate deformations in a local scale, but optimizing MI from the entire image, the estimated deformations for certain structures could be dominated by the surrounding unrelated structures. Also, since there often exist multiple structures in each image, the intensity correlation between two images could be complex and highly nonlinear, which makes global MI unable to precisely guide local image deformation. To solve these issues, we propose a hierarchical inter-modality registration method by robust feature matching. Specifically, we first select a small set of key points at salient image locations to drive the entire image registration. Since the original image features computed from different modalities are often difficult for direct comparison, we propose to learn their common feature representations by projecting them from their native feature spaces to a common space, where the correlations between corresponding features are maximized. Due to the large heterogeneity between two high-dimension feature distributions, we employ Kernel CCA (Canonical Correlation Analysis) to reveal such non-linear feature mappings. Then, our registration method can take advantage of the learned common features to reliably establish correspondences for key points from different modality images by robust feature matching. As more and more key points take part in the registration, our hierarchical feature-based image registration method can efficiently estimate the deformation pathway between two inter-modality images in a global to local manner. We have applied our proposed registration method to prostate CT and MR images, as well as the infant MR brain images in the first year of life. Experimental results show that our method can achieve more accurate registration results, compared to other state-of-the-art image registration methods.