Feature-based Image Registration Research Articles

Enhancing aerial image registration: outlier filtering through feature classification

In the context of feature-based image registration, the crucial task of outlier removal plays a pivotal role in achieving precise registration accuracy. This research introduces an innovative binary classifier founded on an adaptive approach for effectively identifying and eliminating outliers. The methodology begins with the utilization of the scale invariant feature transform (SIFT) to extract features from two images, initially matched using the Euclidian distance metrics. Subsequently, a classification procedure is executed to segregate the feature points into two categories: genuine matches (inliers) and spurious matches (outliers), which is accomplished through the brute-force matcher (BFM) technique. To enhance this process further, a novel classifier rooted in the random forest algorithm is introduced. This classifier is trained and tested using a comprehensive dataset curated for this study. The newly proposed classifier plays a pivotal role in attenuating the influence of outliers, ultimately leading to refined image registration process characterized by enhanced accuracy. The effectiveness of this outlier removal approach is assessed through a meticulous analysis of positional and classification accuracy. Additionally, we offer comparative insights by evaluating the performance of selected algorithm on our dataset.

Comparative Analysis of Color Space and Channel, Detector, and Descriptor for Feature-Based Image Registration.

The current study aimed to quantify the value of color spaces and channels as a potential superior replacement for standard grayscale images, as well as the relative performance of open-source detectors and descriptors for general feature-based image registration purposes, based on a large benchmark dataset. The public dataset UDIS-D, with 1106 diverse image pairs, was selected. In total, 21 color spaces or channels including RGB, XYZ, Y'CrCb, HLS, L*a*b* and their corresponding channels in addition to grayscale, nine feature detectors including AKAZE, BRISK, CSE, FAST, HL, KAZE, ORB, SIFT, and TBMR, and 11 feature descriptors including AKAZE, BB, BRIEF, BRISK, DAISY, FREAK, KAZE, LATCH, ORB, SIFT, and VGG were evaluated according to reprojection error (RE), root mean square error (RMSE), structural similarity index measure (SSIM), registration failure rate, and feature number, based on 1,950,984 image registrations. No meaningful benefits from color space or channel were observed, although XYZ, RGB color space and L* color channel were able to outperform grayscale by a very minor margin. Per the dataset, the best-performing color space or channel, detector, and descriptor were XYZ/RGB, SIFT/FAST, and AKAZE. The most robust color space or channel, detector, and descriptor were L*a*b*, TBMR, and VGG. The color channel, detector, and descriptor with the most initial detector features and final homography features were Z/L*, FAST, and KAZE. In terms of the best overall unfailing combinations, XYZ/RGB+SIFT/FAST+VGG/SIFT seemed to provide the highest image registration quality, while Z+FAST+VGG provided the most image features.

A Rigid Image Registration by Combined Local Features and Genetic Algorithms

Abstract Image registration is an essential pre-processing step required for many image processing applications such as medical imaging and computer vision. The aim is to geometrically align two or more images of the same scene by establishing a mapping that relies on each point from one image to its corresponding point of another image. Scale invariant feature transform (SIFT) and speeded up robust features (SURF) are well-liked local features descriptors that have been extensively utilised for feature-based image registration due to their inherent properties such as invariance, changes in illumination, and noise. Moreover, the task of registration can be viewed as an optimization problem that can be solved by applying genetic algorithms (GAs). This paper presents an efficient feature image registration method based on combined local features and GAs. Firstly, the procedure consists of extracting the local features from the images by combining SIFT and SURF algorithms and matching them to refine the feature set data. Therefore, an adaptive GA based on fitness sharing and elitism techniques is employed to find the optimal rigid transformation parameters that best align the feature points by minimizing a distance metric. The suggested method is applied for registering medical images and the obtained results are significant compared to other feature-based approaches with reasonable computation time.

Feature Based Image Registration Using ORB and CNN for Remote Sensing Images

Feature Based Image Registration Using ORB and CNN for Remote Sensing Images

Feature based image registration using CNN features for satellite images having varying illumination level

Feature based image registration using CNN features for satellite images having varying illumination level

A study of multi-target image-based displacement measurement approach for field testing of bridges

ABSTRACT As the demand for field testing of bridges grows, so does the need to optimize field testing procedures to allow for a simplified testing strategy that can be employed more effectively. With the advent of computer vision, there has been limited research exploring Feature-Based Image Registration (FBIR) methods for structural testing of bridges. In this paper, the potential of a simple FBIR approach to accurately capture submillimeter displacements using consumer-grade cameras will be demonstrated through a field test on a reinforced concrete slab bridge and on a full-scale bridge deck specimen in the laboratory. The internal and external parameters that influence the results of this measurement strategy were investigated by using various camera positions during the laboratory tests and applying different threshold parameters to the Speeded-Up Robust Features algorithm used for the feature detection and matching. The FBIR method demonstrates great potential, producing an average measurement accuracy within 1.6% of conventional displacement sensors during the field test and 3.3% during the laboratory tests. Altogether, the advantages to this image-based measurement approach enhance the load testing strategy to be implemented by bridge owners at much lower costs and with minimal complication and field setup.

An Optimized SIFT-OCT Algorithm for Stitching Aerial Images of a Loblolly Pine Plantation

When producing orthomosaic from aerial images of a forested area, challenges arise when the forest canopy is closed, and tie points are hard to find between images. The recent development in deep leaning has shed some light in tackling this problem with an algorithm that examines each image pixel-by-pixel. The scale-invariant feature transform (SIFT) algorithm and its many variants are widely used in feature-based image stitching, which is ideal for orthomosaic production. However, although feature-based image registration can find many feature points in forest image stitching, the similarity between images is too high, resulting in a low correct matching rate and long splicing time. To counter this problem by considering the characteristics of forest images, the inverse cosine function ratio of the unit vector dot product (arccos) is introduced into the SIFT-OCT (SIFT skipping the first scale-space octave) algorithm to overcome the shortfalls of too long a matching time caused by too many feature points for matching. Then, the fast sample consensus (FSC) algorithm was introduced to realize the deletion of mismatched point pairs and improve the matching accuracy. This optimized method was tested on three sets of forest images, representing the forest core, edge, and road areas of a loblolly pine plantation. The same process was repeated by using the regular SIFT and SIFT-OCT algorithms for comparison. The results showed the optimized SIFT-OCT algorithm not only greatly reduced the splicing time, but also increased the correct matching rate.

Registration of polarimetric images for in vivo skin diagnostics.

.Significance: Mueller matrix (MM) polarimetry is a promising tool for the detection of skin cancer. Polarimetric in vivo measurements often suffer from misalignment of the polarimetric images due to motion, which can lead to false results.Aim: We aim to provide an easy-to-implement polarimetric image data registration method to ensure proper image alignment.Approach: A feature-based image registration is implemented for an MM polarimeter for phantom and in vivo human skin measurements.Results: We show that the keypoint-based registration of polarimetric images is necessary for in vivo skin polarimetry to ensure reliable results. Further, we deliver an efficient semiautomated method for the registration of polarimetric images.Conclusions: Image registration for in vivo polarimetry of human skin is required for improved diagnostics and can be efficiently enhanced with a keypoint-based approach.

Feature Matching for Remote-Sensing Image Registration via Neighborhood Topological and Affine Consistency

Feature matching is a key method of feature-based image registration, which refers to establishing reliable correspondence between feature points extracted from two images. In order to eliminate false matchings from the initial matchings, we propose a simple and efficient method. The key principle of our method is to maintain the topological and affine transformation consistency among the neighborhood matches. We formulate this problem as a mathematical model and derive a closed solution with linear time and space complexity. More specifically, our method can remove mismatches from thousands of hypothetical correspondences within a few milliseconds. We conduct qualitative and quantitative experiments on our method on different types of remote-sensing datasets. The experimental results show that our method is general, and it can deal with all kinds of remote-sensing image pairs, whether rigid or non-rigid image deformation or image pairs with various shadow, projection distortion, noise, and geometric distortion. Furthermore, it is two orders of magnitude faster and more accurate than state-of-the-art methods and can be used for real-time applications.

Image registration method using representative feature detection and iterative coherent spatial mapping for infrared medical images with flat regions

In the registration of medical images, nonrigid registration targets, images with large displacement caused by different postures of the human body, and frequent variations in image intensity due to physiological phenomena are substantial problems that make medical images less suitable for intensity-based image registration modes. These problems also greatly increase the difficulty and complexity of feature detection and matching for feature-based image registration modes. This research introduces an automatic image registration algorithm for infrared medical images that offers the following benefits: effective detection of feature points in flat regions (cold patterns) that appear due to changes in the human body’s thermal patterns, improved mismatch removal through coherent spatial mapping for improved feature point matching, and large-displacement optical flow for optimal transformation. This method was compared with various classical gold standard image registration methods to evaluate its performance. The models were compared for the three key steps of the registration process—feature detection, feature point matching, and image transformation—and the results are presented visually and quantitatively. The results demonstrate that the proposed method outperforms existing methods in all tasks, including in terms of the features detected, uniformity of feature points, matching accuracy, and control point sparsity, and achieves optimal image transformation. The performance of the proposed method with four common image types was also evaluated, and the results verify that the proposed method has a high degree of stability and can effectively register medical images under a variety of conditions.

Robust Registration of Multispectral Satellite Images Based on Structural and Geometrical Similarity

Accurate registration of multispectral satellite images is a challenging task due to the significant and nonlinear radiometric differences between these data. To address this problem, this letter explores the strategy of geometrical similarity between triplets of feature points, and it is combined with the structural similarity between images in a feature-based image registration framework. The underlying principle is that the structural and geometrical similarities generally preserve across the images being registered. In this feature-based image registration framework, a set of control points (CPs) are first detected. Then, the geometric similarity between triplets of CPs is defined, followed by a ranking operation of these triplets of CPs. The highly ranked triplets are used to estimate a spatial transformation between images. Finally, initial matches obtained by a benchmark registration technique are refined by the estimated transformation. The experimental results demonstrate the great effectiveness of the proposed technique for registering multispectral satellite images.

DRFD-Net: Using Dual Receptive Field Descriptors for Multitemporal Optical Remote Sensing Image Registration

Multitemporal optical remote sensing image registration is still a challenging problem for current feature-based image registration algorithms due to the complex nonlinear discrepancies arising from diverse factors, including illumination, weather, and surface condition changes. To address the issue, this article attempts to combine the dual receptive field descriptors (DRFDs) constructed by a novel deep convolutional network. In addition, a novel inner loss function (ILF) that imposes constraints on the intermediate descriptors is adopted in order to consolidate the distinguishability of the descriptors when the overlapping areas of the input image patches are large. Subsequently, the dual feature distance maps (DFDMs) are built on the basis of the DRFDs and combined with features from accelerated segment test (FAST) key points for efficient and accurate correspondence establishment across the source image and the target image. Eventually, an iterative algorithm is proposed to remove the possible outliers. Experiments show that the combination of DRFDs trained with the ILF performs better than current learnable local descriptors, such as L2-Net, HardNet, and SOSNet. The image registration results using our method are more accurate than the methods based on learnable descriptors, such as L2-Net, HardNet, and SOSNet, and handcrafted descriptors, such as scale-invariant feature transform (SIFT), SURF, and ORB.

An Image Registration Method Using Deep Residual Network Features for Multisource High-Resolution Remote Sensing Images

Accurate registration for multisource high-resolution remote sensing images is an essential step for various remote sensing applications. Due to the complexity of the feature and texture information of high-resolution remote sensing images, especially for images covering earthquake disasters, feature-based image registration methods need a more helpful feature descriptor to improve the accuracy. However, traditional image registration methods that only use local features at low levels have difficulty representing the features of the matching points. To improve the accuracy of matching features for multisource high-resolution remote sensing images, an image registration method based on a deep residual network (ResNet) and scale-invariant feature transform (SIFT) was proposed. It used the fusion of SIFT features and ResNet features on the basis of the traditional algorithm to achieve image registration. The proposed method consists of two parts: model construction and training and image registration using a combination of SIFT and ResNet34 features. First, a registration sample set constructed from high-resolution satellite remote sensing images was used to fine-tune the network to obtain the ResNet model. Then, for the image to be registered, the Shi_Tomas algorithm and the combination of SIFT and ResNet features were used for feature extraction to complete the image registration. Considering the difference in image sizes and scenes, five pairs of images were used to conduct experiments to verify the effectiveness of the method in different practical applications. The experimental results showed that the proposed method can achieve higher accuracies and more tie points than traditional feature-based methods.

A robust edge detection algorithm based on feature-based image registration (FBIR) using improved canny with fuzzy logic (ICWFL)

A robust edge detection algorithm based on feature-based image registration (FBIR) using improved canny with fuzzy logic (ICWFL)

Feature Matching Based on Minimum Relative Motion Entropy for Image Registration

Accurate point matching is widely used, and it is a critical and challenging process in feature-based image registration. To improve feature matching accuracy on putative matches with heavy outliers and similar local structures, an accurate and robust feature point matching algorithm based on minimum relative motion entropy (MRME) is proposed, in which the relative motion between the putative matches and their K-nearest neighbors is formulated. Based on the relative motion clustering result, the relative motion entropy is defined to find the coincident relative motions. According to relative motions with MRME, the outliers are removed in a two-stage feature match strategy. With quasi-linear time complexity, outliers with random or irregular relative motion are removed efficiently and accurately, while inliers with coincident relative motion are retained. Three data sets with repetitive patterns, viewpoint changes, low overlapping areas, and local deformations are used to demonstrate the performance of the proposed algorithm. MRME is shown to be more robust and accurate than ten state-of-the-art feature matching algorithms.

Non-rigid image registration using a modified fuzzy feature-based inference system for 3D cardiac motion estimation

Background and objectiveNon-rigid image registration is a well-established method for estimating cardiac motion on 3D echocardiographic images. However, such images have relatively poor spatio-temporal resolution making registration challenging. Some of the main challenges are extracting features relevant to the registration problem and defining a suitable geometrical transformation to be applied. The latter can be tackled using a fuzzy inference system considering its potential in transformation modeling. From this point of view, feature-based image registration can be considered an identification problem in which the transformation parameters are computed through an optimization process. This study, thus, aims to estimate cardiac motion on 3D echocardiographic images based on feature-based non-rigid image registration through sets of modified fuzzy rules. MethodsThe 3D volume features were extracted with the popular scale-invariant feature transform (SIFT) descriptors in 3D space. Sets of fuzzy rules were generated according to the extracted features to register every two consecutive frames. Finally, some supplementary rules modified the registration rule for estimating cardiac motion. ResultsApplying the fuzzy feature-based inference system on the STRAUS synthetic database showed the proposed method to be competitive with other well-established registration algorithms in terms of tracking error and accuracy of strain estimates. The proposed algorithm yielded a tracking error of 1 mm and a relative circumferential strain error of 0.82±4.69%. In addition, the potential of the proposed algorithm for clinical applications was confirmed by evaluating its performance on an in-vivo database called CETUS. ConclusionThis paper proposes a novel registration method based on fuzzy logic which was shown to enable tracking complex cardiac deformations in 3D echocardiographic images with high accuracy.

Robust Local Structure Visualization for Remote Sensing Image Registration

Image registration is a fundamental and important task in remote sensing. In this article, we focus on feature-based image registration. Existing attempts often require estimating a transformation model or imposing relaxed geometric constraints to establish reliable feature correspondences. However, a parametric model cannot handle image pairs undergoing complex transformations, and relaxed methods discard a lot of structure information and the results are often coarse. To solve the above issues, we propose a local structure visualization descriptor to preserve the original structure information, and cast the feature matching task into an evaluation of the consensus of visual structure under a convolutional neural network. This strategy can effectively measure the similarity of neighborhood structure for mismatch removal. In summary, our method does not depend on a specific transformation model and can process arbitrary remote sensing images (e.g., different deformations, severe outliers, various rotations, and scaling changes). To demonstrate the robustness of our strategy for image registration, extensive experiments on various real remote sensing images for feature matching are conducted and compared against nine state-of-the-art methods, where our method gives the best performances in most scenarios.

A Novel Keypoint Detector Combining Corners and Blobs for Remote Sensing Image Registration

Keypoint detection is a crucial step for feature-based image registration. The traditional detectors only extract one type of keypoint such as a corner or a blob, which is not quite beneficial to image registration. Accordingly, this letter presents a novel keypoint detector that aims to simultaneously extract corners and blobs. The proposed detector is named as Harris-Difference of Gaussian (DoG), which combines the advantages of the Harris-Laplace corner detector and the DoG blob detector. In the definition of Harris-DoG, we first build an image scale space and extract the corners by using the multiscale Harris detector. Then, these corners are screened by an automatic scale selection technique based on their DoG responses. This can make the corners robust to scale changes. Meanwhile, DoG also is used to detect the blobs in the image scale space by a nonmaxima suppression scheme. Finally, the scale invariant feature transform (SIFT) descriptors are computed for the detected corners and blobs, and they are applied together for image registration. The proposed Harris-DoG has been tested by using three pairs of multisensor remote sensing images. The experimental results show that Harris-DoG can effectively increase the number of correct matches and improve the registration accuracy compared with the state-of-the-art keypoint detectors.

INVESTIGATION OF IMAGE STITCHING REFINEMENT WITH ENHANCED CORRELATION COEFFICIENT

Image stitching combines multiple overlapping scenes into a panorama. It can be applied in video stitching, super-resolution, and medical imaging. Post-processing routines such as exposure compensation, seam line adjustment, and blending are often performed to increase its visual appeal. Our aim is to increase the registration accuracy between adjacent image pairs in a stitching algorithm. This is done by adding an area-based registration step, namely Enhanced Correlation Coefficient (ECC), which refines the original feature-based image registration. The performance of registrations in the stitching algorithm is evaluated using root-mean-squared error of control points, structural similarity index, and universal image quality index. The Boat and Graffiti datasets from the Oxford Robotics Database are used for experiments. It is found that ECC largely improves registrations in stitching except for a slight increase in root-mean-squared error of control points in the Boat dataset. In addition, ECC’s enhancement makes the registration outperform the ground truth in one image of both datasets.

Integration of optimal spatial distributed tie-points in RANSAC-based image registration

ABSTRACT Feature-based image registration requires the identification of correct tie-points between the image pair. In this paper, an improved outlier method is proposed to find correct matching results of optimal distribution based on RANSAC (RANdom SAmple Consensus) algorithm. The main feature of the proposed method is that an optimal spatial designation of tie-points method using stratified random selection (SRS), is integrated into RANSAC framework to filter out the mismatched features that exist in the massive initial matches generated by SIFT operator in order to estimate mapping function accurately. In this way, the selection of relatively disperse and evenly distributed tie-points based on adaptive stratified partition can make RANSAC efficient. We carried out experiments on the registration of three pairs of satellite images. The proposed SIFT-SRS-RANSAC method leads to higher matching and registration accuracy when comparing with the performance of SIFT-RANSAC and SIFT-bucketing-RANSAC algorithms.