Image Registration Algorithm Research Articles

Semi-automatic 2D-to-3D video conversion based on background sprite generation

This paper presents a technique for semi-automatic 2D-to-3D stereo video conversion, which is known to provide user intervention in assigning foreground/background depths for key frames and then get depth maps for non-key frames via automatic depth propagation. Our algorithm treats foreground and background separately. For foregrounds, kernel pixels are identified and then used as the seeds for graph-cut segmentation for each non-key frame independently, resulting in results not limited by objects’ motion activity. For backgrounds, all video frames, after foregrounds being removed, are integrated into a common background sprite model (BSM) based on a relay-frame-based image registration algorithm. Users can then draw background depths for BSM in an integrated manner, thus reducing human efforts significantly. Experimental results show that our method is capable of retaining more faithful foreground depth boundaries (by 1.6–2.7 dB) and smoother background depths than prior works. This advantage is helpful for 3D display and 3D perception.

SRP‐AKAZE: an improved accelerated KAZE algorithm based on sparse random projection

The AKAZE algorithm is a typical image registration algorithm that has the advantage of high computational efficiency based on non‐linear diffusion. However, it is weaker than the scale‐invariant feature transformation (SIFT) algorithm in terms of robustness and stability. We propose a new and improved version of the AKAZE algorithm by using the SIFT descriptor based on sparse random projection (SRP). The proposed method not only retains the advantage of high efficiency of the AKAZE algorithm in feature detection but also has the stability of the SIFT descriptor. Moreover, the computational complexity due to the high dimension of the SIFT descriptor, which limits the speed of feature matching, is drastically reduced by the SRP strategy. Experiments on several benchmark image datasets demonstrate that the proposed algorithm can significantly improve the stability of the AKAZE algorithm, and the results suggest the better matching performance and robustness of the feature descriptor.

Algorithm Selection in Multimodal Medical Image Registration

Over the past few decades, fast<strong>-</strong>growth has occurred in the area of medical image acquisition devices, and physicians now rely on the utilization of medical images for the diagnosis, treatment plans, and surgical guidance. Researchers have classified medical images according to two structures: anatomical and functional structures. Due to this classification, the data obtained from two or more images of the same object frequently provide complementary and more abundant information through a process known as multimodal medical model registration. Image registration is spatially mapping the coordinate system of the two images obtained from a different viewpoint and utilizing various sensors. Several automatic multimodal medical image registration algorithms have been introduced based on types of medical images and their applications to increase the reliability, robustness, and accuracy. Due to the diversity in imaging and the different demands<strong> </strong>for applications, there is no single registration algorithm that can do that. This paper introduces a novel method for developing a multimodal medical image registration system that can select the most accepted registration algorithm from a group of registration algorithms for a variety of input datasets. The method described here is based on a machine learning technique that selects the most promising candidate. Several experiments have been conducted, and the results reveal that the novel approach leads to considerably faster reliability, accuracy, and more robustness registration algorithm selection.

ANHIR: Automatic Non-Rigid Histological Image Registration Challenge.

Automatic Non-rigid Histological Image Registration (ANHIR) challenge was organized to compare the performance of image registration algorithms on several kinds of microscopy histology images in a fair and independent manner. We have assembled 8 datasets, containing 355 images with 18 different stains, resulting in 481 image pairs to be registered. Registration accuracy was evaluated using manually placed landmarks. In total, 256 teams registered for the challenge, 10 submitted the results, and 6 participated in the workshop. Here, we present the results of 7 well-performing methods from the challenge together with 6 well-known existing methods. The best methods used coarse but robust initial alignment, followed by non-rigid registration, used multiresolution, and were carefully tuned for the data at hand. They outperformed off-the-shelf methods, mostly by being more robust. The best methods could successfully register over 98% of all landmarks and their mean landmark registration accuracy (TRE) was 0.44% of the image diagonal. The challenge remains open to submissions and all images are available for download.

SIFT Image Feature Extraction for an Efficient Image Registration

This paper presents an image registration algorithm based on SIFT (Scale Invariant Feature Transform).The obtained descriptors and key points by the SIFT confirms that, the algorithm is very robust to scaling, noise, translation and rotation. At the beginning, the key points are extracted from the image. Later to Match the obtained points, dot products between the unit vectors are calculated. Finally, transformation matrix is obtained by applying RANSAC algorithm. Experimental results shows that the algorithm extracts the better key points, which can be used for used for image registration applications.

An Overview on the Latest Nature-Inspired and Metaheuristics-Based Image Registration Algorithms

The development of automated image registration (IR) methods is a well-known issue within the computer vision (CV) field and it has been largely addressed from multiple viewpoints. IR has been applied to a high number of real-world scenarios ranging from remote sensing to medical imaging, artificial vision, and computer-aided design. In the last two decades, there has been an outstanding interest in the application of new optimization approaches for dealing with the main drawbacks present in the early IR methods, e.g., the Iterative Closest Point (ICP) algorithm. In particular, nature-inspired computation, e.g., evolutionary computation (EC), provides computational models that have their origin in evolution theories of nature. Moreover, other general purpose algorithms known as metaheuristics are also considered in this category of methods. Both nature-inspired and metaheuristic algorithms have been extensively adopted for tackling the IR problem, thus becoming a reliable alternative for optimization purposes. In this contribution, we aim to perform a comprehensive overview of the last decade (2009–2019) regarding the successful usage of this family of optimization approaches when facing the IR problem. Specifically, twenty-four methods (around 16 percent) of more than one hundred and fifty different contributions in the state-of-the-art have been selected. Several enhancements have been accordingly provided based on the promising outcomes shown by specific algorithmic designs. Finally, our research has shown that the field of nature-inspired and metaheuristic algorithms has increased its interest in the last decade to address the IR problem, and it has been highlighted that there is still room for improvement.

Structure tensor‐based SIFT algorithm for SAR image registration

The scale-invariant feature transform (SIFT) algorithm is the most widely used feature extraction as well as a feature matching method in remote sensing image registration. However, the performance of this algorithm is affected by the influence of speckle noise in synthetic aperture radar (SAR) images. It reduces the number of correct matches as well as the correct matching rate in SAR image registration. Moreover, SAR image registration is considered to be a challenging task as the images generally have significant geometric as well as intensity variations. To address these problems, a structure tensor-based SIFT algorithm is proposed to register the SAR images. At first, the tensor diffusion technique is used to construct the scale layers. Then, the features are extracted in the scale layers. Finally, feature matching is performed between the input SAR images and correct matches are identified. The proposed method can increase the number of correct matches as well as position accuracy in registration. Experiments have been conducted on five SAR image pairs to verify the effectiveness of the method.

Accelerating B-spline interpolation on GPUs: Application to medical image registration

Background and ObjectiveB-spline interpolation (BSI) is a popular technique in the context of medical imaging due to its adaptability and robustness in 3D object modeling. A field that utilizes BSI is Image Guided Surgery (IGS). IGS provides navigation using medical images, which can be segmented and reconstructed into 3D models, often through BSI. Image registration tasks also use BSI to transform medical imaging data collected before the surgery and intra-operative data collected during the surgery into a common coordinate space. However, such IGS tasks are computationally demanding, especially when applied to 3D medical images, due to the complexity and amount of data involved. Therefore, optimization of IGS algorithms is greatly desirable, for example, to perform image registration tasks intra-operatively and to enable real-time applications. A traditional CPU does not have sufficient computing power to achieve these goals and, thus, it is preferable to rely on GPUs. In this paper, we introduce a novel GPU implementation of BSI to accelerate the calculation of the deformation field in non-rigid image registration algorithms. MethodsOur BSI implementation on GPUs minimizes the data that needs to be moved between memory and processing cores during loading of the input grid, and leverages the large on-chip GPU register file for reuse of input values. Moreover, we re-formulate our method as trilinear interpolations to reduce computational complexity and increase accuracy. To provide pre-clinical validation of our method and demonstrate its benefits in medical applications, we integrate our improved BSI into a registration workflow for compensation of liver deformation (caused by pneumoperitoneum, i.e., inflation of the abdomen) and evaluate its performance. ResultsOur approach improves the performance of BSI by an average of 6.5× and interpolation accuracy by 2× compared to three state-of-the-art GPU implementations. Through pre-clinical validation, we demonstrate that our optimized interpolation accelerates a non-rigid image registration algorithm, which is based on the Free Form Deformation (FFD) method, by up to 34%. ConclusionOur study shows that we can achieve significant performance and accuracy gains with our novel parallelization scheme that makes effective use of the GPU resources. We show that our method improves the performance of real medical imaging registration applications used in practice today.

FPGA-accelerated adaptive projection-based image registration

In this paper, a high-speed hardware-based image registration is proposed exploiting parallelism and adaptive sampling technique to fulfill the requirement of high-speed portable multimedia devices. This technique computes radial and angular projections in parallel way without converting the image into polar domain, but by adjusting the number of samples along angular direction according to radius length which decreases computational load. Further, a complete image registration algorithm without using any geometric transformation is proposed. The software-based implementation of the proposed algorithm is 1.33 times faster than its latest available method in the literature. The proposed algorithm is mapped in field-programmable gate array (FPGA, Virtex6-xc6vlx760-2-ff1760) and it utilizes $$2.03\%$$ Slice LUTs, $$35.14\%$$ LUT-FF pair and $$1.27\%$$ DSP48E1s; and maximum clock frequency is 266 Mz. The hardware-based implementation of the proposed algorithm is $$10^4$$ times faster than software counterpart.

Data clustering to select clinically-relevant test cases for algorithm benchmarking and characterization

Algorithm benchmarking and characterization are an important part of algorithm development and validation prior to clinical implementation. However, benchmarking may be limited to a small collection of test cases due to the resource-intensive nature of establishing ‘ground-truth’ references. This study proposes a framework for selecting test cases to assess algorithm and workflow equivalence. Effective test case selection may minimize the number of ground-truth comparisons required to establish robust and clinically relevant benchmarking and characterization results.To demonstrate the proposed framework, we clustered differences between two independent workflows estimating during-treatment dose objective violations for 15 head and neck cancer patients (15 planning CTs, 105 on-unit CBCTs). Each workflow used a different deformable image registration algorithm to estimate inter-fractional anatomy and contour changes. The Hopkins statistic tested whether workflow output was inherently clustered and k-medoid clustering formalized cluster assignment. Further statistical analyses verified the relevance of clusters to algorithm output. Data at cluster centers (‘medoids’) were considered as candidate test cases representative of workflow-relevant algorithm differences.The framework indicated that differences in estimated dose objective violations were naturally grouped (Hopkins = 0.75, providing 90% confidence). K-medoid clustering identified five clusters which stratified workflow differences (MANOVA: p < 0.001) in estimated parotid gland D50%, spinal cord/brainstem Dmax, and high dose CTV coverage dose violations (Kendall’s tau: p < 0.05). Systematic algorithm differences resulting in workflow discrepancies were: parotid gland volumes (ANOVA: p < 0.001), external contour deformations (t-test: p = 0.022), and CTV-to-PTV margins (t-test: 0.009), respectively. Five candidate test cases were verified as representative of the five clusters.The framework successfully clustered workflow outputs and identified five test cases representative of clinically relevant algorithm discrepancies. This approach may improve the allocation of resources during the benchmarking and characterization process and the applicability of results to clinical data.

Iterative Phase Correlation Algorithm for High-precision Subpixel Image Registration

Abstract Many astrophysical observations and measurement techniques that rely on data from images include an image registration step. The results of such techniques thus heavily rely on the precision of the registration. We present an Iterative Phase Correlation (IPC) algorithm, which is an extension of the well-known phase correlation method of image registration and is ideally suited for problems, where the subpixel registration accuracy plays a crucial role. Furthermore, a sophisticated and reliable method of optimal IPC parameter estimation is described. The paper includes examples of such optimized parameters for Solar Dynamics Observatory (SDO)/Helioseismic and Magnetic Imager, SDO/Atmospheric Imaging Assembly, and Solar Terrestrial Relations Observatory A/B Sun Earth Connection Coronal and Heliospheric Investigation images. The new method (both with or without the parameter optimization step) significantly outperforms standard image registration methods, such as (non-iterative) phase correlation or (normalized) cross correlation in the sense of subpixel accuracy. A step-by-step pseudocode implementation is also included.

Angular compounding for speckle reduction in optical coherence tomography using geometric image registration algorithm and digital focusing

Optical coherence tomography (OCT) suffers from speckle noise due to the high spatial coherence of the utilized light source, leading to significant reductions in image quality and diagnostic capabilities. In the past, angular compounding techniques have been applied to suppress speckle noise. However, existing image registration methods usually guarantee pure angular compounding only within a relatively small field of view in the focal region, but produce spatial averaging in the other regions, resulting in resolution loss and image blur. This work develops an image registration model to correctly localize the real-space location of every pixel in an OCT image, for all depths. The registered images captured at different angles are fused into a speckle-reduced composite image. Digital focusing, based on the convolution of the complex OCT images and the conjugate of the point spread function (PSF), is studied to further enhance lateral resolution and contrast. As demonstrated by experiments, angular compounding with our improved image registration techniques and digital focusing, can effectively suppress speckle noise, enhance resolution and contrast, and reveal fine structures in ex-vivo imaged tissue.

Validation of Deformable Image Registration by Using a B-spline and Optical-Flow Algorithm in Head and Neck Cancer Cases

To verify the usefulness of the recalculated dose (reDose) with deformed image (dCT) and contour structures (dST), we obtained the reDose in the treatment planning system by using dCT and dST, and analyzed the accuracy of both the deformed dose (dDose) acquired by using commercialized deformable image registration (DIR) algorithms and the reDose by comparing it with replan dose (rDose). Because dCT and dST may affect the reDose calculation for the treatment planning system, an evaluation of image and organ contour accuracy was also performed. The results of a similarity evaluation of dCT and dST showed that the conformity index (CI) was below 0.54 for the parotid and the gross target volume (GTV) with the B-spline algorithm while the optical-flow algorithm showed 0.55 or more except for the GTV. The mean distance to conformity (MDC) showed 3 mm or more for all organs, except for the spinal cord. The cross correlation (CC) was more than 0.80 for the optical-flow algorithm; however, B-spline algorithm showed a value below than 0.79. For the dose analysis, dDose and reDose revealed that the maximum dose and the minimum dose were more than 2 Gy on the parotid and trachea, respectively. The reDose was more similar to the rDose than to the dDose. According to the results, we found the usefulness of the reDose with the dCT and dST by re-calculation, but inferring the dose by just using dDose can lead to an incorrect treatment plan. The commercialized DIR algorithms showed that the deformation of the CT image, the contour structure, and the dose distribution requires verification in several ways before use in real clinical treatment. The proposed assessment method for dDose accuracy is expected to be applied to the head and neck as well as many other organs. However, a contour structure with a large change in volume should be re-delineated and a new treatment plan established.

Computer Three-Dimensional Positioning and Reconstruction of Continuous Slice Images of Biological Tissues

The physiological and pathological changes of organisms will have certain effects on the morphology of tissues and organs. Conversely, the morphological changes of tissues and organs can also reflect the physiological and pathological changes of organisms to some extent. The bio-tissue slice can provide people with two-dimensional information of the tissue structure on a certain section. In this paper, the image registration algorithm based on mutual information is used to register the slice images of different tissues, and then the image segmentation is used to improve the registration accuracy. The three-dimensional data of the organizational structure is obtained, and the three-dimensional image of the tissue is reconstructed by computer processing. The reconstruction of different tissue slices is used to prove that the algorithm is fast and accurate, and the reconstruction speed of the 3D model is improved, which is of great significance for shortening the diagnostic waiting time.

Recaptured Screen Image Demoiréing

In many situations, such as transferring data between devices and recording precious moments, we would like to capture the contents on screens using digital cameras for convenience. These recaptured screen images and videos suffer from a special type of degradation called “moiré pattern”, which is caused by the aliasing between the grid of display screen and the array of camera sensor. However, few works are proposed to tackle this problem. Considering the great success of convolutional neural networks (CNNs) in image restoration, we propose a CNN-based moiré removal method for recaptured screen images. There are mainly two contributions in this paper. First, for the generation of training data, we propose an image registration algorithm via global homography transform and local patch matching to compensate the significant viewpoint disparity between the recaptured screen image and the moiré-free image obtained via screenshot. We construct a moiré removal and brightness improvement (MRBI) database with aligned moiré-free and moiré images. Second, we propose a convolutional neural Network with Additive and Multiplicative modules (termed as AMNet) to transfer the low light moiré image to the bright moiré-free image. The proposed network is trained with pixel-wise loss, perceptual loss, and adversarial loss. Extensive experiments on 340 test images demonstrate that the proposed method outperforms state-of-the-art moiré removal methods.

Resolution enhancement of transmission electron microscopy by super-resolution radial fluctuations

Super-resolution fluorescence microscopy techniques have enabled dramatic development in modern biology due to their capability to discern features smaller than the diffraction limit of light. Recently, super-resolution radial fluctuations (SRRF), an analytical approach that is capable of generating super-resolution images easily without the need for specialized hardware or photoswitchable fluorophores, has been presented. While SRRF has only been demonstrated on fluorescence microscopes, in principle, this method can be used to generate super-resolution images on any imaging platforms with intrinsic radial symmetric point spread functions. In this work, we show that SRRF can be utilized to enhance the resolution and quality of transmission electron microscopy (TEM) images. By including an image registration algorithm to correct for sample drift, the SRRF-TEM approach substantially enhances the resolution of TEM images of three different samples acquired with a commercial TEM system. We quantify the resolution improvement in SRRF-TEM and evaluate how SRRF parameters affect the resolution and quality of SRRF-TEM results.

Paper currency defect detection algorithm using quaternion uniform strength

In this paper, we propose a novel paper currency defect detection algorithm using quaternion uniform strength. We first build paper currency image preprocessing integration framework which includes intensity balancing, paper currency location, and geometric correction. We then propose a global–local paper currency image registration algorithm by moving key areas within certain range which can eliminate the false difference effectively. Finally, the quaternion uniform strength is calculated by using quaternion convolution edge detector. The defect degree of paper currency is determined by using the quaternion uniform color difference. The proposed algorithm is tested using different datasets from five countries: CNY, USD, EUR, VND, and RUB. The experimental results demonstrate that the proposed algorithm yields better results than the existing state-of-the-art paper currency defect detection techniques. The demo of the proposed paper currency defect detection algorithm will be publicly available.

Image Registration Algorithm Based on Convolutional Neural Network and Local Homography Transformation

In order to overcome the poor robustness of traditional image registration algorithms in illuminating and solving the problem of low accuracy of a learning-based image homography matrix estimation algorithm, an image registration algorithm based on convolutional neural network (CNN) and local homography transformation is proposed. Firstly, to ensure the diversity of samples, a sample and label generation method based on moving direct linear transformation (MDLT) is designed. The generated samples and labels can effectively reflect the local characteristics of images and are suitable for training the CNN model with which multiple pairs of local matching points between two images to be registered can be calculated. Then, the local homography matrices between the two images are estimated by using the MDLT and finally the image registration can be realized. The experimental results show that the proposed image registration algorithm achieves higher accuracy than other commonly used algorithms such as the SIFT, ORB, ECC, and APAP algorithms, as well as another two learning-based algorithms, and it has good robustness for different types of illumination imaging.

Research on Image Registration Algorithm and Its Application in Photovoltaic Images

Due to high similarity interference of photovoltaic images, repetitive EL image detection and photovoltaic image stitching pose a great challenge. This article proposes two application algorithms and a registration method, which are repetitive EL image detection and photovoltaic image stitching based on the proposed AKAZE-LATCH+GMS registration method. Considering the existence of duplicate EL images acquired indoors, a repetitive EL image detection based on global feature consistency and local cell's texture similarity is proposed. The AKAZE-LATCH+GMS is used to calculate global consistent feature ratio and dHash is used to quantify local cell's similarity to determine whether it is repetitive or not. To solve the problem of high similarity of outdoor photovoltaic images, which leads to dislocation of stitching, an image stitching algorithm based on geographic information is proposed in this article. Compared with AKAZE+GMS and ORB+GMS, the registration experimental results show that the AKAZE-LATCH+GMS method can obtain more initial matching features and higher correct matching rate. Indoor EL images with geometric and illumination differences are used to verify the proposed detection method, which can effectively eliminate repetitive EL images. Aerial photovoltaic images, acquired from the same or different view, are used to verify the validity of the stitching method, and the photovoltaic images can be stitched correctly compared with the stitching method without geographic information.

Fast quantitative time lapse displacement imaging of endothelial cell invasion.

In order to unravel rapid mechano-chemical feedback mechanisms in sprouting angiogenesis, we combine selective plane illumination microscopy (SPIM) and tailored image registration algorithms — further referred to as SPIM-based displacement microscopy — with an in vitro model of angiogenesis. SPIM successfully tackles the problem of imaging large volumes while upholding the spatial resolution required for the analysis of matrix displacements at a subcellular level. Applied to in vitro angiogenic sprouts, this unique methodological combination relates subcellular activity — minute to second time scale growing and retracting of protrusions — of a multicellular systems to the surrounding matrix deformations with an exceptional temporal resolution of 1 minute for a stack with multiple sprouts simultaneously or every 4 seconds for a single sprout, which is 20 times faster than with a conventional confocal setup. Our study reveals collective but non-synchronised, non-continuous activity of adjacent sprouting cells along with correlations between matrix deformations and protrusion dynamics.