Efficient Image Registration Research Articles

UTR: A UNet-like transformer for efficient unsupervised medical image registration

The existing medical image registration algorithms have the problem of low registration accuracy when processing large deformation medical images. In order to improve registration performance and utilize the global context extraction ability of Transformers without causing high computational complexity, a UNet-like Transformer model combining CNN and Transformer was constructed for 3D medical image registration tasks. We use the Efficient Global Local Attention (EGLA) mechanism to construct a Transformer encoder to further address the difficulty of modeling long-distance dependencies in existing medical image registration networks. We leverage the local modeling capabilities of CNN and the long-distance information capture capabilities of Transformer to achieve high-precision registration. The algorithm has undergone detailed validation experiments on two public datasets. The qualitative and quantitative registration results validate the effectiveness of the proposed model.

High-precision point cloud registration method based on volume image correlation

With the rapid development of binocular reconstruction, fringe projection profilometry, and time of flight, 3D imaging technology has been widely applied in the field of 3D measurement. However, due to limited measurement range and self-occlusion, point cloud registration methods are often used to obtain larger or more complete 3D contours. Although many scholars have proposed various point cloud registration methods, the accuracy and efficiency of point cloud registration still need to be further improved, especially for point clouds with different density or non-rigid transformation. Image registration technology based on image correlation has been developed for many years and has achieved great success in fields such as computer vision, photomechanics, and photogrammetry. Therefore, a simple and direct idea in this paper is to transform the point cloud registration problem into volume image correlation problem. By this, an efficient image registration method based on fast Fourier transform and an inverse compositional Gaussian Newton optimization method that only needs to calculate the Hessian matrix once can be introduced into the point cloud registration field, which can greatly improve the speed and accuracy of point cloud registration. Comparative experiments have shown that our method has doubled the accuracy and efficiency compared to the iterative closest point (ICP) method, and its practicality has also been verified in impeller reconstruction experiments.

A Rigid Image Registration by Combined Local Features and Genetic Algorithms

Abstract Image registration is an essential pre-processing step required for many image processing applications such as medical imaging and computer vision. The aim is to geometrically align two or more images of the same scene by establishing a mapping that relies on each point from one image to its corresponding point of another image. Scale invariant feature transform (SIFT) and speeded up robust features (SURF) are well-liked local features descriptors that have been extensively utilised for feature-based image registration due to their inherent properties such as invariance, changes in illumination, and noise. Moreover, the task of registration can be viewed as an optimization problem that can be solved by applying genetic algorithms (GAs). This paper presents an efficient feature image registration method based on combined local features and GAs. Firstly, the procedure consists of extracting the local features from the images by combining SIFT and SURF algorithms and matching them to refine the feature set data. Therefore, an adaptive GA based on fitness sharing and elitism techniques is employed to find the optimal rigid transformation parameters that best align the feature points by minimizing a distance metric. The suggested method is applied for registering medical images and the obtained results are significant compared to other feature-based approaches with reasonable computation time.

R2Net: Efficient and flexible diffeomorphic image registration using Lipschitz continuous residual networks.

Classical diffeomorphic image registration methods, while being accurate, face the challenges of high computational costs. Deep learning based approaches provide a fast alternative to address these issues; however, most existing deep solutions either lose the good property of diffeomorphism or have limited flexibility to capture large deformations, under the assumption that deformations are driven by stationary velocity fields (SVFs). Also, the adopted squaring and scaling technique for integrating SVFs is time- and memory-consuming, hindering deep methods from handling large image volumes. In this paper, we present an unsupervised diffeomorphic image registration framework, which uses deep residual networks (ResNets) as numerical approximations of the underlying continuous diffeomorphic setting governed by ordinary differential equations, which is parameterized by either SVFs or time-varying (non-stationary) velocity fields. This flexible parameterization in our Residual Registration Network (R2Net) not only provides the model's ability to capture large deformation but also reduces the time and memory cost when integrating velocity fields for deformation generation. Also, we introduce a Lipschitz continuity constraint into the ResNet block to help achieve diffeomorphic deformations. To enhance the ability of our model for handling images with large volume sizes, we employ a hierarchical extension with a multi-phase learning strategy to solve the image registration task in a coarse-to-fine fashion. We demonstrate our models on four 3D image registration tasks with a wide range of anatomies, including brain MRIs, cine cardiac MRIs, and lung CT scans. Compared to classical methods SyN and diffeomorphic VoxelMorph, our models achieve comparable or better registration accuracy with much smoother deformations. Our source code is available online at https://github.com/ankitajoshi15/R2Net.

Hephaestus : Codesigning and Automating 3D Image Registration on Reconfigurable Architectures

Healthcare is a pivotal research field, and medical imaging is crucial in many applications. Therefore finding new architectural and algorithmic solutions would benefit highly repetitive image processing procedures. One of the most complex tasks in this sense is image registration, which finds the optimal geometric alignment among 3D image stacks and is widely employed in healthcare and robotics. Given the high computational demand of such a procedure, hardware accelerators are promising real-time and energy-efficient solutions, but they are complex to design and integrate within software pipelines. Therefore, this work presents an automation framework called Hephaestus that generates efficient 3D image registration pipelines combined with reconfigurable accelerators. Moreover, to alleviate the burden from the software, we codesign software-programmable accelerators that can adapt at run-time to the image volume dimensions. Hephaestus features a cross-platform abstraction layer that enables transparently high-performance and embedded systems deployment. However, given the computational complexity of 3D image registration, the embedded devices become a relevant and complex setting being constrained in memory; thus, they require further attention and tailoring of the accelerators and registration application to reach satisfactory results. Therefore, with Hephaestus , we also propose an approximation mechanism that enables such devices to perform the 3D image registration and even achieve, in some cases, the accuracy of the high-performance ones. Overall, Hephaestus demonstrates 1.85× of maximum speedup, 2.35× of efficiency improvement with respect to the State of the Art, a maximum speedup of 2.51× and 2.76× efficiency improvements against our software, while attaining state-of-the-art accuracy on 3D registrations.

Efficient and robust image registration for two-dimensional micro-X-ray fluorescence measurements

Solving image registration problem in micro-X-ray uorescence spectrometry (µ-XRF) is a first step towards application of multi-image super-resolution technique.

Cross-Resolution Distillation for Efficient 3D Medical Image Registration

Images captured in clinic such as MRI scans are usually in 3D formats with high spatial resolutions. Existing learning-based models for medical image registration consume large GPU memories and long inference time, which is difficult to be deployed in resource-limited diagnosis scenarios. To address this problem, instead of shrinking the model size as in previous works, we turn to reducing the input resolution of existing registration models and boosting their performance through knowledge distillation. Specifically, we propose a cross-resolution distillation (CRD) scheme, which is designed to train low-resolution models under the guidance of corresponding high-resolution models. Nevertheless, due to the resolution gap between features in high/low-resolution models, straightforward distillation is difficult to apply. To overcome this challenge, we first introduce a feature-shifted teacher (FST) to shift and fuse features of high/low-resolution models. Then, we exploit this teacher model to guide the learning of the low-resolution student model with distillation losses on both features and deformation fields. Finally, we only need to use the distilled student model during inference. Experimental results on four 3D medical image datasets demonstrate that the low-resolution models trained through our CRD scheme use fewer than 20% GPU memories and less than 20% inference time while achieving competitive performance compared with corresponding high-resolution models.

Efficient convolution-based pairwise elastic image registration on three multimodal similarity metrics

This paper proposes a complete convolutional formulation for 2D multimodal pairwise image registration problems based on free-form deformations. We have reformulated in terms of discrete 1D convolutions the evaluation of spatial transformations, the regularization term, and their gradients for three different multimodal registration metrics, namely, normalized cross correlation, mutual information, and normalized mutual information. A sufficient condition on the metric gradient is provided for further extension to other metrics. The proposed approach has been tested, as a proof of concept, on contrast-enhanced first-pass perfusion cardiac magnetic resonance images. Execution times have been compared with the corresponding execution times of the classical tensor product formulation, both on CPU and GPU. The speed-up achieved by using convolutions instead of tensor products depends on the image size and the number of control points considered, the larger those magnitudes, the greater the execution time reduction. Furthermore, the speed-up will be more significant when gradient operations constitute the major bottleneck in the optimization process.

Fast and large-converge-radius inverse compositional Levenberg–Marquardt algorithm for digital image correlation: principle, validation, and open-source toolbox

This paper presents an inverse compositional Levenberg-Marquardt (IC-LM) algorithm for robust, efficient, and accurate image registration in digital image correlation (DIC). In essence, the IC-LM algorithm is a mixture of the classical inverse compositional Gaussian-Newton (IC-GN) and gradient descent algorithms. Further normalization of the local coordinate and image intensity is also introduced to adaptively initialize the damping parameter in the IC-LM algorithm. The proposed IC-LM algorithm is proven to hold a larger converge radius while having comparable accuracy, precision, and efficiency compared with the classical IC-GN algorithm. The efficient reliability-guided displacement tracking strategy is also merged into the IC-LM algorithm to provide an accurate initial guess for all calculation points. For the sake of reproducibility of this algorithm, the open-source MATLAB toolbox featuring the IC-LM algorithm is available on GitHub (https://github.com/cbbuaa/DIC_ICLM_MATLAB).

Development of an Image Registration Technique for Fluvial Hyperspectral Imagery Using an Optical Flow Algorithm.

Fluvial remote sensing has been used to monitor diverse riverine properties through processes such as river bathymetry and visual detection of suspended sediment, algal blooms, and bed materials more efficiently than laborious and expensive in-situ measurements. Red–green–blue (RGB) optical sensors have been widely used in traditional fluvial remote sensing. However, owing to their three confined bands, they rely on visual inspection for qualitative assessments and are limited to performing quantitative and accurate monitoring. Recent advances in hyperspectral imaging in the fluvial domain have enabled hyperspectral images to be geared with more than 150 spectral bands. Thus, various riverine properties can be quantitatively characterized using sensors in low-altitude unmanned aerial vehicles (UAVs) with a high spatial resolution. Many efforts are ongoing to take full advantage of hyperspectral band information in fluvial research. Although geo-referenced hyperspectral images can be acquired for satellites and manned airplanes, few attempts have been made using UAVs. This is mainly because the synthesis of line-scanned images on top of image registration using UAVs is more difficult owing to the highly sensitive and heavy image driven by dense spatial resolution. Therefore, in this study, we propose a practical technique for achieving high spatial accuracy in UAV-based fluvial hyperspectral imaging through efficient image registration using an optical flow algorithm. Template matching algorithms are the most common image registration technique in RGB-based remote sensing; however, they require many calculations and can be error-prone depending on the user, as decisions regarding various parameters are required. Furthermore, the spatial accuracy of this technique needs to be verified, as it has not been widely applied to hyperspectral imagery. The proposed technique resulted in an average reduction of spatial errors by 91.9%, compared to the case where the image registration technique was not applied, and by 78.7% compared to template matching.

A Coarse-to-Fine Deformable Transformation Framework for Unsupervised Multi-Contrast MR Image Registration with Dual Consistency Constraint.

Multi-contrast magnetic resonance (MR) image registration is useful in the clinic to achieve fast and accurate imaging-based disease diagnosis and treatment planning. Nevertheless, the efficiency and performance of the existing registration algorithms can still be improved. In this paper, we propose a novel unsupervised learning-based framework to achieve accurate and efficient multi-contrast MR image registration. Specifically, an end-to-end coarse-to-fine network architecture consisting of affine and deformable transformations is designed to improve the robustness and achieve end-to-end registration. Furthermore, a dual consistency constraint and a new prior knowledge-based loss function are developed to enhance the registration performances. The proposed method has been evaluated on a clinical dataset containing 555 cases, and encouraging performances have been achieved. Compared to the commonly utilized registration methods, including VoxelMorph, SyN, and LT-Net, the proposed method achieves better registration performance with a Dice score of 0.8397± 0.0756 in identifying stroke lesions. With regards to the registration speed, our method is about 10 times faster than the most competitive method of SyN (Affine) when testing on a CPU. Moreover, we prove that our method can still perform well on more challenging tasks with lacking scanning information data, showing the high robustness for the clinical application.

A Generalized Tool for Accurate and Efficient Image Registration of UAV Multi-lens Multispectral Cameras by N-SURF Matching

The original multispectral (MS) images obtained from multi-lens multispectral cameras (MSCs) have significant misregistration errors, which require image registration for precise spectral measurement. However, due to the nonlinearity intensity differences among MS images, performing image matching is difficult to find sufficient correct matches (CMs) for image registration, and results in a complex coarse-to-fine solution. Based on the modification of speed-up robust feature (SURF), we proposed a normalized SURF (N-SURF) that can significantly increase the amount of CMs among different pairs of MS images and make one-step image registration possible. In this study, we first introduce N-SURF and adopt different MS datasets acquired from three representative MSCs (MCA-12, Altum, and Sequoia) to evaluate its matching ability. Meanwhile, we utilized three image transformation models—affine transform (AT), projective transform (PT), and an extended projective transform (EPT) to correct the misregistration errors of MSCs and evaluate their co-registration correctness. The results show that N-SURF can obtain 6–20 times more CMs than SURF and can successfully match all pairs of MS images, while SURF failed in the cases of significant spectral differences. Moreover, visual comparison, accuracy assessment, and residual analysis show that EPT can more accurately correct the viewpoint and lens distortion differences of MSCs than AT and PT, and it can obtain co-registration accuracy of 0.2–0.4 pixels. Subsequently, using the successful N-SURF matching and EPT model, we developed an automatic MS image registration tool that is suitable for various multilens MSCs.

SIFT Image Feature Extraction for an Efficient Image Registration

This paper presents an image registration algorithm based on SIFT (Scale Invariant Feature Transform).The obtained descriptors and key points by the SIFT confirms that, the algorithm is very robust to scaling, noise, translation and rotation. At the beginning, the key points are extracted from the image. Later to Match the obtained points, dot products between the unit vectors are calculated. Finally, transformation matrix is obtained by applying RANSAC algorithm. Experimental results shows that the algorithm extracts the better key points, which can be used for used for image registration applications.

Non-Rigid Multi-Modal 3D Medical Image Registration Based on Foveated Modality Independent Neighborhood Descriptor.

The non-rigid multi-modal three-dimensional (3D) medical image registration is highly challenging due to the difficulty in the construction of similarity measure and the solution of non-rigid transformation parameters. A novel structural representation based registration method is proposed to address these problems. Firstly, an improved modality independent neighborhood descriptor (MIND) that is based on the foveated nonlocal self-similarity is designed for the effective structural representations of 3D medical images to transform multi-modal image registration into mono-modal one. The sum of absolute differences between structural representations is computed as the similarity measure. Subsequently, the foveated MIND based spatial constraint is introduced into the Markov random field (MRF) optimization to reduce the number of transformation parameters and restrict the calculation of the energy function in the image region involving non-rigid deformation. Finally, the accurate and efficient 3D medical image registration is realized by minimizing the similarity measure based MRF energy function. Extensive experiments on 3D positron emission tomography (PET), computed tomography (CT), T1, T2, and (proton density) PD weighted magnetic resonance (MR) images with synthetic deformation demonstrate that the proposed method has higher computational efficiency and registration accuracy in terms of target registration error (TRE) than the registration methods that are based on the hybrid L-BFGS-B and cat swarm optimization (HLCSO), the sum of squared differences on entropy images, the MIND, and the self-similarity context (SSC) descriptor, except that it provides slightly bigger TRE than the HLCSO for CT-PET image registration. Experiments on real MR and ultrasound images with unknown deformation have also be done to demonstrate the practicality and superiority of the proposed method.

An Automatic Medical Image Registration Approach Based on Common Sub-regions of Interest

Accurate and efficient image registration, based on interested common sub-regions is still a challenging task in medical image analysis. This paper presents an automatic features based approach for the rigid and deformable registration of medical images using interested common sub-regions. In the proposed approach, interested common sub-regions in two images (target image and source image) are automatically detected and locally registered. The final global registration is performed, using the transformation parameters obtained from the local registration. Registration using interested common sub-regions is always required in image guided surgery (IGS) and other medical procedures because it considers only the desired objects in medical images instead of the whole image contents. The proposed interested common sub-regions based registration is compared with the two states-of-the-art methods on MR images of human brain. In the experiments of rigid and deformable registrations, we show that our approach outperforms in terms of both the accuracy and time efficiency. The results reveal that interested common sub-region based registration can achieve good performance, regarding both the accuracy as well as the the time efficiency in monomodal brain image registration. In addition, the proposed approach also indicates the potential for multimodal images of different human organs.

Efficient diffeomorphic metric image registration via stationary velocity

Diffeomorphic image registration is a fundamental tool for MRI analysis, and fast growing image data demand highly efficient registration methods. Stationary velocity based method has much faster speed than non-stationary one in producing diffeomorphism, but how to achieve validity for large deformation while keeping its efficiency remains to be solved. To this end, we have proposed a new simplified model for optimal stationary velocity by representing temporal integration form of transformation variation via a single point model, e.g. the middle time point, according to mean value theorem and smoothness of transformation. This model can maintain the same registration accuracy as integral model but reduce the time cost a lot. It also shows as a better strategy than first order approximation model for large deformation mapping. Comparative study has also been conducted between this method and non-stationary approach on both synthesized and real brain images, and this approach demonstrated comparable whole brain mapping accuracy with much faster speed. Results showed efficacy of this approach in reducing complexity of stationary velocity based diffeomorphic registration while achieving validity for large deformation.

Efficient and Robust Direct Image Registration Based on Joint Geometric and Photometric Lie Algebra.

This paper considers the joint geometric and photometric image registration problem. The inverse compositional (IC) algorithm and the efficient second-order minimization (ESM) algorithm are two typical efficient methods applied to the geometric registration problem. Their efficiency stems from the utilization of the group structure of geometric transformations. To allow for photometric variations, the dual IC algorithm (DIC) proposed by Bartoli performs joint geometric and photometric image registration by extending the IC algorithm. The group structures of both geometric and photometric transformations are exploited. Despite the robustness to large photometric variations, DIC is vulnerable to large geometric deformations. The ESM algorithm is extended by Silveira et al. to address photometric variations. In their approach, the photometric transformations are modeled in Euclidean space. Their approach is robust to relatively large geometric and photometric transformations; however, it is not efficient for large photometric variations. We propose a new efficient and robust image registration method by exploiting the non-Euclidean Lie group structure of joint geometric and photometric transformations for both grayscale and color images. The image registration is formulated as a nonlinear least squares problem. In our method, the geometric and photometric transformations are jointly parameterized by their corresponding Lie algebras. Based on this parameterization approach, the second-order approximation strategy of ESM is employed to optimize the joint geometric and photometric parameters. The error function in the nonlinear least squares problem is approximated by a second-order Taylor expansion with respect to joint geometric and photometric parameters without computing the Hessian matrix. For further efficiency, independent convergence criteria for geometric and photometric parameters are used in the iterative optimization process. The superiority of our proposed method over the previous methods, in terms of efficiency, accuracy, and robustness, is demonstrated through extensive experiments on synthetic and real data.

Efficient feature-based image registration by mapping sparsified surfaces

With the advancement in the digital camera technology, the use of high resolution images and videos has been widespread in the modern society. In particular, image and video frame registration is frequently applied in computer graphics and film production. However, conventional registration approaches usually require long computational time for high resolution images and video frames. This hinders the application of the registration approaches in the modern industries. In this work, we first propose a new image representation method to accelerate the registration process by triangulating the images effectively. For each high resolution image or video frame, we compute an optimal coarse triangulation which captures the important features of the image. Then, we apply a surface registration algorithm to obtain a registration map which is used to compute the registration of the high resolution image. Experimental results suggest that our overall algorithm is efficient and capable to achieve a high compression rate while the accuracy of the registration is well retained when compared with the conventional grid-based approach. Also, the computational time of the registration is significantly reduced using our triangulation-based approach.

Regularized Surface and Point Landmarks Based Efficient Non-Rigid Medical Image Registration

Medical image registration is one of the fundamental tasks in medical image processing. It has various applications in field of image guided surgery (IGS) and computer assisted diagnosis (CAD). A set of non-linear methods have been already developed for inter-subject and intra-subject 3D medical image registration. However, efficient registration in terms of accuracy and speed is one of the most demanded of today surgical navigation (SN) systems. This paper is a result of a series of experiments which utilizes Fast Radial Basis Function (RBF) technique to register one or more medical images non-rigidly. Initially, a set of curves are extracted using a combined watershed and active contours algorithm and then tiled and converted to a regular surface using a global parameterization algorithm. It is shown that the registration accuracy improves when higher number of salient features (i.e. anatomical point landmarks and surfaces) are used and it also has no impact on the speed of the algorithm. The results show that the target registration error is less than 2 mm and has sub-second performance on intra-subject registration of MR image real datasets. It is observed that the Fast RBF algorithm is relatively insensitive to the increasing number of point landmarks used as compared with the competing feature based algorithms.

High-Accuracy Subpixel Image Registration With Large Displacements

This paper aims to achieve computationally efficient and high-accuracy subpixel image registration with large displacements under the rotation–scale–translation model. This paper employs the classical phase correlation algorithm and the Lucas–Kanade (LK) algorithm in a two-stage coarse-to-fine framework, for which the motivation is from the observation that the two algorithms exhibit strong complementary property between convergence range and subpixel accuracy. In this framework, the LK algorithm will also become computationally efficient owing to the small residual displacement. On the other hand, this paper takes into account the residual model with respect to the compensation scheme explicitly, and deduces formulas for the final results combination, which is expected to be closer to the true displacement vector and thus further improve the estimation accuracy. Since the compensation can be applied to either the target image or the reference image, two algorithms are presented accordingly, and analysis as well as comparison are also performed. Finally, both simulations and real image experiments are performed to verify the motivation, and the results are consistent with the analysis.