Non-rigid Medical Image Registration Research Articles

An Adaptive Region-Based Transformer for Nonrigid Medical Image Registration With a Self-Constructing Latent Graph.

Nonrigid registration of medical images is formulated usually as an optimization problem with the aim of seeking out the deformation field between a referential-moving image pair. During the past several years, advances have been achieved in the convolutional neural network (CNN)-based registration of images, whose performance was superior to most conventional methods. More lately, the long-range spatial correlations in images have been learned by incorporating an attention-based model into the transformer network. However, medical images often contain plural regions with structures that vary in size. The majority of the CNN-and transformer-based approaches adopt embedding of patches that are identical in size, disallowing representation of the inter-regional structural disparities within an image. Besides, it probably leads to the structural and semantical inconsistencies of objects as well. To address this issue, we put forward an innovative module called region-based structural relevance embedding (RSRE), which allows adaptive embedding of an image into unequally-sized structural regions based on the similarity of self-constructing latent graph instead of utilizing patches that are identical in size. Additionally, a transformer is integrated with the proposed module to serve as an adaptive region-based transformer (ART) for registering medical images nonrigidly. As demonstrated by the experimental outcomes, our ART is superior to the advanced nonrigid registration approaches in performance, whose Dice score is 0.734 on the LPBA40 dataset with 0.318% foldings for deformation field, and is 0.873 on the ADNI dataset with 0.331% foldings.

DRT: Deformable Region-Based Transformer for Nonrigid Medical Image Registration With a Constraint of Orientation

Nonrigid medical image registration is a key technology for clinical disease diagnosis by integrating the vital information taken from input images acquired using multiple imaging sensors. Recently, transformer networks have attracted attention in image registration, but splitting images into patches remains a problem. Although there are usually multiple structural regions with different sizes and shapes in medical images, current transformer methods usually use an equal-sized or rectangular patch embedding that cannot represent the structural differences and directionality of different regions. In addition, it may also destroy the semantic consistency of objects. To address this problem, we propose a new deformable region-based structural relevance embedding (DRSRE) module which learns to adaptively split the images into deformable structural regions of different sizes and shapes with orientation constraints rather than using equal-sized or rectangular patches. In this way, our method can represent the structural differences and directionality between regions and effectively preserve the semantic consistency in regions to calculate accurate transformation relationships for nonrigid image registration. We term the DRSRE module with a transformer part as a deformable region-based transformer (DRT) and conduct extensive evaluations of DRT in the context of nonrigid medical image registration, and the results validate that the proposed DRT outperforms the state-of-the-art nonrigid registration methods, achieving 0.748 and 0.886 Dice scores with 0.217% and 0.308% foldings for deformation field on the LPBA40 and ADNI datasets.

Non-rigid registration of medical images based on {S}_2^1left({Delta}_{mn}^{(2)}right) non-tensor product B-spline

In this study, a non-tensor product B-spline algorithm is applied to the search space of the registration process, and a new method of image non-rigid registration is proposed. The tensor product B-spline is a function defined in the two directions of x and y, while the non-tensor product B-spline {S}_2^1left({Delta}_{mn}^{(2)}right) is defined in four directions on the 2-type triangulation. For certain problems, using non-tensor product B-splines to describe the non-rigid deformation of an image can more accurately extract the four-directional information of the image, thereby describing the global or local non-rigid deformation of the image in more directions. Indeed, it provides a method to solve the problem of image deformation in multiple directions. In addition, the region of interest of medical images is irregular, and usually no value exists on the boundary triangle. The value of the basis function of the non-tensor product B-spline on the boundary triangle is only 0. The algorithm process is optimized. The algorithm performs completely automatic non-rigid registration of computed tomography and magnetic resonance imaging images of patients. In particular, this study compares the performance of the proposed algorithm with the tensor product B-spline registration algorithm. The results elucidate that the proposed algorithm clearly improves the accuracy.

Toward Computing Cross-Modality Symmetric Non-Rigid Medical Image Registration

This paper describes a new non-rigid approach to register images from same- and cross-imaging modalities such as magnetic resonance imaging, computed tomography, and 3D rotational angiography. The deformation is a key challenge in medical image registration. We have proposed a diffeomorphism-based method to tackle this problem using an optimized framework. A non stationary velocity field is used to minimize the effect of forces that are derived from the image gradients. Furthermore, we propose a similarity energy function, based on the gray scale distribution, to limit the fluctuations while approaching the local minima. The proposed method is evaluated on both private and public datasets; the results show that the values of mean square error (MSE), normalized cross-correlation (NCC), structural similarity (SS), mutual information (MI), feature similarity index (FSIM), and mean absolute error (MAE) are 1.3136, 0.9962, 0.9897, 0.883, 0.9922, and 1.52± 2.09, respectively. Both qualitative and quantitative evaluation show promising registration accuracy reflecting the potential of the proposed method.

Uncertainty quantification in non-rigid image registration via stochastic gradient Markov chain Monte Carlo

We develop a new Bayesian model for non-rigid registration of three-dimensional medical images, with a focus on uncertainty quantification. Probabilistic registration of large images with calibrated uncertainty estimates is difficult for both computational and modelling reasons. To address the computational issues, we explore connections between the Markov chain Monte Carlo by backpropagation and the variational inference by backpropagation frameworks, in order to efficiently draw samples from the posterior distribution of transformation parameters. To address the modelling issues, we formulate a Bayesian model for image registration that overcomes the existing barriers when using a dense, high-dimensional, and diffeomorphic transformation parametrisation. This results in improved calibration of uncertainty estimates. We compare the model in terms of both image registration accuracy and uncertainty quantification to VoxelMorph, a state-of-the-art image registration model based on deep learning.

Bayesian Fully Convolutional Networks for Brain Image Registration.

The purpose of medical image registration is to find geometric transformations that align two medical images so that the corresponding voxels on two images are spatially consistent. Nonrigid medical image registration is a key step in medical image processing, such as image comparison, data fusion, target recognition, and pathological change analysis. Existing registration methods only consider registration accuracy but largely neglect the uncertainty of registration results. In this work, a method based on the Bayesian fully convolutional neural network is proposed for nonrigid medical image registration. The proposed method can generate a geometric uncertainty map to calculate the uncertainty of registration results. This uncertainty can be interpreted as a confidence interval, which is essential for judging whether the source data are abnormal. Moreover, the proposed method introduces group normalization, which is conducive to the network convergence of the Bayesian neural network. Some representative learning-based image registration methods are compared with the proposed method on different image datasets. Experimental results show that the registration accuracy of the proposed method is better than that of the methods, and its antifolding performance is comparable to that of fast image registration and VoxelMorph. Furthermore, the proposed method can evaluate the uncertainty of registration results.

Non-Rigid Image Registration based on Parameterized Surfaces: Application to 3D Cardiac Motion Image Analysis

This paper describes the Fast Radial Basis Function (RBF) method for cardiac motion tracking in 3D CT using non-rigid medical image registration based on parameterized (regular) surfaces. The technique is a point-based registration evaluation algorithm which does register 3D MR or CT images in real time. We first extract the surface of the whole heart 3D CT and its contrast enhanced part (left ventricle (LV) blood cavity) of each dataset with a semiautomatic contouring and a fully-automatic triangulation method followed by a global surface parameterization and optimization algorithm. In second step, a set of registration experiments are run to calculate the deformation field at various phases of cardiac motion or cycle from CT images, which results into significant deformation during each phase of a cycle. The surface points of the whole heart and LV are used to register the source systole image to various diastole target images taken at different phases during a heart beat. Our registration accuracy improves with the increase in number of salient feature points (i.e. optimized parameterized surfaces) and it has no effect on the speed of the algorithm (i.e. still less than a second). The results show that the target registration error is less than 3[Formula: see text]mm (2.53) and the performance of the Fast RBF algorithm is less than a second using a whole heart CT dataset of a single patient taken over the course of the entire cardiac cycle. At the end, the results for recovery (or analysis) of bigger deformation in heart CT images using the Fast RBF algorithm is compared to the state-of-the-art Free Form Deformation (FFD) registration technique. It is proved that the Fast RBF method is performing better in speed and slightly less accurate than the FFD (when measured in terms of NMI) due to iterative nature of the latter.

Multi-Modality Non-rigid Image Registration Using Local Similarity Estimations

In this study, we focus on improving the efficiency and accuracy of nonrigid multi-modality registration of medical images. In this regard, we analyze the potentials of using the point similarity measurement approach as an alternative to global computation of mutual information (MI), which is still the most renown multi-modality similarity measure. The improvement capabilities are illustrated using the popular B-spline transformation model. The proposed solution is a combination of three related improvements of the most straightforward implementation, i.e., efficient computation of the voxel displacement field, local estimation of similarity and usage of a static image intensity dependence estimate. Five image registration prototypes were implemented to show contribution and dependence of the proposed improvements. When all the proposed improvements are applied, a significant reduction of computational cost and increased accuracy are obtained. The concept offers additional improvement opportunities by incorporating prior knowledge and machine learning techniques into the static intensity dependence estimation.

Non-rigid medical image registration using multi-scale residual deep fully convolutional networks

Medical image registration technology is important in medical image processing, which can establish correspondence between complex medical images and ensure the comparability of image data. Since the deep convolutional neural network model is prone to degradation and the convolution kernel has limited detection range during training, a Multi-scale residual full convolutional network (MS-ResFCN) model is proposed for unsupervised non-rigid medical image registration tasks. The model introduces a residual structure in the Fully Convolutional Network (FCN) model to ensure effective and stable training. At the same time, a hierarchical multi-scale convolution kernel is constructed within a single convolutional layer of the residual structure, which enhances the nonlinear mapping ability of the network. The local texture information and the contextual information of the image are synchronously extracted and fused to enrich the diversity of features. The experiment results on LPBA40 data sets show that the MS-ResFCN model can effectively eliminate the degradation phenomenon of the deep network during the training process and extract multi-scale features, which achieves better feature representation ability and registration accuracy.

Validation of a diffeomorphic registration algorithm using true deformation computed from thin plate spline interpolation.

Despite the inter and intraobserver variabilities, manual contours are commonly used as surrogates for ground truth in the validation process for nonrigid medical image registration. In contrast, this study proposes the use of thin plate spline interpolation to create a true deformation field. A diffeomorphic registration method was compared to the true deformation field along with three other algorithms and was evaluated on simulated cardiac motion deformation over 10 subjects from the Automated Cardiac Diagnosis Challenge (ACDC) dataset. Two sequential registration approaches were undertaken: with respect to the first frame, and with respect to the previous frame. The Dice score was calculated between the simulated and warped contours for the two approaches: diffeomorphic registration method =0.991 and 0.997, RealTITracker (L2L2 method) = 0.971 and 0.977, RealTITracker (L2L1 method) = 0.975 and 0.978, and Elastix = 0.976 and 0.994. The results demonstrate the robust performance of the diffeomorphic registration method.Clinical relevance This establishes a validation of a registration method that can be used for segmentation of chambers of the heart.

A CNN-Based Approach for Lung 3D-CT Registration

Deep learning techniques have been applied to certain rigid or non-rigid medical image registration due to its potential advantages in meeting the clinical requirements of real-time and accuracy. Based on the deep learning model, this study aims to explore specific network models suitable for lung CT images. The proposed model took unlabeled 3D image pairs as input, and the convolutional neural network (CNN) was utilized and identified as a function with ability of sharing parameters to obtain displacement field. The image pair could be aligned by applying the acquired displacement field to the target image through spatial transformation. The similarity between the aligned image pair combined with the constraints on the displacement field was taken as the objective function to obtain the optimal parameters. Two models with different depths were designed and the consequent registration effects with different optimization methods and convolution kernel sizes were explored. The results proved that the designs with deeper level using Adam optimizer and smaller convolution kernels in obtaining displacement fields had higher accuracy and stronger robustness. The accuracy of the unsupervised model was comparable to state-of-the-art methods, while operating orders of magnitude faster. This study proposed a feasible registration method for lung 3D-CT, and its usefulness in aligning CT images has been demonstrated.

Nonrigid registration of medical image based on adaptive local structure tensor and normalized mutual information.

Nonrigid registration of medical images is especially critical in clinical treatment. Mutual information is a popular similarity measure for medical image registration; however, only the intensity statistical characteristics of the global consistency of image are considered in MI, and the spatial information is ignored. In this paper, a novel intensity‐based similarity measure combining normalized mutual information with spatial information for nonrigid medical image registration is proposed. The different parameters of Gaussian filtering are defined according to the regional variance, the adaptive Gaussian filtering is introduced into the local structure tensor. Then, the obtained adaptive local structure tensor is used to extract the spatial information and define the weighting function. Finally, normalized mutual information is distributed to each pixel, and the discrete normalized mutual information is multiplied with a weighting term to obtain a new measure. The novel measure fully considers the spatial information of the image neighborhood, gives the location of the strong spatial information a larger weight, and the registration of the strong gradient regions has a priority over the small gradient regions. The simulated brain image with single‐modality and multimodality are used for registration validation experiments. The results show that the new similarity measure improves the registration accuracy and robustness compared with the classical registration algorithm, reduces the risk of falling into local extremes during the registration process.

Non-rigid medical image registration using image field in Demons algorithm

In medical imaging application, non-rigid registration is an important step that requires both accuracy and efficiency to align the deformed query image with the reference image. For non-rigid registration, Demons is generally used as an effective algorithm. However, since the computational model of Demons uses only the gradient information, the directional information of the image has not been fully utilized. In this paper, we proposed a novel non-rigid registration method for processing medical images by introducing the image field to the Demons algorithm. As direction information is very important for spatial transformation in registration and the image fields contain the direction of the image, by combing image fields with the traditional model based registration algorithm, we introduce the orientation field directly to the model for estimating the distortion, and thus make a better use of the direction information of images and simplify the deformation model. Different performance measures are used to evaluate the qualitative measures of the proposed scheme. Experiments on brain images and fundus images show the improvement of the proposed algorithm compared to the other existing state-of-the-art methods.

Integrating Morphological Edge Detection and Mutual Information for Nonrigid Registration of Medical Images.

Medical images are widely used within healthcare and medical research. There is an increased interest in precisely correlating information in these images through registration techniques for investigative and therapeutic purposes. This work proposes and evaluates an improved measure function for registration of carotid ultrasound and magnetic resonance images (MRI) taken at different times. To achieve this, a morphological edge detection operator has been designed to extract the vital edge information from images which is integrated with the Mutual Information (MI) to carry out the registration process. The improved performance of proposed registration measure function is demonstrated using four quality metrics: Correlation Coefficient (CC), Structural Similarity Index (SSIM), Visual Information Fidelity (VIF) and Gradient Magnitude Similarity Deviation (GMSD). The qualitative validation has also been done through visual inspection of the registered image pairs by clinical radiologists. The experimental results showed that the proposed method outperformed the existing method (based on integrated MI and standard edge detection) for both ultrasound and MR images in terms of CC by about 4.67%, SSIM by 3.21%, VIF by 18.5%, and decreased GMSD by 37.01%. Whereas, in comparison to the standard MI based method, the proposed method has increased CC by 16.29%, SSIM by 16.13%, VIF by 52.56% and decreased GMSD by 66.06%, approximately. Thus, the proposed method improves the registration accuracy when the original images are corrupted by noise, have low intensity values or missing data.

Nonrigid Medical Image Registration Using an Information Theoretic Measure Based on Arimoto Entropy with Gradient Distributions.

This paper introduces a new nonrigid registration approach for medical images applying an information theoretic measure based on Arimoto entropy with gradient distributions. A normalized dissimilarity measure based on Arimoto entropy is presented, which is employed to measure the independence between two images. In addition, a regularization term is integrated into the cost function to obtain the smooth elastic deformation. To take the spatial information between voxels into account, the distance of gradient distributions is constructed. The goal of nonrigid alignment is to find the optimal solution of a cost function including a dissimilarity measure, a regularization term, and a distance term between the gradient distributions of two images to be registered, which would achieve a minimum value when two misaligned images are perfectly registered using limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) optimization scheme. To evaluate the test results of our presented algorithm in non-rigid medical image registration, experiments on simulated three-dimension (3D) brain magnetic resonance imaging (MR) images, real 3D thoracic computed tomography (CT) volumes and 3D cardiac CT volumes were carried out on elastix package. Comparison studies including mutual information (MI) and the approach without considering spatial information were conducted. These results demonstrate a slight improvement in accuracy of non-rigid registration.

The same modality medical image registration with large deformation and clinical application based on adaptive diffeomorphic multi-resolution demons

BackgroundDiffeomorphic demons can not only guarantee smooth and reversible deformation, but also avoid unreasonable deformation. However, the number of iterations which has great influence on the registration result needs to be set manually.MethodsThis study proposed a novel method to exploit the adaptive diffeomorphic multi-resolution demons algorithm to the non-rigid registration of the same modality medical images with large deformation. Firstly an optimized non-rigid registration framework and the diffeomorphism strategy were used, and then a similarity energy function based on the grey value was designed as registration metric, lastly termination condition was set based on the variation of this metric and iterations can be stopped adaptively. Quantitative analyses based on the registration evaluation indexes were conducted to prove the validity of this method.ResultsRegistration result of synthetic image and the same modality MRI and CT image was compared with those obtained by other demons algorithms. Quantitative analyses demonstrated the proposed method’s superiority. Medical image with large deformation was produced by rotational distortion and extrusion transform, and the same modality image registration with large deformation was performed successfully. Quantitative analyses showed that the registration evaluation indexes remained stable with an increase in transform strength. This method can be also applied to pulmonary medical image registration with large deformation successfully, and it showed the clinical application value. The influence of different driving forces and parameters on the registration result was analysed, and the result demonstrated that the proposed method is effective and robust.ConclusionsThis method can solve the non-rigid registration problem of the same modality medical image with large deformation showing promise for diagnostic pulmonary imaging applications.

Co-Sparse Analysis Model Based Image Registration to Compensate Brain Shift by Using Intra-Operative Ultrasound Imaging.

Notwithstanding the widespread use of image guided neurosurgery systems in recent years, the accuracy of these systems is strongly limited by the intra-operative deformation of the brain tissue, the so-called brain shift. Intra-operative ultrasound (iUS) imaging as an effective solution to compensate complex brain shift phenomena update patients coordinate during surgery by registration of the intra-operative ultrasound and the pre-operative MRI data that is a challenging problem.In this work a non-rigid multimodal image registration technique based on co-sparse analysis model is proposed. This model captures the interdependency of two image modalities; MRI as an intensity image and iUS as a depth image. Based on this model, the transformation between the two modalities is minimized by using a bimodal pair of analysis operators which are learned by optimizing a joint co-sparsity function using a conjugate gradient.Experimental validation of our algorithm confirms that our registration approach outperforms several of other state-of-the-art registration methods quantitatively. The evaluation was performed using seven patient dataset with the mean registration error of only 1.83 mm. Our intensity-based co-sparse analysis model has improved the accuracy of non-rigid multimodal medical image registration by 15.37% compared to the curvelet based residual complexity as a powerful registration method, in a computational time compatible with clinical use.

Non-rigid Image Registration by Minimizing Weighted Residual Complexity

Background: Non-rigid registration of medical images with intensity distortions is a difficult problem due to the change in pixel intensity. It is caused by contrast agent or intensity bias field. Methods: In some cases, this problem can be solved using Residual Complexity (RC) method. However, relative modification of parameter in residual complexity would result in completely different experimental effect. Another drawback is sensitivity to noise. To handle this problem, a new intensity-based similarity measure, Weighted Residual Complexity (WRC) has been proposed for effective medical image registration in this paper. Specifically, the local entropy image of two images is computed to be aligned respectively. Then, a weighting function using a function of the local entropy difference is modeled. The weighting function is used to weight the residual image in residual complexity adaptively. The residual image is defined as the difference between reference image and warped floating image. Results: The weighting function assigns smaller weight to residual image if the corresponding pixel value is larger in local entropy difference. The proposed technique was applied to simulative and real medical images. The contrast experiments were made with mutual information, diffeomorphic demons and residual complexity. Conclusion: Also, the analysis of experimental results was made qualitatively and quantitatively, which indicates that this new approach gives a better performance than diffeomorphic demons, mutual information and residual complexity. Keywords: Weighting function, local entropy, residual complexity, intensity distortion, image registration, WRC.

Non-rigid registration of medical images using a hierarchical framework with normalized mutual information

Medical image registration plays a crucial role in medical field. Existing medical registration methods can be categorized as rigid registration and non-rigid registration methods. Compared with rigid registration, the non-rigid registration is more suitable for representing the anatomy’s complicated deformation. In this study, we develop a non-rigid registration method in which Normalized Mutual Information (NMI) is used as similarity measurement and B-splines are used for transformation. The algorithm is implemented using a hierarchical framework such that the accuracy of registration is improved by the coarse-to-fine registration schema. Experiments demonstrate that the proposed method produces more accurate results than the affine and demons methods in registration of pelvis CBCT images, which provides an alternative approach to support non-rigid registration of medical images with intensity variation. The proposed algorithm shows potential for facilitating image-guided radiation therapy with higher treatment dose delivered to a precise location through the use of image registration.

Adaptive Diffeomorphic Multiresolution Demons and Their Application to Same Modality Medical Image Registration with Large Deformation.

Diffeomorphic demons can guarantee smooth and reversible deformation and avoid unreasonable deformation. However, the number of iterations needs to be set manually, and this greatly influences the registration result. In order to solve this problem, we proposed adaptive diffeomorphic multiresolution demons in this paper. We used an optimized framework with nonrigid registration and diffeomorphism strategy, designed a similarity energy function based on grey value, and stopped iterations adaptively. This method was tested by synthetic image and same modality medical image. Large deformation was simulated by rotational distortion and extrusion transform, medical image registration with large deformation was performed, and quantitative analyses were conducted using the registration evaluation indexes, and the influence of different driving forces and parameters on the registration result was analyzed. The registration results of same modality medical images were compared with those obtained using active demons, additive demons, and diffeomorphic demons. Quantitative analyses showed that the proposed method's normalized cross-correlation coefficient and structural similarity were the highest and mean square error was the lowest. Medical image registration with large deformation could be performed successfully; evaluation indexes remained stable with an increase in deformation strength. The proposed method is effective and robust, and it can be applied to nonrigid registration of same modality medical images with large deformation.