Medical Image Registration Approach Research Articles

Enhancing voxel-based dosimetry accuracy with an unsupervised deep learning approach for hybrid medical image registration.

Deformable registration is required to generate a time-integrated activity (TIA) map which is essential for voxel-based dosimetry. The conventional iterative registration algorithm using anatomical images (e.g., computed tomography (CT)) could result in registration errors in functional images (e.g., single photon emission computed tomography (SPECT) or positron emission tomography (PET)). Various deep learning-based registration tools have been proposed, but studies specifically focused on the registration of serial hybrid images were not found. In this study, we introduce CoRX-NET, a novel unsupervised deep learning network designed for deformable registration of hybrid medical images. The CoRX-NET structure is based on the Swin-transformer (ST), allowing for the representation of complex spatial connections in images. Its self-attention mechanism aids in the effective exchange and integration of information across diverse image regions. To augment the amalgamation of SPECT and CT features, cross-stitch layers have been integrated into the network. Two different 177 Lu DOTATATE SPECT/CT datasets were acquired at different medical centers. 22 sets from Seoul National University and 14 sets from Sunway Medical Centre are used for training/internal validation and external validation respectively. The CoRX-NET architecture builds upon the ST, enabling the modeling of intricate spatial relationships within images. To further enhance the fusion of SPECT and CT features, cross-stitch layers have been incorporated within the network. The network takes a pair of SPECT/CT images (e.g., fixed and moving images) and generates a deformed SPECT/CT image. The performance of the network was compared with Elastix and TransMorph using L1 loss and structural similarity index measure (SSIM) of CT, SSIM of normalized SPECT, and local normalized cross correlation (LNCC) of SPECT as metrics. The voxel-wise root mean square errors (RMSE) of TIA were compared among the different methods. The ablation study revealed that cross-stitch layers improved SPECT/CT registration performance. The cross-stitch layers notably enhance SSIM (internal validation: 0.9614vs. 0.9653, external validation: 0.9159vs. 0.9189) and LNCC of normalized SPECT images (internal validation: 0.7512vs. 0.7670, external validation: 0.8027vs. 0.8027). CoRX-NET with the cross-stitch layer achieved superior performance metrics compared to Elastix and TransMorph, except for CT SSIM in the external dataset. When qualitatively analyzed for both internal and external validation cases, CoRX-NET consistently demonstrated superior SPECT registration results. In addition, CoRX-NET accomplished SPECT/CT image registration in less than 6s, whereas Elastix required approximately 50s using the same PC's CPU. When employing CoRX-NET, it was observed that the voxel-wise RMSE values for TIA were approximately 27% lower for the kidney and 33% lower for the tumor, compared to when Elastix was used. This study represents a major advancement in achieving precise SPECT/CT registration using an unsupervised deep learning network. It outperforms conventional methods like Elastix and TransMorph, reducing uncertainties in TIA maps for more accurate dose assessments.

An approach for medical image registration using a Double Exponential-Dynamic Group Based Cooperative Optimization algorithm

An approach for medical image registration using a Double Exponential-Dynamic Group Based Cooperative Optimization algorithm

SOLID: a novel similarity metric for mono-modal and multi-modal deformable image registration

Medical image registration is an integral part of various clinical applications including image guidance, motion tracking, therapy assessment and diagnosis. We present a robust approach for mono-modal and multi-modal medical image registration. To this end, we propose the novel shape operator based local image distance (SOLID) which estimates the similarity of images by comparing their second-order curvature information. Our similarity metric is rigorously tailored to be suitable for comparing images from different medical imaging modalities or image contrasts. A critical element of our method is the extraction of local features using higher-order shape information, enabling the accurate identification and registration of smaller structures. In order to assess the efficacy of the proposed similarity metric, we have implemented a variational image registration algorithm that relies on the principle of matching the curvature information of the given images. The performance of the proposed algorithm has been evaluated against various alternative state-of-the-art variational registration algorithms. Our experiments involve mono-modal as well as multi-modal and cross-contrast co-registration tasks in a broad variety of anatomical regions. Compared to the evaluated alternative registration methods, the results indicate a very favorable accuracy, precision and robustness of the proposed SOLID method in various highly challenging registration tasks.

AMNet: Adaptive multi-level network for deformable registration of 3D brain MR images.

Three-dimensional (3D) deformable image registration is a fundamental technique in medical image analysis tasks. Although it has been extensively investigated, current deep-learning-based registration models may face the challenges posed by deformations with various degrees of complexity. This paper proposes an adaptive multi-level registration network (AMNet) to retain the continuity of the deformation field and to achieve high-performance registration for 3D brain MR images. First, we design a lightweight registration network with an adaptive growth strategy to learn deformation field from multi-level wavelet sub-bands, which facilitates both global and local optimization and achieves registration with high performance. Second, our AMNet is designed for image-wise registration, which adapts the local importance of a region in accordance with the complexity degrees of its deformation, and thereafter improves the registration efficiency and maintains the continuity of the deformation field. Experimental results from five publicly-available brain MR datasets and a synthetic brain MR dataset show that our method achieves superior performance against state-of-the-art medical image registration approaches.

Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning.

Image registration is a fundamental medical image analysis task, and a wide variety of approaches have been proposed. However, only a few studies have comprehensively compared medical image registration approaches on a wide range of clinically relevant tasks. This limits the development of registration methods, the adoption of research advances into practice, and a fair benchmark across competing approaches. The Learn2Reg challenge addresses these limitations by providing a multi-task medical image registration data set for comprehensive characterisation of deformable registration algorithms. A continuous evaluation will be possible at https:// learn2reg.grand-challenge.org. Learn2Reg covers a wide range of anatomies (brain, abdomen, and thorax), modalities (ultrasound, CT, MR), availability of annotations, as well as intra- and inter-patient registration evaluation. We established an easily accessible framework for training and validation of 3D registration methods, which enabled the compilation of results of over 65 individual method submissions from more than 20 unique teams. We used a complementary set of metrics, including robustness, accuracy, plausibility, and runtime, enabling unique insight into the current state-of-the-art of medical image registration. This paper describes datasets, tasks, evaluation methods and results of the challenge, as well as results of further analysis of transferability to new datasets, the importance of label supervision, and resulting bias. While no single approach worked best across all tasks, many methodological aspects could be identified that push the performance of medical image registration to new state-of-the-art performance. Furthermore, we demystified the common belief that conventional registration methods have to be much slower than deep-learning-based methods.

Networks for Joint Affine and Non-parametric Image Registration.

We introduce an end-to-end deep-learning framework for 3D medical image registration. In contrast to existing approaches, our framework combines two registration methods: an affine registration and a vector momentum-parameterized stationary velocity field (vSVF) model. Specifically, it consists of three stages. In the first stage, a multi-step affine network predicts affine transform parameters. In the second stage, we use a U-Net-like network to generate a momentum, from which a velocity field can be computed via smoothing. Finally, in the third stage, we employ a self-iterable map-based vSVF component to provide a non-parametric refinement based on the current estimate of the transformation map. Once the model is trained, a registration is completed in one forward pass. To evaluate the performance, we conducted longitudinal and cross-subject experiments on 3D magnetic resonance images (MRI) of the knee of the Osteoarthritis Initiative (OAI) dataset. Results show that our framework achieves comparable performance to state-of-the-art medical image registration approaches, but it is much faster, with a better control of transformation regularity including the ability to produce approximately symmetric transformations, and combining affine as well as non-parametric registration.

Adversarial learning for deformable registration of brain MR image using a multi-scale fully convolutional network

Background and objectiveDeformable registration is very significant for various clinical image applications. Unfortunately, existing conventional medical image registration approaches, which involve time-consuming iterative optimization, have not reached the level of routine clinical practice in terms of registration time and robustness. The aim of this study is to propose a tuning-free 3D image registration model based on adversarial deep network, and to achieve rapid and high-accurate registration. MethodsWe propose a fully convolutional network (FCN) to regress the 3D dense deformation field in one shot from the to-be-registered image pair. To precisely regress the complex deformation and produce optimal registration, we design the FCN as a novel multi-scale frame to capture the complementary multi-scale image features and effectively characterize the spatial correspondence between the image pair. Moreover, we learn a discriminator network simultaneously to discriminate the registered two images, where the discrimination loss helps further update the FCN. Thus by the adversarial training strategy, the registration network is urged to produce well-registered two images that are indistinguishable for the discriminator. ResultsWe perform registration experiments on four different brain MR datasets using the model trained by ANDI database. Compared with some state-of-the-art registration algorithms including other newest deeplearning-based methods, the proposed method provides a considerable increase of large than 4% in terms of Dice similarity coefficient (DSC). Moreover, our model also obtains comparable distance errors. More significantly, our model can achieve a high-accurate 3D registration result in average 0.74 s, with roughly hundred speed-up over conventional registration methods. ConclusionsThe proposed model shows consistent high performance for various registration tasks under a second without any additional parameter tuning, which proves its potential for real-time clinical applications.

An Automatic Medical Image Registration Approach Based on Common Sub-regions of Interest

Accurate and efficient image registration, based on interested common sub-regions is still a challenging task in medical image analysis. This paper presents an automatic features based approach for the rigid and deformable registration of medical images using interested common sub-regions. In the proposed approach, interested common sub-regions in two images (target image and source image) are automatically detected and locally registered. The final global registration is performed, using the transformation parameters obtained from the local registration. Registration using interested common sub-regions is always required in image guided surgery (IGS) and other medical procedures because it considers only the desired objects in medical images instead of the whole image contents. The proposed interested common sub-regions based registration is compared with the two states-of-the-art methods on MR images of human brain. In the experiments of rigid and deformable registrations, we show that our approach outperforms in terms of both the accuracy and time efficiency. The results reveal that interested common sub-region based registration can achieve good performance, regarding both the accuracy as well as the the time efficiency in monomodal brain image registration. In addition, the proposed approach also indicates the potential for multimodal images of different human organs.

Intensity-based hierarchical elastic registration using approximating splines

We introduce a new hierarchical approach for elastic medical image registration using approximating splines. In order to obtain the dense deformation field, we employ Gaussian elastic body splines (GEBS) that incorporate anisotropic landmark errors and rotation information. Since the GEBS approach is based on a physical model in form of analytical solutions of the Navier equation, it can very well cope with the local as well as global deformations present in the images by varying the standard deviation of the Gaussian forces. The proposed GEBS approximating model is integrated into the elastic hierarchical image registration framework, which decomposes a nonrigid registration problem into numerous local rigid transformations. The approximating GEBS registration scheme incorporates anisotropic landmark errors as well as rotation information. The anisotropic landmark localization uncertainties can be estimated directly from the image data, and in this case, they represent the minimal stochastic localization error, i.e., the Cramér-Rao bound. The rotation information of each landmark obtained from the hierarchical procedure is transposed in an additional angular landmark, doubling the number of landmarks in the GEBS model. The modified hierarchical registration using the approximating GEBS model is applied to register 161 image pairs from a digital mammogram database. The obtained results are very encouraging, and the proposed approach significantly improved all registrations comparing the mean-square error in relation to approximating TPS with the rotation information. On artificially deformed breast images, the newly proposed method performed better than the state-of-the-art registration algorithm introduced by Rueckert et al. (IEEE Trans Med Imaging 18:712-721, 1999). The average error per breast tissue pixel was less than 2.23 pixels compared to 2.46 pixels for Rueckert's method. The proposed hierarchical elastic image registration approach incorporates the GEBS approximation scheme extended with anisotropic landmark localization uncertainties as well as rotation information. Our experimental results show that the proposed scheme improved the registration result significantly.

REGION BASED MEDICAL IMAGE REGISTRATION APPROACH USING A MODIFIED ARTIFICIAL IMMUNE SYSTEM

A high accuracy medical image registration is needed to achieve an efficient medical diagnosis and computer aided surgeries, so that a novel region based medical image registration approach is presented in this paper. This approach consists of four sequential steps, image segmentation, point correspondence using a modified artificial immune system, point selection and finally LMS technique is used to estimate the warp parameters. The proposed approach provides a high accuracy and it doesn’t require any additional treatment or feature extraction as some other methods do. To demonstrate the effectiveness of the proposed approach it tested for many image pairs and it is compared with many registration algorithms such aspoint-based image registration using an artificial immune system (AIS),ICP and RANSAC algorithms.

Image Processing for Automated Flaw Detection and CMYK model for Color Image Segmentation using Type 2 Fuzzy Sets

Infrared (IR) thermography has evolved in recent years from being an emerging nondestructive testing (NDT) technique to a viable approach for both aerospace manufacturing and in-service inspections. One of the drawbacks of thermography techniques is that no standard signal processing has been universally adopted and that different algorithms yield different sizing results. Additionally, the data interpretation is not as simple as with other NDT techniques. In this paper the most common signal processing techniques applied to pulsed thermography, which include derivative processing, pulsed phase thermography and principal component analysis, are applied in an attempt to simplify the damage detection and sizing process. The pulsed thermography experiments were carried out on 25 impacted panels made of carbon fiber epoxy material. Despite using similar panels and the same experiment parameters, the damage detection and sizing processes are not straight forward. It is concluded that some algorithms provide easier detection capability than others. However, on their own, the different algorithms lack the robustness to make the damage detection and sizing processes reliable and fully automated. And Optimization of the similarity measure is an essential theme in medical image registration. In this paper, a novel continuous medical image registration approach (CMIR) is proposed. This is our extension work of the previous one where we did a segmentation part of any particular image with a custom algorithm .The CMIR, considering the feedback from users and their preferences on the trade-off between global registration and local registration, extracts the concerned region by user interaction and continuously optimizing the registration result. Experiment results show that CMIR is robust, and more effective compared with the basic optimization algorithm. Image registration, as a precondition of image fusion, has been a critical technique in clinical diagnosis. It can be classified into global registration and local registration. Global registration is used most frequently, which could give a good approximation in most cases and do not need to determine many parameters. Local registration can give detailed information about the concerned regions, which is the critical region in the image. Finding the maximum of the similarity measure is an essential problem in medical image registration. Our work is concentrating on that particular section with the synergy of Tpe-2 fuzzy logic invoked in it.

Non-rigid image registration of brain magnetic resonance images using graph-cuts

We present a graph-cuts based method for non-rigid medical image registration on brain magnetic resonance images. In this paper, the non-rigid medical image registration problem is reformulated as a discrete labeling problem. Based on a voxel-to-voxel intensity similarity measure, each voxel in the source image is assigned a displacement label, which represents a displacement vector indicating which position in the floating image it is spatially corresponding to. In the proposed method, a smoothness constraint based on the first derivative is used to penalize sharp changes in the adjacent displacement labels across voxels. The image registration problem is therefore modeled by two energy terms based on intensity similarity and smoothness of the displacement field. These energy terms are submodular and can be optimized by using the graph-cuts method via α‐expansions, which is a powerful combinatorial optimization tool and capable of yielding either a global minimum or a local minimum in a strong sense. Using the realistic brain phantoms obtained from the Simulated Brain Database, we compare the registration results of the proposed method with two state-of-the-art medical image registration approaches: free-form deformation based method and demons method. In addition, the registration results are also compared with that of the linear programming based image registration method. It is found that the proposed method is more robust against different challenging non-rigid registration cases with consistently higher registration accuracy than those three methods, and gives realistic recovered deformation fields.

Image registration using modified adaptive polar transform

Abstract Image registration is of great significance to medicine and remote sensing, so a lot of techniques have been developed within the context of one or the other discipline. This paper proposes an approach for medical image registration using Modified Gabor Wavelet Transform (MGWT) for Modified Adaptive Polar Transform (MAPT). This algorithm can be used to register images of the same or different modalities. This transform analyzes periodic signal components and presents the advantage of being independent of the window length. The performance of the Modified Gabor Wavelet Transform is compared with previous methods like Log Polar Transform and Adaptive Polar Transform. The results show that MGWT outperforms all evaluated model-independent methods with respect to identification accuracy. These results show that the basis of errors produced by the previous methods is the fixed working scale. The new method not only avoids this basis of errors but also makes a tool available for detailed study.

Multimodal image registration using floating regressors in the joint intensity scatter plot

This paper presents a new approach for multimodal medical image registration and compares it to normalized mutual information (NMI) and the correlation ratio (CR). Like NMI and CR, the new method's measure of registration quality is based on the distribution of points in the joint intensity scatter plot (JISP); compact clusters indicate good registration. This method iteratively fits the JISP clusters with regressors (in the form of points and line segments), and uses those regressors to efficiently compute the next motion increment. The result is a striking, dynamic process in which the regressors float around the JISP, tracking groups of points as they contract into tight clusters. One of the method's strengths is that it is intuitive and customizable, offering a multitude of ways to incorporate prior knowledge to guide the registration process. Moreover, the method is adaptive, and can adjust itself to fit data that does not quite match the prior model. Finally, the method is efficiently expandable to higher-dimensional scatter plots, avoiding the "curse of dimensionality" inherent in histogram-based registration methods such as MI and NMI. In two sets of experiments, a simple implementation of the new registration framework is shown to be comparable to (if not superior to) state-of-the-art implementations of NMI and CR in both accuracy and convergence robustness.

AUTOMATIC ELASTIC MEDICAL IMAGE REGISTRATION BASED ON IMAGE INTENSITY

An automatic elastic medical image registration approach is proposed, based on image intensity. The algorithm is divided into two steps. In Step 1, global affine registration is first used to establish an initial guess and the resulting images can be assumed to have only small local elastic deformations. The mapped images are then used as inputs in Step 2, during which, the study image is modeled as elastic sheet by being divided into sub-images. Moving the individual sub-image in the reference image, the local displacement vectors are found and the global elastic transformation is achieved by assimilating all of the local transformation into a continuous transformation. The algorithm has been validated by simulated data, noisy data and clinical tomographic data. Both experiments and theoretical analysis have demonstrated that the proposed algorithm has a superior computational performance and can register images automatically with an improved accuracy.

Novel automatic registration for medical image

As an important integral part of medial image modeling, medical image registration provides unique and necessary information for many clinical applications. In this paper, a novel nonlinear medical image registration approach is proposed. By combining intensity-based registration method with landmark-based registration method, the proposed algorithm can register medical images automatically and accurately. Firstly, the global affine registration method based on image intensity is used to initialize a starting estimation of the robust registration algorithm. Then, by using iterative landmark based algorithm, the affine registration results are registered to further improve the registration accuracy, in which, the novel automatic landmark localization method is introduced and the thin-plate spline is used to refine the registration precision. The clinical MR data are used in our experiments to demonstrate the effectiveness of the proposed algorithm.