Modality Independent Neighborhood Descriptor Research Articles

Diffeomorphic transformer-based abdomen MRI-CT deformable image registration.

Stereotactic body radiotherapy (SBRT) is a well-established treatment modality for liver metastases in patients unsuitable for surgery. Both CT and MRI are useful during treatment planning for accurate target delineation and to reduce potential organs-at-risk (OAR) toxicity from radiation. MRI-CT deformable image registration (DIR) is required to propagate the contours defined on high-contrast MRI to CT images. An accurate DIR method could lead to more precisely defined treatment volumes and superior OAR sparing on the treatment plan. Therefore, it is beneficial to develop an accurate MRI-CT DIR for liver SBRT. To create a new deep learning model that can estimate the deformation vector field (DVF) for directly registering abdominal MRI-CT images. The proposed method assumed a diffeomorphic deformation. By using topology-preserved deformation features extracted from the probabilistic diffeomorphic registration model, abdominal motion can be accurately obtained and utilized for DVF estimation. The model integrated Swin transformers, which have demonstrated superior performance in motion tracking, into the convolutional neural network (CNN) for deformation feature extraction. The model was optimized using a cross-modality image similarity loss and a surface matching loss. To compute the image loss, a modality-independent neighborhood descriptor (MIND) was used between the deformed MRI and CT images. The surface matching loss was determined by measuring the distance between the warped coordinates of the surfaces of contoured structures on the MRI and CT images. To evaluate the performance of the model, a retrospective study was carried out on a group of 50 liver cases that underwent rigid registration of MRI and CT scans. The deformed MRI image was assessed against the CT image using the target registration error (TRE), Dice similarity coefficient (DSC), and mean surface distance (MSD) between the deformed contours of the MRI image and manual contours of the CT image. When compared to only rigid registration, DIR with the proposed method resulted in an increase of the mean DSC values of the liver and portal vein from 0.850±0.102 and 0.628±0.129 to 0.903±0.044 and 0.763±0.073, a decrease of the mean MSD of the liver from 7.216±4.513mm to 3.232±1.483mm, and a decrease of the TRE from 26.238±2.769mm to 8.492±1.058mm. The proposed DIR method based on a diffeomorphic transformer provides an effective and efficient way to generate an accurate DVF from an MRI-CT image pair of the abdomen. It could be utilized in the current treatment planning workflow for liver SBRT.

Structure-aware independently trained multi-scale registration network for cardiac images.

Image registration is a primary task in various medical image analysis applications. However, cardiac image registration is difficult due to the large non-rigid deformation of the heart and the complex anatomical structure. This paper proposes a structure-aware independently trained multi-scale registration network (SIMReg) to address this challenge. Using image pairs of different resolutions, independently train each registration network to extract image features of large deformation image pairs at different resolutions. In the testing stage, the large deformation registration is decomposed into a multi-scale registration process, and the deformation fields of different resolutions are fused by a step-by-step deformation method, thus solving the difficulty of directly processing large deformation. Meanwhile, the targeted introduction of MIND (modality independent neighborhood descriptor) structural features to guide network training enhances the registration of cardiac structural contours and improves the registration effect of local details. Experiments were carried out on the open cardiac dataset ACDC (automated cardiac diagnosis challenge), and the average Dice value of the experimental results of the proposed method was 0.833. Comparative experiments showed that the proposed SIMReg could better solve the problem of heart image registration and achieve a better registration effect on cardiac images.

MacJNet: weakly-supervised multimodal image deformable registration using joint learning framework and multi-sampling cascaded MIND

Deformable multimodal image registration plays a key role in medical image analysis. It remains a challenge to find accurate dense correspondences between multimodal images due to the significant intensity distortion and the large deformation. macJNet is proposed to align the multimodal medical images, which is a weakly-supervised multimodal image deformable registration method using a joint learning framework and multi-sampling cascaded modality independent neighborhood descriptor (macMIND). The joint learning framework consists of a multimodal image registration network and two segmentation networks. The proposed macMIND is a modality-independent image structure descriptor to provide dense correspondence for registration, which incorporates multi-orientation and multi-scale sampling patterns to build self-similarity context. It greatly enhances the representation ability of cross-modal features in the registration network. The semi-supervised segmentation networks generate anatomical labels to provide semantics correspondence for registration, and the registration network helps to improve the performance of multimodal image segmentation by providing the consistency of anatomical labels. 3D CT-MR liver image dataset with 118 samples is built for evaluation, and comprehensive experiments have been conducted to demonstrate that macJNet achieves superior performance over state-of-the-art multi-modality medical image registration methods.

A transformer-based hierarchical registration framework for multimodality deformable image registration.

Deformable image registration (DIR) between daily and reference images is fundamentally important for adaptive radiotherapy. In the last decade, deep learning-based image registration methods have been developed with faster computation time and improved robustness compared to traditional methods. However, the registration performance is often degraded in extra-cranial sites with large volume containing multiple anatomic regions, such as Computed Tomography (CT)/Magnetic Resonance (MR) images used in head and neck (HN) radiotherapy. In this study, we developed a hierarchical deformable image registration (DIR) framework, Patch-based Registration Network (Patch-RegNet), to improve the accuracy and speed of CT-MR and MR-MR registration for head-and-neck MR-Linac treatments. Patch-RegNet includes three steps: a whole volume global registration, a patch-based local registration, and a patch-based deformable registration. Following a whole-volume rigid registration, the input images were divided into overlapping patches. Then a patch-based rigid registration was applied to achieve accurate local alignment for subsequent DIR. We developed a ViT-Morph model, a combination of a convolutional neural network (CNN) and the Vision Transformer (ViT), for the patch-based DIR. A modality independent neighborhood descriptor was adopted in our model as the similarity metric to account for both inter-modality and intra-modality registration. The CT-MR and MR-MR DIR models were trained with 242 CT-MR and 213 MR-MR image pairs from 36 patients, respectively, and both tested with 24 image pairs (CT-MR and MR-MR) from 6 other patients. The registration performance was evaluated with 7 manually contoured organs (brainstem, spinal cord, mandible, left/right parotids, left/right submandibular glands) by comparing with the traditional registration methods in Monaco treatment planning system and the popular deep learning-based DIR framework, Voxelmorph. Evaluation results show that our method outperformed VoxelMorph by 6 % for CT-MR registration, and 4 % for MR-MR registration based on DSC measurements. Our hierarchical registration framework has been demonstrated achieving significantly improved DIR accuracy of both CT-MR and MR-MR registration for head-and-neck MR-guided adaptive radiotherapy.

Deep rigid registration for slice-to-volume in real time

Slice-to-volume registration that achieves high accuracy in real time is one of the enabling technologies for clinical imaging scenarios. Recently, deep learning methods have been explored to significantly improve the accuracy and efficiency of the registration. Nevertheless, such a 2D/3D registration problem is very challenging due to several considerable barriers including highly computational cost brought by dense sampling in six dimensional parameter space and significant modal appearance differences. We proposed a Differentiable resampling based Slice-to-Volume Registration network, which can achieve real-time and accurate registration in both mono- and multi-modal scenarios. The proposed network learns the out-of-plane transformation parameters in the spherical coordinates that decide the content and is invariant to the remaining in-plane parameters by feeding 2D rigid transformed data on-the-fly, thus reducing the training sample size remarkably. To further improve the performance, we introduce an auxiliary image similarity connected by differentiable resampling. For multi-modal scenarios, we optionally include the enhanced modality independent neighborhood descriptor to unify different modalities into common space. Training size is considerably reduced to 30k per dataset with half of all possible orientations and 80% of the volume space covered while the average registration error drops to 1.84 mm and 6.49 mm respectively for mono- and multi-modal experiments. Extensive experiments show the proposed methods’ superiority over existing deep learning methods for real-time 2D/3D rigid registration in both mono- and multi-modal settings.

Non-rigid multi-modal brain image registration based on two-stage generative adversarial nets

This paper has proposed the two-stage generative adversarial nets (GAN) based multi-modal brain image registration method. The first stage uses the GAN to indirectly recover image deformation and the second stage employs the GAN to directly estimate the registered image. At the first stage, the GAN uses structural representations of reference and float images generated by the improved modality independent neighborhood descriptor (MIND) as inputs, and outputs the predicted structural representation of the registered image. At the second stage, the GAN estimates the intensities of the registered image according to the generated structural representations at the first stage. The two GANs designed based on the U-Net are trained using the different loss functions related to mean square error, mean structural similarity (MSSIM) and the least squares loss. Experiments on the 3D brain images from the Atlas, BrainWeb and RIRE datasets show that compared with state-of-the-art registration methods based on sum of squared differences on entropy images (ESSD), MIND, self-similarity context (SSC), voxelmorph (Morph), volume tweening network (VTN) and adversarial similarity network (ASN), the proposed method has competitive or better registration performance in terms of dice value, MSSIM and computational efficiency.

Evaluation of fully automated myocardial segmentation techniques in native and contrast‐enhanced T1‐mapping cardiovascular magnetic resonance images using fully convolutional neural networks

T1-mapping cardiac magnetic resonance (CMR) imaging permits noninvasive quantification of myocardial fibrosis (MF); however, manual delineation of myocardial boundaries is time-consuming and introduces user-dependent variability for such measurements. In this study, we compare several automated pipelines for myocardial segmentation of the left ventricle (LV) in native and contrast-enhanced T1-maps using fully convolutional neural networks (CNNs). Sixty patients with known MF across three distinct cardiomyopathy states (20 ischemic (ICM), 20 dilated (DCM), and 20 hypertrophic (HCM)) underwent a standard CMR imaging protocol inclusive of cinematic (CINE), late gadolinium enhancement (LGE), and pre/post-contrast T1 imaging. Native and contrast-enhanced T1-mapping was performed using a shortened modified Look-Locker imaging (shMOLLI) technique at the basal, mid-level, and/or apex of the LV. Myocardial segmentations in native and post-contrast T1-maps were performed using three state-of-the-art CNN-based methods: standard U-Net, densely connected neural networks (Dense Nets), and attention networks (Attention Nets) after dividing the dataset using fivefold cross validation. These direct segmentation techniques were compared to an alternative registration-based segmentation method, wherein spatially corresponding CINE images are segmented automatically using U-Net, and a nonrigid registration technique transforms and propagates CINE contours to the myocardial regions of T1-maps. The methodologies were validated in 125 native and 100 contrast-enhanced T1-maps using standard segmentation accuracy metrics. Pearson correlation coefficient r and Bland-Altman analysis were used to compare the computed global T1 values derived by manual, U-Net, and CINE registration methodologies. The U-Net-based method yielded optimal results in myocardial segmentation of native, contrast-enhanced, and CINE images compared to Dense Nets and Attention Nets. The direct U-Net-based method outperformed the CINE registration-based method in native T1-maps, yielding Dice similarity coefficient (DSC) of 82.7±12% compared to 81.4±6.9% (P<0.0001). However, in contrast-enhanced T1-maps, the CINE-registration-based method outperformed direct U-Net segmentation, yielding DSC of 77.0±9.6% vs 74.2±18% across all patient groups (P=0.0014) and specifically 73.2±7.3% vs 65.5±18% in the ICM patient group. High linear correlation of global T1 values was demonstrated in Pearson analysis of the U-Net-based technique and the CINE-registration technique in both native T1-maps (r=0.93, P<0.0001 and r=0.87, P<0.0001, respectively) and contrast-enhanced T1-maps (r=0.93, P<0.0001 and r=0.98, P<0.0001, respectively). The direct U-Net-based myocardial segmentation technique provided accurate, fully automated segmentations in native and contrast-enhanced T1-maps. Myocardial borders can alternatively be segmented from spatially matched CINE images and applied to T1-maps via deformation and propagation through a modality-independent neighborhood descriptor (MIND). The direct U-Net approach is more efficient in myocardial segmentation of native T1-maps and eliminates cross-technique dependence. However, the CINE-registration-based technique may be more appropriate for contrast-enhanced T1-maps and/or for patients with dense regions of replacement fibrosis, such as those with ICM.

Unsupervised MR-to-CT Synthesis Using Structure-Constrained CycleGAN.

Synthesizing a CT image from an available MR image has recently emerged as a key goal in radiotherapy treatment planning for cancer patients. CycleGANs have achieved promising results on unsupervised MR-to-CT image synthesis; however, because they have no direct constraints between input and synthetic images, cycleGANs do not guarantee structural consistency between these two images. This means that anatomical geometry can be shifted in the synthetic CT images, clearly a highly undesirable outcome in the given application. In this paper, we propose a structure-constrained cycleGAN for unsupervised MR-to-CT synthesis by defining an extra structure-consistency loss based on the modality independent neighborhood descriptor. We also utilize a spectral normalization technique to stabilize the training process and a self-attention module to model the long-range spatial dependencies in the synthetic images. Results on unpaired brain and abdomen MR-to-CT image synthesis show that our method produces better synthetic CT images in both accuracy and visual quality as compared to other unsupervised synthesis methods. We also show that an approximate affine pre-registration for unpaired training data can improve synthesis results.

Deformable MR-CBCT prostate registration using biomechanically constrained deep learning networks.

Radiotherapeutic dose escalation to dominant intraprostatic lesions (DIL) in prostate cancer could potentially improve tumor control. The purpose of this study was to develop a method to accurately register multiparametric magnetic resonance imaging (MRI) with CBCT images for improved DIL delineation, treatment planning, and dose monitoring in prostate radiotherapy. We proposed a novel registration framework which considers biomechanical constraint when deforming the MR to CBCT. The registration framework consists of two segmentation convolutional neural networks (CNN) for MR and CBCT prostate segmentation, and a three-dimensional (3D) point cloud (PC) matching network. Image intensity-based rigid registration was first performed to initialize the alignment between MR and CBCT prostate. The aligned prostates were then meshed into tetrahedron elements to generate volumetric PC representation of the prostate shapes. The 3D PC matching network was developed to predict a PC motion vector field which can deform the MRI prostate PC to match the CBCT prostate PC. To regularize the network's motion prediction with biomechanical constraints, finite element (FE) modeling-generated motion fields were used to train the network. MRI and CBCT images of 50 patients with intraprostatic fiducial markers were used in this study. Registration results were evaluated using three metrics including dice similarity coefficient (DSC), mean surface distance (MSD), and target registration error (TRE). In addition to spatial registration accuracy, Jacobian determinant and strain tensors were calculated to assess the physical fidelity of the deformation field. The mean and standard deviation of our method were 0.93±0.01, 1.66±0.10mm, and 2.68±1.91mm for DSC, MSD, and TRE, respectively. The mean TRE of the proposed method was reduced by 29.1%, 14.3%, and 11.6% as compared to image intensity-based rigid registration, coherent point drifting (CPD) nonrigid surface registration, and modality-independent neighborhood descriptor (MIND) registration, respectively. We developed a new framework to accurately register the prostate on MRI to CBCT images for external beam radiotherapy. The proposed method could be used to aid DIL delineation on CBCT, treatment planning, dose escalation to DIL, and dose monitoring.

MINDFLOW BASED DENSE MATCHING BETWEEN TIR AND RGB IMAGES

Abstract. Image registration is a fundamental issue in photogrammetry and remote sensing, which targets to find the alignment between different images. Recently, registration of images from difference sensors become the hot topic. The registered images from different sensors are able to offer additional information, which help with different tasks like segmentation, classification, and even emergency analysis. In this paper, we proposed a registration strategy to calculate the dominant orientation difference and then achieve the dense alignment of Thermal Infrared (TIR) image and RGB image with MINDflow. Firstly, the orientation difference of TIR images and RGB images is calculated by finding the dominant image orientations based on phase congruency. Then, the modality independent neighborhood descriptor (MIND) together with global optical flow algorithm are adopted as MINDflow for dense matching. Our method is tested in the image sets containing TIR images and RGB images captured separately but in the same construction site areas. The results show that it is able to achieve the optimal results with features of significance even for dramatically radiometric differences between TIR images and RGB images. By comparing the results with other descriptor, our method is more robust and keep the features of objects in the images.

Selection of a Similarity Measure Combination for a Wide Range of Multimodal Image Registration Cases

Many similarity measures (SMs) were proposed to measure the similarity between multimodal remote sensing (RS) images. Each SM is efficient to a different degree in different registration cases (we consider visible-to-infrared, visible-to-radar, visible-to-digital elevation model (DEM), and radar-to-DEM ones), but no SM was shown to outperform all other SMs in all cases. In this article, we investigate the possibility of deriving a more powerful SM by combining two or more existing SMs. This combined SM relies on a binary linear support vector machine (SVM) classifier trained using real RS images. In the general registration case, we order SMs according to their impact on the combined SM performance. The three most important SMs include two structural SMs based on modality independent neighborhood descriptor (MIND) and scale-invariant feature transform-octave (SIFT-OCT) descriptors and one area-based logarithmic likelihood ratio (logLR) SM: the former ones are more robust to structural changes of image intensity between registered modes, the latter one is to image noise. Importantly, we demonstrate that a single combined SM can be applied in the general case as well as in each particular considered registration case. As compared to existing multimodal SMs, the proposed combined SM [based on five existing SMs, namely, MIND, logLR, SIFT-OCT, phase correlation (PC), histogram of orientated phase congruency (HOPC)] increases the area under the curve (AUC) by from 1% to 21%. From a practical point of view, we demonstrate that complex multimodal image pairs can be successfully registered with the proposed combined SM, while existing single SMs fail to detect enough correspondences for registration. Our results demonstrate that MIND, SIFT, and logLR SMs capture essential aspects of the similarity between RS modes, and their properties are complementary for designing a new more efficient multimodal SM.

Non-Rigid Multi-Modal 3D Medical Image Registration Based on Foveated Modality Independent Neighborhood Descriptor.

The non-rigid multi-modal three-dimensional (3D) medical image registration is highly challenging due to the difficulty in the construction of similarity measure and the solution of non-rigid transformation parameters. A novel structural representation based registration method is proposed to address these problems. Firstly, an improved modality independent neighborhood descriptor (MIND) that is based on the foveated nonlocal self-similarity is designed for the effective structural representations of 3D medical images to transform multi-modal image registration into mono-modal one. The sum of absolute differences between structural representations is computed as the similarity measure. Subsequently, the foveated MIND based spatial constraint is introduced into the Markov random field (MRF) optimization to reduce the number of transformation parameters and restrict the calculation of the energy function in the image region involving non-rigid deformation. Finally, the accurate and efficient 3D medical image registration is realized by minimizing the similarity measure based MRF energy function. Extensive experiments on 3D positron emission tomography (PET), computed tomography (CT), T1, T2, and (proton density) PD weighted magnetic resonance (MR) images with synthetic deformation demonstrate that the proposed method has higher computational efficiency and registration accuracy in terms of target registration error (TRE) than the registration methods that are based on the hybrid L-BFGS-B and cat swarm optimization (HLCSO), the sum of squared differences on entropy images, the MIND, and the self-similarity context (SSC) descriptor, except that it provides slightly bigger TRE than the HLCSO for CT-PET image registration. Experiments on real MR and ultrasound images with unknown deformation have also be done to demonstrate the practicality and superiority of the proposed method.

PCANet-Based Structural Representation for Nonrigid Multimodal Medical Image Registration.

Nonrigid multimodal image registration remains a challenging task in medical image processing and analysis. The structural representation (SR)-based registration methods have attracted much attention recently. However, the existing SR methods cannot provide satisfactory registration accuracy due to the utilization of hand-designed features for structural representation. To address this problem, the structural representation method based on the improved version of the simple deep learning network named PCANet is proposed for medical image registration. In the proposed method, PCANet is firstly trained on numerous medical images to learn convolution kernels for this network. Then, a pair of input medical images to be registered is processed by the learned PCANet. The features extracted by various layers in the PCANet are fused to produce multilevel features. The structural representation images are constructed for two input images based on nonlinear transformation of these multilevel features. The Euclidean distance between structural representation images is calculated and used as the similarity metrics. The objective function defined by the similarity metrics is optimized by L-BFGS method to obtain parameters of the free-form deformation (FFD) model. Extensive experiments on simulated and real multimodal image datasets show that compared with the state-of-the-art registration methods, such as modality-independent neighborhood descriptor (MIND), normalized mutual information (NMI), Weber local descriptor (WLD), and the sum of squared differences on entropy images (ESSD), the proposed method provides better registration performance in terms of target registration error (TRE) and subjective human vision.

Self-similarity inspired local descriptor for non-rigid multi-modal image registration

Non-rigid multi-modal image registration plays an important role in medical image analysis. It remains a challenging problem due to the significant intensity distortion and the non-rigid transformation between images. Existing registration methods based on the information theoretic measures and image representations cannot address this problem effectively. In this paper, we have proposed a novel self-similarity inspired local descriptor to determine the similarity metrics, a key component in image registration. The self-similarity is determined based on the Zernike moments of image patches in a local neighborhood, and it is utilized to generate the local descriptor. The Euclidean distance between the local descriptors computed in the reference and moving images is used as the similarity metrics. Distinctively, the proposed local descriptor can provide an effective representation of complicated image features due to its robustness to noise and rotational invariance, which provides the proposed method with good registration performance. Extensive experiments on both simulated and real multi-modal image datasets demonstrate that the proposed method has higher registration accuracy appreciated by the target registration error (TRE) than the state-of-the-art registration methods based on the normalized mutual information (NMI), the sum of squared differences on entropy images (ESSD), the Weber local descriptor (WLD) and the modality independent neighborhood descriptor (MIND).

Local structure orientation descriptor based on intra-image similarity for multimodal registration of liver ultrasound and MR images

PurposeUltrasound (US)-magnetic resonance (MR) fusion imaging is a profitable tool for image-guided abdominal diagnosis and biopsy. However, the automatic registration of liver US and MR images remains a challenging task. An effective local structure orientation descriptor (LSOD) for use in registering multimodal images is proposed in this study. MethodsLSOD utilizes a normalized similarity distance vector of intra-image patch pairs to extract intensity change orientations from intensity value changes in a local area. The multimodal similarity measure is then derived using the LSOD vector difference. Experiments were performed on simulated US and liver 2D US–3D MR images from a phantom, two healthy volunteers, and seven patients. ResultsUsing the LSOD-based method, the root-mean-square target registration errors (RMS-TREs) were 1.76±1.90mm/2.03±0.84mm in phantom/clinical experiments. All of the results outperformed those obtained using modality independent neighborhood descriptor (MIND)- and linear correlation of linear combination (LC2)-based methods (phantom/clinical: 5.23±3.35mm/4.32±3.63mm and 9.79±5.03mm/6.29±3.85mm, respectively). The registration cover range for all subjects of the LSOD-based method was 9.16mm, which was larger than those of the MIND- and LC2-based methods (5.06 and 5.12mm, respectively). ConclusionsThe results demonstrated that the LSOD-based registration method could robustly register 2D US and 3D MR images of different liver sections with acceptable accuracy for clinical requirements. This approach is useful for the practical clinical application of the US–MR fusion imaging technique.

MIND Demons: Symmetric Diffeomorphic Deformable Registration of MR and CT for Image-Guided Spine Surgery.

Intraoperative localization of target anatomy and critical structures defined in preoperative MR/CT images can be achieved through the use of multimodality deformable registration. We propose a symmetric diffeomorphic deformable registration algorithm incorporating a modality-independent neighborhood descriptor (MIND) and a robust Huber metric for MR-to-CT registration. The method, called MIND Demons, finds a deformation field between two images by optimizing an energy functional that incorporates both the forward and inverse deformations, smoothness on the integrated velocity fields, a modality-insensitive similarity function suitable to multimodality images, and smoothness on the diffeomorphisms themselves. Direct optimization without relying on the exponential map and stationary velocity field approximation used in conventional diffeomorphic Demons is carried out using a Gauss-Newton method for fast convergence. Registration performance and sensitivity to registration parameters were analyzed in simulation, phantom experiments, and clinical studies emulating application in image-guided spine surgery, and results were compared to mutual information (MI) free-form deformation (FFD), local MI (LMI) FFD, normalized MI (NMI) Demons, and MIND with a diffusion-based registration method (MIND-elastic). The method yielded sub-voxel invertibility (0.008 mm) and nonzero-positive Jacobian determinants. It also showed improved registration accuracy in comparison to the reference methods, with mean target registration error (TRE) of 1.7 mm compared to 11.3, 3.1, 5.6, and 2.4 mm for MI FFD, LMI FFD, NMI Demons, and MIND-elastic methods, respectively. Validation in clinical studies demonstrated realistic deformations with sub-voxel TRE in cases of cervical, thoracic, and lumbar spine.

MIND Demons for MR-to-CT Deformable Image Registration In Image-Guided Spine Surgery.

Localization of target anatomy and critical structures defined in preoperative MR images can be achieved by means of multi-modality deformable registration to intraoperative CT. We propose a symmetric diffeomorphic deformable registration algorithm incorporating a modality independent neighborhood descriptor (MIND) and a robust Huber metric for MR-to-CT registration. The method, called MIND Demons, solves for the deformation field between two images by optimizing an energy functional that incorporates both the forward and inverse deformations, smoothness on the velocity fields and the diffeomorphisms, a modality-insensitive similarity function suitable to multi-modality images, and constraints on geodesics in Lagrangian coordinates. Direct optimization (without relying on an exponential map of stationary velocity fields used in conventional diffeomorphic Demons) is carried out using a Gauss-Newton method for fast convergence. Registration performance and sensitivity to registration parameters were analyzed in simulation, in phantom experiments, and clinical studies emulating application in image-guided spine surgery, and results were compared to conventional mutual information (MI) free-form deformation (FFD), local MI (LMI) FFD, and normalized MI (NMI) Demons. The method yielded sub-voxel invertibility (0.006 mm) and nonsingular spatial Jacobians with capability to preserve local orientation and topology. It demonstrated improved registration accuracy in comparison to the reference methods, with mean target registration error (TRE) of 1.5 mm compared to 10.9, 2.3, and 4.6 mm for MI FFD, LMI FFD, and NMI Demons methods, respectively. Validation in clinical studies demonstrated realistic deformation with sub-voxel TRE in cases of cervical, thoracic, and lumbar spine. A modality-independent deformable registration method has been developed to estimate a viscoelastic diffeomorphic map between preoperative MR and intraoperative CT. The method yields registration accuracy suitable to application in image-guided spine surgery across a broad range of anatomical sites and modes of deformation.

Towards Personalized Statistical Deformable Model and Hybrid Point Matching for Robust MR-TRUS Registration

Registration and fusion of magnetic resonance (MR) and 3D transrectal ultrasound (TRUS) images of the prostate gland can provide high-quality guidance for prostate interventions. However, accurate MR-TRUS registration remains a challenging task, due to the great intensity variation between two modalities, the lack of intrinsic fiducials within the prostate, the large gland deformation caused by the TRUS probe insertion, and distinctive biomechanical properties in patients and prostate zones. To address these challenges, a personalized model-to-surface registration approach is proposed in this study. The main contributions of this paper can be threefold. First, a new personalized statistical deformable model (PSDM) is proposed with the finite element analysis and the patient-specific tissue parameters measured from the ultrasound elastography. Second, a hybrid point matching method is developed by introducing the modality independent neighborhood descriptor (MIND) to weight the Euclidean distance between points to establish reliable surface point correspondence. Third, the hybrid point matching is further guided by the PSDM for more physically plausible deformation estimation. Eighteen sets of patient data are included to test the efficacy of the proposed method. The experimental results demonstrate that our approach provides more accurate and robust MR-TRUS registration than state-of-the-art methods do. The averaged target registration error is 1.44 mm, which meets the clinical requirement of 1.9 mm for the accurate tumor volume detection. It can be concluded that the presented method can effectively fuse the heterogeneous image information in the elastography, MR, and TRUS to attain satisfactory image alignment performance.

Image Registration Based on Autocorrelation of Local Structure.

Registration of images in the presence of intra-image signal fluctuations is a challenging task. The definition of an appropriate objective function measuring the similarity between the images is crucial for accurate registration. This paper introduces an objective function that embeds local phase features derived from the monogenic signal in the modality independent neighborhood descriptor (MIND). The image similarity relies on the autocorrelation of local structure (ALOST) which has two important properties: 1) low sensitivity to space-variant intensity distortions (e.g., differences in contrast enhancement in MRI); 2) high distinctiveness for 'salient' image features such as edges. The ALOST method is quantitatively compared to the MIND approach based on three different datasets: thoracic CT images, synthetic and real abdominal MR images. The proposed method outperformed the NMI and MIND similarity measures on these three datasets. The registration of dynamic contrast enhanced and post-contrast MR images of patients with Crohn's disease led to relative contrast enhancement measures with the highest correlation (r=0.56) to the Crohn's disease endoscopic index of severity.

Three-Dimensional Nonrigid MR-TRUS Registration Using Dual Optimization.

In this study, we proposed an efficient nonrigid magnetic resonance (MR) to transrectal ultrasound (TRUS) deformable registration method in order to improve the accuracy of targeting suspicious regions during a three dimensional (3-D) TRUS guided prostate biopsy. The proposed deformable registration approach employs the multi-channel modality independent neighborhood descriptor (MIND) as the local similarity feature across the two modalities of MR and TRUS, and a novel and efficient duality-based convex optimization-based algorithmic scheme was introduced to extract the deformations and align the two MIND descriptors. The registration accuracy was evaluated using 20 patient images by calculating the TRE using manually identified corresponding intrinsic fiducials in the whole gland and peripheral zone. Additional performance metrics [Dice similarity coefficient (DSC), mean absolute surface distance (MAD), and maximum absolute surface distance (MAXD)] were also calculated by comparing the MR and TRUS manually segmented prostate surfaces in the registered images. Experimental results showed that the proposed method yielded an overall median TRE of 1.76 mm. The results obtained in terms of DSC showed an average of 80.8±7.8% for the apex of the prostate, 92.0±3.4% for the mid-gland, 81.7±6.4% for the base and 85.7±4.7% for the whole gland. The surface distance calculations showed an overall average of 1.84±0.52 mm for MAD and 6.90±2.07 mm for MAXD.