3D Non-rigid Registration Research Articles

Robust Fast Inter-Bin Image Registration for Undersampled Coronary MRI Based on a Learned Motion Prior.

To propose a 3D nonrigid registration method that accurately estimates the 3D displacement field from artifact-corrupted Coronary Magnetic Resonance Angiography (CMRA) images. We developed a novel registration framework for registration of artifact-corrupted images based on a 3D U-Net initializer and a deep unrolling network. By leveraging a supervised learning framework with training labels estimated from fully-sampled images, the unrolling network learns a task-specific motion prior which reduces motion estimation biases caused by undersampling artifacts in the source images. We evaluated the proposed method, UNROLL, against an iterative Free-Form Deformation (FFD) registration method and a recently proposed Respiratory Motion Estimation network (RespME-net) for 6-fold (in-distribution) and 11-fold (out-of-distribution) accelerated CMRA. Compared to the baseline methods, UNROLL improved both the accuracy of motion estimation and the quality of motion-compensated CMRA reconstruction at 6-fold acceleration. Furthermore, even at 11-fold acceleration, which was not included during training, UNROLL still generated more accurate displacement fields than the baseline methods. The computational time of UNROLL for the whole 3D volume was only 2 seconds. By incorporating a learned respiratory motion prior, the proposed method achieves highly accurate motion estimation despite the large acceleration. This work introduces a fast and accurate method to estimate the displacement field from low-quality source images. It has the potential to significantly improve the quality of motion-compensated reconstruction for highly accelerated 3D CMRA.

Fast and Robust Non-Rigid Registration Using Accelerated Majorization-Minimization

Non-rigid 3D registration, which deforms a source 3D shape in a non-rigid way to align with a target 3D shape, is a classical problem in computer vision. Such problems can be challenging because of imperfect data (noise, outliers and partial overlap) and high degrees of freedom. Existing methods typically adopt the lp type robust norm to measure the alignment error and regularize the smoothness of deformation, and use a proximal algorithm to solve the resulting non-smooth optimization problem. However, the slow convergence of such algorithms limits their wide applications. In this paper, we propose a formulation for robust non-rigid registration based on a globally smooth robust norm for alignment and regularization, which can effectively handle outliers and partial overlaps. The problem is solved using the majorization-minimization algorithm, which reduces each iteration to a convex quadratic problem with a closed-form solution. We further apply Anderson acceleration to speed up the convergence of the solver, enabling the solver to run efficiently on devices with limited compute capability. Extensive experiments demonstrate the effectiveness of our method for non-rigid alignment between two shapes with outliers and partial overlaps, with quantitative evaluation showing that it outperforms state-of-the-art methods in terms of registration accuracy and computational speed. The source code is available at https://github.com/yaoyx689/AMM_NRR.

An alternately optimized generative adversarial network with texture and content constraints for deformable registration of 3D ultrasound images

Objective. 3D ultrasound non-rigid registration is significant for intraoperative motion compensation. Nevertheless, distorted textures in the registered image due to the poor image quality and low signal-to-noise ratio of ultrasound images reduce the accuracy and efficiency of the existing methods. Approach. A novel 3D ultrasound non-rigid registration objective function with texture and content constraints in both image space and multiscale feature space based on an unsupervised generative adversarial network based registration framework is proposed to eliminate distorted textures. A similarity metric in the image space is formulated based on combining self-structural constraint with intensity to strengthen the robustness to abnormal intensity change compared with common intensity-based metrics. The proposed framework takes two discriminators as feature extractors to formulate the texture and content similarity between the registered image and the fixed image in the multiscale feature space respectively. A distinctive alternating training strategy is established to jointly optimize the combination of various similarity loss functions to overcome the difficulty and instability of training convergence and balance the training of generator and discriminators. Main results. Compared with five registration methods, the proposed method is evaluated both with small and large deformations, and achieves the best registration accuracy with average target registration error of 1.089 mm and 2.139 mm in cases of small and large deformations, respectively. The performance on peak signal to noise ratio (PSNR) and structural similarity (SSIM) also proves the effective constraints on distorted textures of the proposed method (PSNR is 31.693 dB and SSIM is 0.9 in the case of small deformation; PSNR is 28.177 dB and SSIM is 0.853 in the case of large deformation). Significance. The proposed 3D ultrasound non-rigid registration method based on texture and content constraints with the distinctive alternating training strategy can eliminate the distorted textures with improving the registration accuracy.

MR-based quantitative measurement of human soft tissue internal strains for pressure ulcer prevention.

Pressure ulcers are a severe disease affecting patients that are bedridden or in a wheelchair bound for long periods of time. These wounds can develop in the deep layers of the skin of specific parts of the body, mostly on heels or sacrum, making them hard to detect in their early stages. Strain levels have been identified as a direct danger indicator for triggering pressure ulcers. Prevention could be possible with the implementation of subject-specific Finite Element (FE) models. However, generation and validation of such FE models is a complex task, and the current implemented techniques offer only a partial solution of the entire problem considering only external displacements and pressures, or cadaveric samples. In this paper, we propose an in vivo solution based on the 3D non-rigid registration between two Magnetic Resonance (MR) images, one in an unloaded configuration and the other deformed by means of a plate or an indenter. From the results of the image registration, the displacement field and subsequent strain maps for the soft tissues were computed. An extensive study, considering different cases (on heel pad and sacrum regions) was performed to evaluate the reproducibility and accuracy of the results obtained with this methodology. The implemented technique can give insight for several applications. It adds a useful tool for better understanding the propagation of deformations in the heel soft tissues that could generate pressure ulcers. This methodology can be used to obtain data on the material properties of the soft tissues to define constitutive laws for FE simulations and finally it offers a promising technique for validating FE models.

Application value of CAIPIRINHA-VIBE with MOCO in liver magnetic resonance examination

ObjectiveTo compare the image quality of VIBE sequence using controlled aliasing in parallel imaging results in higher acceleration (CAIPIRINHA-VIBE) and using generalized autocalibrating partially parallel acquisitions (GRAPPA-VIBE) in liver magnetic resonance examination, and to evaluate the effect of non-rigid 3D-registration motion correction (MOCO) combined with CAIPIRINHA-VIBE on liver spatial location registration. MethodsA total of 85 patients underwent pre-contrast GRAPPA-VIBE and CAIPIRINHA-VIBE breath-hold scan in the mask phase, and then underwent CAIPIRINHA-VIBE breath-hold scan in arterial phase, portal vein phase and delay phase after administration. After the scanning of four phases of CAIPIRINHA-VIBE completed, 3D images without and with MOCO of each phase were automatically generated. The images quality of GRAPPA-VIBE and CAIPIRINHA-VIBE without MOCO in the mask phase was scored subjectively by two physicians. The number of slices at the top of the diaphragm in the arterial phase was taken as the base slice, and that in the other stages subtracted with the base slice for CAIPIRINHA without and with MOCO. The range of diaphragm movement in each phase was counted by + N/- N statistics. ResultsThe image quality and the scores of CAIPIRINHA-VIBE were significantly higher than those of GRAPPA-VIBE in respiratory motion artifact suppression, liver edge sharpness and intrahepatic vascular sharpness (p < 0.05). The spatial position consistency of the liver with MOCO is significantly better than that without MOCO. ConclusionCAIPIRINHA-VIBE with MOCO can be used instead of conventional GRAPPA-VIBE sequence in upper abdominal MRI enhancement examination, especially for patients with poor breath-hold.

Optimizing early cancer diagnosis and detection using a temporal subtraction technique

To optimize the early diagnosis and detection of lung cancer, computer-aided diagnostic (CAD) systems have been a useful tool for analyzing medical images. The temporal subtraction technique, which is a CAD system, performs the subtraction operation between the current image and the previous image on the same patient, and supports observation by emphasizing the temporal changes. However, the temporal subtraction technique for 3D images, such as thoracic CT images, has not yet been established. There is a need to develop efficient and highly accurate 3D nonrigid registration techniques to reduce subtraction artifacts. This study aims to develop a 3D nonrigid registration technique to establish a 3D temporal subtraction technique. In particular, we focus on the Finite Element Method, which is versatile, applicable to a wide range of fields, and capable of handling any shape. Our new method was examined on 46 clinical cases with multidetector row computed tomography images. As a result, the proposed method improved by 6.93% (p = 3.0 × 10−6) compared to the conventional methods in terms of the rate of reduction of artifacts, and the effectiveness was verified. Therefore, this study contributes to the literature on early detection and treatment.

Finite-Element Based Image Registration for Endovascular Aortic Aneurysm Repair

In this paper we introduce a new method for the registration between preoperative and intraoperative computerized tomography (CT) images used in endovascular interventions for aortic aneurysm repair. The method relies on a 3D finite-element model (FEM) of the aortic centerline reconstructed from preoperative CT scans. Intraoperative 2D fluoroscopic images are used to deform the 3D FEM and align it onto the current aortic geometry. The method was evaluated on clinical datasets for which a reference CT scan was available to evaluate the registration errors made by our method and to compare them with other registration methods based on rigid transformations. Errors were estimated based on the predicted locations of landmarks positioned at different branch ostia. It appeared that our method always reduced the registration errors of at least 20% compared to gold standard 3D rigid registration and permitted to reach a global precision of 3.8 mm and a renal precision of 2.6 mm, which is a significant improvement compatible with surgical specifications. Finally, the major asset of our method is that it only requires one fluoroscopic intraoperative 2D image to perform the 3D non-rigid registration. This would reduce patient irradiation and cut the costs compared to traditional methods.

Global 3D Non-Rigid Registration of Deformable Objects Using a Single RGB-D Camera.

We present a novel global non-rigid registration method for dynamic 3D objects. Our method allows objects to undergo large non-rigid deformations and achieves high-quality results even with substantial pose change or camera motion between views. In addition, our method does not require a template prior and uses less raw data than tracking-based methods since only a sparse set of scans is needed. We simultaneously compute the deformations of all the scans by optimizing a global alignment problem to avoid the well-known loop closure problem and use an as-rigid-as-possible constraint to eliminate the shrinkage problem of the deformed shapes, especially near open boundaries of scans. To cope with large-scale problems, we design a coarse-to-fine multi-resolution scheme, which also avoids the optimization being trapped into local minima. The proposed method is evaluated on public datasets and real datasets captured by an RGB-D sensor. The experimental results demonstrate that the proposed method obtains better results than several state-of-the-art methods.

3D non-rigid registration using color: Color Coherent Point Drift

Research into object deformations using computer vision techniques has been under intense study in recent years. A widely used technique is 3D non-rigid registration to estimate the transformation between two instances of a deforming structure. Despite many previous developments on this topic, it remains a challenging problem. In this paper we propose a novel approach to non-rigid registration combining two data spaces in order to robustly calculate the correspondences and transformation between two data sets. In particular, we use point color as well as 3D location as these are the common outputs of RGB-D cameras. We have propose the Color Coherent Point Drift (CCPD) algorithm (an extension of the CPD method (Myronenko and Song, 2010)). Evaluation is performed using synthetic and real data. The synthetic data includes easy shapes that allow evaluation of the effect of noise, outliers and missing data. Moreover, an evaluation of realistic figures obtained using Blensor is carried out. Real data acquired using a general purpose Primesense Carmine sensor is used to validate the CCPD for real shapes. For all tests, the proposed method is compared to the original CPD showing better results in registration accuracy in most cases.

3D non-rigid image registration with inverse consistency constraint on elastodynamics wave equation

In this work, a new inverse consistent non-rigid image registration method is presented. The inter-subject deformations are modeled as elastic waves which tend to propagate over the image domain while recovering the nonlinear discrepancies between the two images. An inverse consistency constraint is introduced into the inertial force that is part of the elastodynamics wave equation which governs the underlying non-rigid deformations. The proposed registration method was analyzed over 3D MR brain scans both quantitatively and qualitatively. Normalized cross-correlation (NCC) was utilized to check the registration accuracy, and it revealed that the performance of proposed registration method with and without inverse consistency constraint is comparable in terms of NCC which increased by $$13\%$$ from its initial value. Moreover, Dice coefficient for segmented structures was computed and compared against state-of-the-art symmetric image normalization method, free form deformation and diffeomorphic demons registration methods. The extent to which the proposed registration scheme enforced inverse consistency was analyzed through inverse consistency error. The results showed that the inverse consistency error reduced by $$99\%$$ with the proposed inverse consistent registration method as compared to the inverse inconsistent counterpart.

Subtraction quality assessment of hepatic dynamic contrast enhanced MRI by using non-rigid 3D-registration for subtraction technique

Subtraction quality assessment of hepatic dynamic contrast enhanced MRI by using non-rigid 3D-registration for subtraction technique

Microscopic validation of whole mouse micro-metastatic tumor imaging agents using cryo-imaging and sliding organ image registration.

We created a metastasis imaging, analysis platform consisting of software and multi-spectral cryo-imaging system suitable for evaluating emerging imaging agents targeting micro-metastatic tumor. We analyzed CREKA-Gd in MRI, followed by cryo-imaging which repeatedly sectioned and tiled microscope images of the tissue block face, providing anatomical bright field and molecular fluorescence, enabling 3D microscopic imaging of the entire mouse with single metastatic cell sensitivity. To register MRI volumes to the cryo bright field reference, we used our standard mutual information, non-rigid registration which proceeded: preprocess → affine → B-spline non-rigid 3D registration. In this report, we created two modified approaches: mask where we registered locally over a smaller rectangular solid, and sliding organ. Briefly, in sliding organ, we segmented the organ, registered the organ and body volumes separately and combined results. Though sliding organ required manual annotation, it provided the best result as a standard to measure other registration methods. Regularization parameters for standard and mask methods were optimized in a grid search. Evaluations consisted of DICE, and visual scoring of a checkerboard display. Standard had accuracy of 2 voxels in all regions except near the kidney, where there were 5 voxels sliding. After mask and sliding organ correction, kidneys sliding were within 2 voxels, and Dice overlap increased 4%-10% in mask compared to standard. Mask generated comparable results with sliding organ and allowed a semi-automatic process.

Evaluation of sampling method effects in 3D non-rigid registration

Since the beginning of 3D computer vision problems, the use of techniques to reduce the data to make it treatable preserving the important aspects of the scene has been necessary. Currently, with the new low-cost RGB-D sensors, which provide a stream of color and 3D data of approximately 30 frames per second, this is getting more relevance. Many applications make use of these sensors and need a preprocessing to downsample the data in order to either reduce the processing time or improve the data (e.g., reducing noise or enhancing the important features). In this paper, we present a comparison of different downsampling techniques which are based on different principles. Concretely, five different downsampling methods are included: a bilinear-based method, a normal-based, a color-based, a combination of the normal and color-based samplings, and a growing neural gas (GNG)-based approach. For the comparison, two different models have been used acquired with the Blensor software. Moreover, to evaluate the effect of the downsampling in a real application, a 3D non-rigid registration is performed with the data sampled. From the experimentation we can conclude that depending on the purpose of the application some kernels of the sampling methods can improve drastically the results. Bilinear- and GNG-based methods provide homogeneous point clouds, but color-based and normal-based provide datasets with higher density of points in areas with specific features. In the non-rigid application, if a color-based sampled point cloud is used, it is possible to properly register two datasets for cases where intensity data are relevant in the model and outperform the results if only a homogeneous sampling is used.

Brain MRI Tumor Segmentation with 3D Intracranial Structure Deformation Features

Extraction of relevant features is of significant importance for brain tumor segmentation systems. To improve brain tumor segmentation accuracy, the authors present an improved feature extraction component that takes advantage of the correlation between intracranial structure deformation and the compression resulting from brain tumor growth. Using 3D nonrigid registration and deformation modeling techniques, the component measures lateral ventricular (LaV) deformation in volumetric magnetic resonance images. By verifying the location of the extracted LaV deformation feature data and applying the features on brain tumor segmentation with widely used classification algorithms, the authors evaluate the proposed component qualitatively and quantitatively with promising results on 11 datasets comprising real and simulated patient images.

3D Non-rigid Image Registration Algorithm for MR-Guided Microwave Thermocoagulation of Liver Tumors

3D Non-rigid Image Registration Algorithm for MR-Guided Microwave Thermocoagulation of Liver Tumors

3D nonrigid medical image registration using a new information theoretic measure

This work presents a novel method for the nonrigid registration of medical images based on the Arimoto entropy, a generalization of the Shannon entropy. The proposed method employed the Jensen–Arimoto divergence measure as a similarity metric to measure the statistical dependence between medical images. Free-form deformations were adopted as the transformation model and the Parzen window estimation was applied to compute the probability distributions. A penalty term is incorporated into the objective function to smooth the nonrigid transformation. The goal of registration is to optimize an objective function consisting of a dissimilarity term and a penalty term, which would be minimal when two deformed images are perfectly aligned using the limited memory BFGS optimization method, and thus to get the optimal geometric transformation. To validate the performance of the proposed method, experiments on both simulated 3D brain MR images and real 3D thoracic CT data sets were designed and performed on the open source elastix package. For the simulated experiments, the registration errors of 3D brain MR images with various magnitudes of known deformations and different levels of noise were measured. For the real data tests, four data sets of 4D thoracic CT from four patients were selected to assess the registration performance of the method, including ten 3D CT images for each 4D CT data covering an entire respiration cycle. These results were compared with the normalized cross correlation and the mutual information methods and show a slight but true improvement in registration accuracy.

3D Non-Rigid Registration for Abdominal PET-CT and MR Images Using Mutual Information and Independent Component Analysis

The aim of this study is to develop a 3D registration algorithm for positron emission tomography/computed tomography (PET/CT) and magnetic resonance (MR) images acquired from independent PET/CT and MR imaging systems. Combined PET/CT images provide anatomic and functional information, and MR images have high resolution for soft tissue. With the registration technique, the strengths of each modality image can be combined to achieve higher performance in diagnosis and radiotherapy planning. The proposed method consists of two stages: normalized mutual information (NMI)-based global matching and independent component analysis (ICA)-based refinement. In global matching, the field of view of the CT and MR images are adjusted to the same size in the preprocessing step. Then, the target image is geometrically transformed, and the similarities between the two images are measured with NMI. The optimization step updates the transformation parameters to efficiently find the best matched parameter set. In the refinement stage, ICA planes from the windowed image slices are extracted and the similarity between the images is measured to determine the transformation parameters of the control points. B-spline–based freeform deformation is performed for the geometric transformation. The results show good agreement between PET/CT and MR images.

2D–3D Non-rigid Registration Using Depth from Gradient Information

This paper introduces a new registration algorithm for image to object registration. This 2D---3D non-rigid registration is achieved by reconstructing a 3D surface from the given image and then registering it to the target surface. First, we calculate image gradients and then these gradients are used to evaluate depth at pixel-level. Registration is carried out upon the depth data. A transformation field is generated using surface gradients. The source surface is updated using the transformation field until it aligns with the target surface. The key advantage of our approach is that it can also be used to perform 2D-2D as well as 3D-3D registrations. The effectiveness of our approach has been judged by applying the algorithm on various test images. The proposed algorithm can be further used in 3D object recognition.

SU‐E‐J‐237: Real‐Time 3D Anatomy Estimation From Undersampled MR Acquisitions

Recent developments made MRI guided radiotherapy feasible. Performing simultaneous imaging during fractions can provide information about changing anatomy by means of deformable image registration for either immediate plan adaptations or accurate dose accumulation on the changing anatomy. In 3D MRI, however, acquisition time is considerable and scales with resolution. Furthermore, intra‐scan motion degrades image quality.In this work, we investigate the sensitivity of registration quality on imageresolution: potentially, by employing spatial undersampling, the acquisition timeof MR images for the purpose of deformable image registration can be reducedsignificantly.On a volunteer, 3D‐MR imaging data was sampled in a navigator‐gated manner, acquiring one axial volume (360×260×100mm3) per 3s during exhale phase. A T1‐weighted FFE sequence was used with an acquired voxel size of (2.5mm3) for a duration of 17min. Deformation vector fields were evaluated for 100 imaging cycles with respect to the initial anatomy using deformable image registration based on optical flow. Subsequently, the imaging data was downsampled by a factor of 2, simulating a fourfold acquisition speed. Displacements of the downsampled volumes were then calculated by the same process.In kidneyliver boundaries and the region around stomach/duodenum, prominent organ drifts could be observed in both the original and the downsampled imaging data. An increasing displacement of approximately 2mm was observed for the kidney, while an area around the stomach showed sudden displacements of 4mm. Comparison of the motile points over time showed high reproducibility between the displacements of high‐resolution and downsampled volumes: over a 17min acquisition, the componentwise RMS error was not more than 0.38mm.Based on the synthetic experiments, 3D nonrigid image registration shows little sensitivity to image resolution and the displacement information is preserved even when halving the resolution. This can be employed to greatly reduce image acquisition times for interventional applications in real‐time.This work was funded by the SoRTS consortium, which includes the industry partners Elekta, Philips and Technolution

3D volumetric displacement and strain analysis of composite polymerization

ObjectiveThe present study aimed at a better understanding of the internal shrinkage patterns within different cavity sizes. MethodsTen cylindrical cavities in two sizes were filled with a flowable composite and scanned using X-ray micro-computed tomography (μ-CT) before filling, before and after polymerization. Three-dimensional (3D) non-rigid image registration was applied to sets of two subsequent μ-CT images, before and after polymerization in order to calculate the displacements and strains caused by polymerization shrinkage. Results3D volumetric displacement analysis disclosed a main vertical component for both the small and large cavities, however in the latter the downward direction reversed to an upward direction from a depth of approximately 2mm due to debonding at the bottom. Air bubbles and voids in the restorations increased upon polymerization, causing a reverse in strain in the surrounding areas. SignificancePolymerization-induced shrinkage stress in composite restorations cannot be measured directly. This exploratory study revealed more information on cavity-size dependent shrinkage patterns and opens the way to more extensive studies using different composite materials and varying geometric cavity configurations.