Non-rigid Image Registration Method Research Articles

Left ventricular myocardial motion tracking in cardiac cine magnetic resonance images based on a biomechanical model.

Cardiac cine magnetic resonance (CCMR) imaging plays an important role in the clinical cardiovascular disease (CVD) examination and evaluation. To accurately reconstruct the displacement field and describe the motion of the left ventricular myocardium (LVM), this study proposes and tests a new approach for tracking myocardial motion of the left ventricle based on a biomechanical model. CCMR imaging data acquired from 103 patients are enrolled, including two simulated and 101 clinical data. A non-rigid image registration method with a combination of a thin-plate spline function and random sample consensus is used to recover the observed displacement field of LVM. Next, a biomechanical model and a material matrix are introduced to solve the dense displacement field of LVM using a finite element framework. Then, the tracking precision and error of results for the two groups are analyzed. Displacement results of the simulated data show correlation coefficient≥0.876 and mean square error≤0.159, while displacement results of the clinical data show Dice≥0.97 and mean contour distance≤0.464. Additionally, the strain results show correlation coefficient≥0.717. This study demonstrates that the proposed new method enables to accurately track the motion of the LVM and evaluate strain, which has clinical auxiliary value in the diagnosis of CVD.

Semi-Supervised Deep Learning-Based Image Registration Method with Volume Penalty for Real-Time Breast Tumor Bed Localization.

Breast-conserving surgery requires supportive radiotherapy to prevent cancer recurrence. However, the task of localizing the tumor bed to be irradiated is not trivial. The automatic image registration could significantly aid the tumor bed localization and lower the radiation dose delivered to the surrounding healthy tissues. This study proposes a novel image registration method dedicated to breast tumor bed localization addressing the problem of missing data due to tumor resection that may be applied to real-time radiotherapy planning. We propose a deep learning-based nonrigid image registration method based on a modified U-Net architecture. The algorithm works simultaneously on several image resolutions to handle large deformations. Moreover, we propose a dedicated volume penalty that introduces the medical knowledge about tumor resection into the registration process. The proposed method may be useful for improving real-time radiation therapy planning after the tumor resection and, thus, lower the surrounding healthy tissues’ irradiation. The data used in this study consist of 30 computed tomography scans acquired in patients with diagnosed breast cancer, before and after tumor surgery. The method is evaluated using the target registration error between manually annotated landmarks, the ratio of tumor volume, and the subjective visual assessment. We compare the proposed method to several other approaches and show that both the multilevel approach and the volume regularization improve the registration results. The mean target registration error is below 6.5 mm, and the relative volume ratio is close to zero. The registration time below 1 s enables the real-time processing. These results show improvements compared to the classical, iterative methods or other learning-based approaches that do not introduce the knowledge about tumor resection into the registration process. In future research, we plan to propose a method dedicated to automatic localization of missing regions that may be used to automatically segment tumors in the source image and scars in the target image.

Non-rigid registration of biomedical image for radiotherapy based on adaptive feature density flow

This paper presents a non-rigid biomedical image registration method based on Adaptive feature density flow (AFDF). Considering the muhiplicative characteristics of metadata signal for instance Magnetic Resonance Imaging (MRI) with high-resolution and high-sensitivity, the Non-Local Means Filtering Adapted to Rician Noise (NLMr) is employed to remove Rician noise rather than Gaussian noise with higher Peak Signal to Noise Ratio (PSNR). The deformation information and local shape information are homogeneous, so a flexible dimension of Fast Local Self-Similarity (FLSS) feature descriptor can be constructed to obtain adaptive FLSS (AFLSS) feature descriptor with adaptive Angle and Radial intervals. On the basis of AFLSS, the Adaptive feature density flow (AFDF) is proposed by calculating the dense feature flow field via Belief Propagation algorithm and thereby further optimized by the Pyramid iterative method. Finally, the image registration is completed by using the bicubic interpolation algorithm according to the dense feature flow field. Aiming at radiotherapy positioning control through precise robust registration of biomedical images, experiments of human brain MR images are carried out. The comparative results with PSNR, Root Mean Square Error (RMSE) and normalize mutual information (NMI) show that AFDF performs better than the optical flow method and Scale Invariant Feature Transform (SIFT) flow method, especially for the biomedical scenario with unconscious creep deformation.

Multistep, automatic and nonrigid image registration method for histology samples acquired using multiple stains

The use of multiple dyes during histological sample preparation can reveal distinct tissue properties. However, since the slide preparation differs for each dye, the tissue slides are being deformed and a nonrigid registration is required before further processing. The registration of histology images is complicated because of: (i) a high resolution of histology images, (ii) complex, large, nonrigid deformations, (iii) difference in the appearance and partially missing data due to the use of multiple dyes. In this work, we propose a multistep, automatic, nonrigid image registration method dedicated to histology samples acquired with multiple stains. The proposed method consists of a feature-based affine registration, an exhaustive rotation alignment, an iterative, intensity-based affine registration, and a nonrigid alignment based on modality independent neighbourhood descriptor coupled with the Demons algorithm. A dedicated failure detection mechanism is proposed to make the method fully automatic, without the necessity of any manual interaction. The described method was proposed by the AGH team during the Automatic Non-rigid Histological Image Registration (ANHIR) challenge. The ANHIR dataset consists of 481 image pairs annotated by histology experts. Moreover, the ANHIR challenge submissions were evaluated using an independent, server-side evaluation tool. The main evaluation criteria was the target registration error normalized by the image diagonal. The median of median target registration error is below 0.19%. The proposed method is currently the second-best in terms of the average ranking of median target registration error, without statistically significant differences compared to the top-ranked method. We provide an open access to the method software and used parameters, making the results fully reproducible.

Multiple Image Features-Based Retinal Image Registration Using Global and Local Geometric Structure Constraints

Retinal image registration is a key step in treating hypertension, diabetes and various retinal global diseases. In current methods of retinal image registration, they generally suffer from a lack of reliable features, missing true correspondences and geometric distortion. To address above problem, we propose a robust non-rigid retinal image registration method using multi-image features and dual constraints (i.e. the global and local geometric structure constraints). Our method contains the following main contributions. (i) A finite mixture model based on multi-feature is constructed for handling different types of image features. (ii) A combination of three features is substituted into the mixture model to improve the complementarities of different features. (iii) Dual constraints are proposed for ensuring the stability of the global and local structures of feature sets in the process of spatial transformation and updating. The performance of our method is evaluated by four main types of retinal images, which shows our method outperforms five state-of-the-art methods in most scenarios, especially when the retinal image has a large angle change.

Non-rigid image registration with dynamic Gaussian component density and space curvature preservation.

Image registration plays an important role in military and civilian applications, such as natural disaster damage assessment, environmental monitoring, ground change detection and military damage assessment, etc. This work presents a new feature-based non-rigid image registration method. The main contributions of this work are: (i) a dynamic Gaussian component density is designed to better exploit available potential image information and provide sufficient inlier pairs for image transformation; (ii) a spatial structure preservation, which consists of an image transformation space curvature preservation and a local spatial structure constrain, is proposed to constrain the image transforming cost as well as the local structure of feature points during feature point set registration. The performances of the proposed method in multi-spectral natural images, lowaltitude aerial images and medical images against four types of nine state-of-the-art methods are tested where our method shows the best performances in most scenarios.

A non-rigid image registration method based on multi-level B-spline and L2-regularization

To solve the problem that the cost function of the classic free-form deformation (FFD) cannot simulate transformation field of images with large elastic deformation or local distortion in image registration better, and to increase the registration accuracy and robustness, a new non-rigid image registration method based on the classic hierarchical FFD is proposed. Since the smooth term has a significant influence on registration accuracy, and its coefficient is not easy to be controlled in the classic hierarchical B-spline based FFD, a L2-regularization term with faster and more stable optimization is introduced in the cost function of the proposed model. By coordinating the coefficients of this regularization term and the smooth term, this novel L2-regularized FFD model is able to solve the problem of low registration accuracy caused by strong smooth constraint while maintaining the images topologies. The introduced L2-regularization term can impose a spatial constraint on the control lattices transformation field, and the over-registration problem can be suppressed to a certain extent, so it can register the images with local large distortion. A series of registration experiments of natural images and medical images show that the new method has an obvious advantage over the classic model in registration accuracy measured by mean square error.

A variational approach to non-rigid image registration with Bregman divergences and multiple features

Variational methods register images by minimizing functionals that balance measures of smoothness and image similarity. However, the specificity of image-similarity measures can limit the application of variational methods. To address this issue, we propose a new energy functional that generalizes the classical variational method for nonrigid image registration. Our generalization uses Bregman divergences as the similarity measure of the registration functional. These divergences include a number of important similarity measures, such as the Squared Loss distance, the KL divergence, the Logistic Loss, the Mahalanobis distance, the Itakura–Saito divergence, and the I-divergence. We derive the Euler Lagrange and gradient-flow equations associated with the new functional. By using Bregman divergences, our registration method can combine various types of image characterization as well as spatial and statistical information. Our experiments show that the use of Bregman divergences improves registration quality.

A GPU-based symmetric non-rigid image registration method in human lung.

Quantitative computed tomography (QCT) of the lungs plays an increasing role in identifying sub-phenotypes of pathologies previously lumped into broad categories such as chronic obstructive pulmonary disease and asthma. Methods for image matching and linking multiple lung volumes have proven useful in linking structure to function and in the identification of regional longitudinal changes. Here, we seek to improve the accuracy of image matching via the use of a symmetric multi-level non-rigid registration employing an inverse consistent (IC) transformation whereby images are registered both in the forward and reverse directions. To develop the symmetric method, two similarity measures, the sum of squared intensity difference (SSD) and the sum of squared tissue volume difference (SSTVD), were used. The method is based on a novel generic mathematical framework to include forward and backward transformations, simultaneously, eliminating the need to compute the inverse transformation. Two implementations were used to assess the proposed method: a two-dimensional (2-D) implementation using synthetic examples with SSD, and a multi-core CPU and graphics processing unit (GPU) implementation with SSTVD for three-dimensional (3-D) human lung datasets (six normal adults studied at total lung capacity (TLC) and functional residual capacity (FRC)). Success was evaluated in terms of the IC transformation consistency serving to link TLC to FRC. 2-D registration on synthetic images, using both symmetric and non-symmetric SSD methods, and comparison of displacement fields showed that the symmetric method gave a symmetrical grid shape and reduced IC errors, with the mean values of IC errors decreased by 37%. Results for both symmetric and non-symmetric transformations of human datasets showed that the symmetric method gave better results for IC errors in all cases, with mean values of IC errors for the symmetric method lower than the non-symmetric methods using both SSD and SSTVD. The GPU version demonstrated an average of 43 times speedup and ~5.2 times speedup over the single-threaded and 12-threaded CPU versions, respectively. Run times with the GPU were as fast as 2min. The symmetric method improved the inverse consistency, aiding the use of image registration in the QCT-based evaluation of the lung.

3D non-rigid image registration with inverse consistency constraint on elastodynamics wave equation

In this work, a new inverse consistent non-rigid image registration method is presented. The inter-subject deformations are modeled as elastic waves which tend to propagate over the image domain while recovering the nonlinear discrepancies between the two images. An inverse consistency constraint is introduced into the inertial force that is part of the elastodynamics wave equation which governs the underlying non-rigid deformations. The proposed registration method was analyzed over 3D MR brain scans both quantitatively and qualitatively. Normalized cross-correlation (NCC) was utilized to check the registration accuracy, and it revealed that the performance of proposed registration method with and without inverse consistency constraint is comparable in terms of NCC which increased by $$13\%$$ from its initial value. Moreover, Dice coefficient for segmented structures was computed and compared against state-of-the-art symmetric image normalization method, free form deformation and diffeomorphic demons registration methods. The extent to which the proposed registration scheme enforced inverse consistency was analyzed through inverse consistency error. The results showed that the inverse consistency error reduced by $$99\%$$ with the proposed inverse consistent registration method as compared to the inverse inconsistent counterpart.

Detecting Regional Stiffness Changes in Aortic Aneurysmal Geometries Using Pressure-Normalized Strain

Transabdominal ultrasound elasticity imaging could improve the assessment of rupture risk for abdominal aortic aneurysms by providing information on the mechanical properties and stress or strain states of vessel walls. We implemented a non-rigid image registration method to visualize the pressure-normalized strain within vascular tissues and adapted it to measure total strain over an entire cardiac cycle. We validated the algorithm's performance with both simulated ultrasound images with known principal strains and anatomically accurate heterogeneous polyvinyl alcohol cryogel vessel phantoms. Patient images of abdominal aortic aneurysm were also used to illustrate the clinical feasibility of our imaging algorithm and the potential value of pressure-normalized strain as a clinical metric. Our results indicated that pressure-normalized strain could be used to identify spatial variations in vessel tissue stiffness. The results of this investigation were sufficiently encouraging to warrant a clinical study measuring abdominal aortic pressure-normalized strain in a patient population with aneurysmal disease.

A hybrid patient-specific biomechanical model based image registration method for the motion estimation of lungs.

This paper presents a new hybrid biomechanical model-based non-rigid image registration method for lung motion estimation. In the proposed method, a patient-specific biomechanical modelling process captures major physically realistic deformations with explicit physical modelling of sliding motion, whilst a subsequent non-rigid image registration process compensates for small residuals. The proposed algorithm was evaluated with 10 4D CT datasets of lung cancer patients. The target registration error (TRE), defined as the Euclidean distance of landmark pairs, was significantly lower with the proposed method (TRE=1.37mm) than with biomechanical modelling (TRE=3.81mm) and intensity-based image registration without specific considerations for sliding motion (TRE=4.57mm). The proposed method achieved a comparable accuracy as several recently developed intensity-based registration algorithms with sliding handling on the same datasets. A detailed comparison on the distributions of TREs with three non-rigid intensity-based algorithms showed that the proposed method performed especially well on estimating the displacement field of lung surface regions (mean TRE=1.33mm, maximum TRE=5.3mm). The effects of biomechanical model parameters (such as Poisson's ratio, friction and tissue heterogeneity) on displacement estimation were investigated. The potential of the algorithm in optimising biomechanical models of lungs through analysing the pattern of displacement compensation from the image registration process has also been demonstrated.

Correction of geometrically distorted underwater images using shift map analysis.

In underwater imaging, water waves cause severe geometric distortions and blurring of the acquired short-exposure images. Corrections for these distortions have been tackled reasonably well by previous efforts but still need improvement in the estimation of pixel shift maps to increase restoration accuracy. This paper presents a new algorithm that efficiently estimates the shift maps from geometrically distorted video sequences and uses those maps to restore the sequences. A nonrigid image registration method is employed to estimate the shift maps of the distorted frames against a reference frame. The sharpest frame of the sequence, determined using a sharpness metric, is chosen as the reference frame. A k-means clustering technique is employed to discard too-blurry frames that could result in inaccuracy in the shift maps' estimation. The estimated pixel shift maps are processed to generate the accurate shift map that is used to dewarp the input frames into their nondistorted forms. The proposed method is applied on several synthetic and real-world video sequences, and the obtained results exhibit significant improvements over the state-of-the-art methods.

Two and Three Dimensional Image Registration Based on B-Spline Composition and Level Sets

AbstractAmethod for non-rigid image registration that is suitable for large deformations is presented. Conventional registration methods embed the image in a B-spline object, and the image is evolved by deforming the B-spline object. In this work, we represent the image using B-spline and deform the image using a composition approach. We also derive a computationally efficient algorithm for calculating the B-spline coefficients and gradients of the image by adopting ideas from signal processing using image filters. We demonstrate the application of our method on several different types of 2D and 3D images and compare it with existing methods.

EVolution: an edge-based variational method for non-rigid multi-modal image registration

Image registration is part of a large variety of medical applications including diagnosis, monitoring disease progression and/or treatment effectiveness and, more recently, therapy guidance. Such applications usually involve several imaging modalities such as ultrasound, computed tomography, positron emission tomography, x-ray or magnetic resonance imaging, either separately or combined.In the current work, we propose a non-rigid multi-modal registration method (namely EVolution: an edge-based variational method for non-rigid multi-modal image registration) that aims at maximizing edge alignment between the images being registered. The proposed algorithm requires only contrasts between physiological tissues, preferably present in both image modalities, and assumes deformable/elastic tissues. Given both is shown to be well suitable for non-rigid co-registration across different image types/contrasts (T1/T2) as well as different modalities (CT/MRI). This is achieved using a variational scheme that provides a fast algorithm with a low number of control parameters. Results obtained on an annotated CT data set were comparable to the ones provided by state-of-the-art multi-modal image registration algorithms, for all tested experimental conditions (image pre-filtering, image intensity variation, noise perturbation). Moreover, we demonstrate that, compared to existing approaches, our method possesses increased robustness to transient structures (i.e. that are only present in some of the images).

DTI Analysis of Presbycusis Using Voxel-Based Analysis.

Presbycusis is the most common sensory deficit in the aging population. A recent study reported using a DTI-based tractography technique to identify a lack of integrity in a portion of the auditory pathway in patients with presbycusis. The aim of our study was to investigate the white matter pathology of patients with presbycusis by using a voxel-based analysis that is highly sensitive to local intensity changes in DTI data. Fifteen patients with presbycusis and 14 age- and sex-matched healthy controls were scanned on a 3T scanner. Fractional anisotropy, mean diffusivity, axial diffusivity, and radial diffusivity were obtained from the DTI data. Intergroup statistics were implemented on these measurements, which were transformed to Montreal Neurological Institute coordinates by using a nonrigid image registration method called large deformation diffeomorphic metric mapping. Increased axial diffusivity, radial diffusivity, and mean diffusivity and decreased fractional anisotropy were found near the right-side hearing-related areas in patients with presbycusis. Increased radial diffusivity and mean diffusivity were also found near a language-related area (Broca area) in patients with presbycusis. Our findings could be important for exploring reliable imaging evidence of presbycusis and could complement an ROI-based approach.

Topology preserving non-rigid image registration using time-varying elasticity model for MRI brain volumes

In this paper, we present a new non-rigid image registration method that imposes a topology preservation constraint on the deformation. We propose to incorporate the time varying elasticity model into the deformable image matching procedure and constrain the Jacobian determinant of the transformation over the entire image domain. The motion of elastic bodies is governed by a hyperbolic partial differential equation, generally termed as elastodynamics wave equation, which we propose to use as a deformation model. We carried out clinical image registration experiments on 3D magnetic resonance brain scans from IBSR database. The results of the proposed registration approach in terms of Kappa index and relative overlap computed over the subcortical structures were compared against the existing topology preserving non-rigid image registration methods and non topology preserving variant of our proposed registration scheme. The Jacobian determinant maps obtained with our proposed registration method were qualitatively and quantitatively analyzed. The results demonstrated that the proposed scheme provides good registration accuracy with smooth transformations, thereby guaranteeing the preservation of topology.

Joint modeling of cell and nuclear shape variation.

Modeling cell shape variation is critical to our understanding of cell biology. Previous work has demonstrated the utility of nonrigid image registration methods for the construction of nonparametric nuclear shape models in which pairwise deformation distances are measured between all shapes and are embedded into a low-dimensional shape space. Using these methods, we explore the relationship between cell shape and nuclear shape. We find that these are frequently dependent on each other and use this as the motivation for the development of combined cell and nuclear shape space models, extending nonparametric cell representations to multiple-component three-dimensional cellular shapes and identifying modes of joint shape variation. We learn a first-order dynamics model to predict cell and nuclear shapes, given shapes at a previous time point. We use this to determine the effects of endogenous protein tags or drugs on the shape dynamics of cell lines and show that tagged C1QBP reduces the correlation between cell and nuclear shape. To reduce the computational cost of learning these models, we demonstrate the ability to reconstruct shape spaces using a fraction of computed pairwise distances. The open-source tools provide a powerful basis for future studies of the molecular basis of cell organization.

Quantification of carotid plaque elasticity and intraplaque neovascularization using contrast-enhanced ultrasound and image registration-based elastography

It is valuable for evaluation of carotid plaque vulnerability to investigate the relation between intraplaque neovascularization (IPN) and plaque elasticity. The contrast-enhanced ultrasound (CEUS) has been used in IPN measurement, but it cannot assess plaque elasticity. The aim of this study was to develop an ultrasound elastography technique based on registration of CEUS sequential images and to use this technique for direct comparison between IPN and plaque elasticity. We employed a nonrigid image registration method using the free-form deformation model to register a pair of clinical CEUS images at systole and diastole. The 2D displacement field of the plaque was estimated and then utilized to calculate the axial and lateral strain distributions within the plaque, from which quantitative strain parameters were obtained. The IPN was measured semiquantitatively with visual assessment and quantitatively with the time–intensity curve analysis and the analysis of contrast agent spatial distributions. Histopathology with CD34 staining for quantification of microvessel density (MVD) was performed on plaques excised by carotid endarterectomy. Simulation experiments showed that the mean absolute error and the root mean squared error of the displacement estimation were 0.325±0.180 pixel (7.2%±3.8%) and 0.556±0.284 pixel (12.3%±6.1%), respectively, demonstrating high accuracy of the elastography technique. Thirty-eight plaques in 29 patients met the inclusion criteria for the elastography and image analysis, where ten plaques underwent endarterectomy. The 95th percentile (A95) and standard deviation (Asd) of the axial strains exhibited significant differences between the low and high grades of IPN visually assessed (p<0.01). A95 (R=0.579; p<0.001) and Asd (R=0.609; p<0.001) were correlated with the enhanced intensity of plaque, and also correlated with the MVD (R=0.793 and 0.817, respectively; p<0.01), suggesting that plaque became softer and more elastically heterogeneous as IPN increased. These findings provide direct and quantitative evidence for the associations between plaque strains and IPN and might be helpful for evaluation of carotid plaque vulnerability and for plaque risk stratification.

Deformable image registration based on elastodynamics

This paper presents a novel method for 2D inter-subject non-rigid image registration. It is motivated by the ideas derived from elastodynamics which is the subclass of linear elastic theory. We propose to model the non-rigid deformations as elastic waves which are characterized by elastodynamics wave equation. Our registration method is formulated in a multi-resolution manner such that it is able to recover significant deformations. The proposed scheme recovers local (non-linear) as well as global deformations. The results of image registration by the proposed technique were validated via similarity measures for real MR brain images, BrainWeb dataset and MR prostate images. We have also used overlap measures (Jaccard distance, Dice coefficient and Overlap coefficient) over the segmented deep-brain structures including: lateral and third ventricles, putamen and caudate nucleus for real image dataset to further validate our proposed registration scheme. The proposed scheme was also compared with demons, Level set and free-form deformations (cubic B-spline) algorithms implemented in ITK toolkit and symmetric normalization method available in ANTs software package. Our proposed algorithm shows marked improvement of the inter-subject non-rigid registration, both qualitatively and quantitatively, over the four above-mentioned methods. This comparison was performed on all the three types of datasets mentioned above. 2D synthetic experiments comprising of Patch-`C', Square---Rectangle and Circle-`C' were also performed with our algorithm and compared with other four methods. Both qualitative and quantitative results show that our method performs better than the other four methods. Results suggest that our proposed algorithm has improved accuracy over lemons, level set and free-form deformations (ITK implemented) and symmetric normalization method (ANTs based), tested over two types of brain and one prostate MRI. The commonly used synthetic experiments also suggest the superiority of our proposed scheme.