Intensity-based Similarity Research Articles

WarpPINN: Cine-MR image registration with physics-informed neural networks.

The diagnosis of heart failure usually includes a global functional assessment, such as ejection fraction measured by magnetic resonance imaging. However, these metrics have low discriminate power to distinguish different cardiomyopathies, which may not affect the global function of the heart. Quantifying local deformations in the form of cardiac strain can provide helpful information, but it remains a challenge. In this work, we introduce WarpPINN, a physics-informed neural network to perform image registration to obtain local metrics of heart deformation. We apply this method to cine magnetic resonance images to estimate the motion during the cardiac cycle. We inform our neural network of the near-incompressibility of cardiac tissue by penalizing the Jacobian of the deformation field. The loss function has two components: an intensity-based similarity term between the reference and the warped template images, and a regularizer that represents the hyperelastic behavior of the tissue. The architecture of the neural network allows us to easily compute the strain via automatic differentiation to assess cardiac activity. We use Fourier feature mappings to overcome the spectral bias of neural networks, allowing us to capture discontinuities in the strain field. The algorithm is tested on synthetic examples and on a cine SSFP MRI benchmark of 15 healthy volunteers, where it is trained to learn the deformation mapping of each case. We outperform current methodologies in landmark tracking and provide physiological strain estimations in the radial and circumferential directions. WarpPINN provides precise measurements of local cardiac deformations that can be used for a better diagnosis of heart failure and can be used for general image registration tasks. Source code is available at https://github.com/fsahli/WarpPINN.

Weakly supervised volumetric prostate registration for MRI-TRUS image driven by signed distance map

Image registration is a fundamental step for MRI-TRUS fusion targeted biopsy. Due to the inherent representational differences between these two image modalities, though, intensity-based similarity losses for registration tend to result in poor performance. To mitigate this, comparison of organ segmentations, functioning as a weak proxy measure of image similarity, has been proposed. Segmentations, though, are limited in their information encoding capabilities. Signed distance maps (SDMs), on the other hand, encode these segmentations into a higher dimensional space where shape and boundary information are implicitly captured, and which, in addition, yield high gradients even for slight mismatches, thus preventing vanishing gradients during deep-network training. Based on these advantages, this study proposes a weakly-supervised deep learning volumetric registration approach driven by a mixed loss that operates both on segmentations and their corresponding SDMs, and which is not only robust to outliers, but also encourages optimal global alignment. Our experimental results, performed on a public prostate MRI-TRUS biopsy dataset, demonstrate that our method outperforms other weakly-supervised registration approaches with a dice similarity coefficient (DSC), Hausdorff distance (HD) and mean surface distance (MSD) of 87.3 ± 11.3, 4.56 ± 1.95 mm, and 0.053 ± 0.026 mm, respectively. We also show that the proposed method effectively preserves the prostate gland's internal structure.

Local similarity of activity patterns during auditory and visual processing

Neuroimaging studies have shown that brain activity is variable and changes according to stimuli and the environmental context, reflecting brain coding or information representations at different processing levels. However, little is known about activity organization that reflects coding strategies. Here, we explored and compared two different coding approaches, spatial via cross-correlation and intensity-based coding using mutual information. Using two fMRI datasets and different seeds, we searched for the spatial and intensity-based similarities with the seeds in brain activity. Our results showed that, apart from the seed regions, significant regions detected by intensity-based similarity analysis differ completely from those found using cross-correlation. These findings may indicate that information shared through spatial coding differs from that transmitted via non-spatial coding processes. Our results suggest that brain coding is organized in several different ways to optimize information processing.

Approach of spectral information-based image registration similarity

For local intensity-based image registration methods, a template of a predefined size is adopted. Some measure of similarity between the images is used to determine when the optimal alignment has occurred. Most intensity-based similarity utilizes the statistical and spatial information of an image. However, there are significant nonlinear radiometric differences between infrared and visible data. Images of natural scenes usually do not have enough spatial features within the template. Hence, many similarities will be useless when dealing with infrared and visible data. Minor eigenvalues (ME) image registration similarity is presented with the exploitation the spectral properties of remote sensing images. ME similarity is based on the linear spectral mixture model and detects control points through searching the minimum of ME of the covariance matrix. Experiments on Landsat-7 satellite Enhanced Thematic Mapper Plus data are performed to verify and evaluate the effectiveness. Transformation performance curves, correct match ratio (CMR), and registration accuracy are also discussed. According to the data, root-mean-square error of phase correlations is 0.0722 pixels and the CMR of ME similarity is nearly 100%. The results on the basis of TM1, TM2, TM3, and TM4 band images indicate that the proposed similarity holds promise for infrared and color image registration in natural scenes, with advantages over previous normal mutual information and gradient mutual information similarities.

Evaluation of a Learning-based Deformable Registration Method on Abdominal CT Images

BackgroundReliable image comparisons, based on fast and accurate deformable registration methods, are recognized as key steps in the diagnosis and follow-up of cancer as well as for radiation therapy planning or surgery. In the particular case of abdominal images, the images to compare often differ widely from each other due to organ deformation, patient motion, movements of gastrointestinal tract or breathing. As a consequence, there is a need for registration methods that can cope with both local and global large and highly non-linear deformations. MethodDeformable registration of medical images traditionally relies on the iterative minimization of a cost function involving a large number of parameters. For complex deformations and large datasets, this process is computationally very demanding, leading to processing times that are incompatible with the clinical routine workflow. Moreover, the highly non-convex nature of these optimization problems leads to a high risk of convergence toward local minima. Recently, deep learning approaches using Convolutional Neural Networks (CNN) have led to major breakthroughs by providing computationally fast unsupervised methods for the registration of 2D and 3D images within seconds. Among all the proposed approaches, the VoxelMorph learning-based framework pioneered to learn in an unsupervised way the complex mapping, parameterized using a CNN, between every couple of 2D or 3D pairs of images and the corresponding deformation field by minimizing a standard intensity-based similarity metrics over the whole learning database. Voxelmorph has so far only been evaluated on brain images. The present study proposes to evaluate this method in the context of inter-subject registration of abdominal CT images, which present a greater challenge in terms of registration than brain images, due to greater anatomical variability and significant organ deformations. ResultsThe performances of VoxelMorph were compared with the current top-performing non-learning-based deformable registration method “Symmetric Normalization” (SyN), implemented in ANTs, on two representative databases: LiTS and 3D-IRCADb-01. Three different experiments were carried out on 2D or 3D data, the atlas-based or pairwise registration, using two different similarity metrics, namely (MSE and CC). Accuracy of the registration was measured by the Dice score, which quantifies the volume overlap for the selected anatomical region.All the three experiments exhibit that the two deformable registration methods significantly outperform the affine registration and that VoxelMorph accuracy is comparable or even better than the reference non-learning based registration method ANTs (SyN), with a drastically reduced computation time. ConclusionBy substituting a time consuming optimization problem, VoxelMorph has made an outstanding achievement in learning-based registration algorithm, where a registration function is trained and thus, able to perform deformable registration almost accurately on abdominal images, while reducing the computation time from minutes to seconds and from seconds to milliseconds in comparison to ANTs (SyN) on a CPU.

Nonrigid registration of medical image based on adaptive local structure tensor and normalized mutual information.

Nonrigid registration of medical images is especially critical in clinical treatment. Mutual information is a popular similarity measure for medical image registration; however, only the intensity statistical characteristics of the global consistency of image are considered in MI, and the spatial information is ignored. In this paper, a novel intensity‐based similarity measure combining normalized mutual information with spatial information for nonrigid medical image registration is proposed. The different parameters of Gaussian filtering are defined according to the regional variance, the adaptive Gaussian filtering is introduced into the local structure tensor. Then, the obtained adaptive local structure tensor is used to extract the spatial information and define the weighting function. Finally, normalized mutual information is distributed to each pixel, and the discrete normalized mutual information is multiplied with a weighting term to obtain a new measure. The novel measure fully considers the spatial information of the image neighborhood, gives the location of the strong spatial information a larger weight, and the registration of the strong gradient regions has a priority over the small gradient regions. The simulated brain image with single‐modality and multimodality are used for registration validation experiments. The results show that the new similarity measure improves the registration accuracy and robustness compared with the classical registration algorithm, reduces the risk of falling into local extremes during the registration process.

A Large-Scale Gene Expression Intensity-Based Similarity Metric for Drug Repositioning

SummaryBiological systems often respond to a specific environmental or genetic perturbation without pervasive gene expression changes. Such robustness to perturbations, however, is not reflected on the current computational strategies that utilize gene expression similarity metrics for drug discovery and repositioning. Here we propose a new expression-intensity-based similarity metric that consistently achieved better performance than other state-of-the-art similarity metrics with respect to the gold-standard clustering of drugs with known mechanisms of action. The new metric directly emphasizes the genes exhibiting the greatest changes in expression in response to a perturbation. Using the new framework to systematically compare 3,332 chemical and 3,934 genetic perturbations across 10 cell types representing diverse cellular signatures, we identified thousands of recurrent and cell type-specific connections. We also experimentally validated two drugs identified by the analysis as potential topoisomerase inhibitors. The new framework is a valuable resource for hypothesis generation, functional testing, and drug repositioning.

Non-rigid Image Registration by Minimizing Weighted Residual Complexity

Background: Non-rigid registration of medical images with intensity distortions is a difficult problem due to the change in pixel intensity. It is caused by contrast agent or intensity bias field. Methods: In some cases, this problem can be solved using Residual Complexity (RC) method. However, relative modification of parameter in residual complexity would result in completely different experimental effect. Another drawback is sensitivity to noise. To handle this problem, a new intensity-based similarity measure, Weighted Residual Complexity (WRC) has been proposed for effective medical image registration in this paper. Specifically, the local entropy image of two images is computed to be aligned respectively. Then, a weighting function using a function of the local entropy difference is modeled. The weighting function is used to weight the residual image in residual complexity adaptively. The residual image is defined as the difference between reference image and warped floating image. Results: The weighting function assigns smaller weight to residual image if the corresponding pixel value is larger in local entropy difference. The proposed technique was applied to simulative and real medical images. The contrast experiments were made with mutual information, diffeomorphic demons and residual complexity. Conclusion: Also, the analysis of experimental results was made qualitatively and quantitatively, which indicates that this new approach gives a better performance than diffeomorphic demons, mutual information and residual complexity. Keywords: Weighting function, local entropy, residual complexity, intensity distortion, image registration, WRC.

A Large-Scale Gene Expression Intensity-Based Similarity Metric for Drug Repositioning

Biological systems often respond to a specific environmental or genetic perturbation without pervasive gene ex pression changes. Such robustness to perturbations, however, is not re flected on the current computational strategies that utilize gene expression si milarity metrics for drug discovery and repositioning. Here we propos e a new expression intensity-based similarity metric that consistently achieved better performance than other state-of-the-art similarity metrics with respect to the gold-standard clustering of drugs with known mechanisms of action. The new metric directly emphasizes the genes ex hibiting the greatest changes in expression in response to a perturbation. Using the new framework to systematically compare 3,332 chemic al and 3,934 genetic perturbations across 10 cell types representing diverse cellular signatures, we identified thousands of recurrent and cell type- specific connections. We also experimentally validated two drugs (an anthelmintic and a loop diuretic) identified by the analysis as potential topoisomerase inhibitors. The new framework is a valuable resource for hypothesis generation, functional testing, and drug repositioning.

RISM: Single-Modal Image Registration via Rank-Induced Similarity Measure.

Similarity measure is an important block in image registration. Most traditional intensity-based similarity measures (e.g., sum-of-squared-difference, correlation coefficient, and mutual information) assume a stationary image and pixel-by-pixel independence. These similarity measures ignore the correlation between pixel intensities; hence, perfect image registration cannot be achieved, especially in the presence of spatially varying intensity distortions. Here, we assume that spatially varying intensity distortion (such as bias field) is a low-rank matrix. Based on this assumption, we formulate the image registration problem as a nonlinear and low-rank matrix decomposition (NLLRMD). Therefore, image registration and correction of spatially varying intensity distortion are simultaneously achieved. We illustrate the uniqueness of NLLRMD, and therefore, we propose the rank of difference image as a robust similarity in the presence of spatially varying intensity distortion. Finally, by incorporating the Gaussian noise, we introduce rank-induced similarity measure based on the singular values of the difference image. This measure produces clinically acceptable registration results on both simulated and real-world problems examined in this paper, and outperforms other state-of-the-art measures such as the residual complexity approach.

Towards measuring neuroimage misalignment

To enhance neuro-navigation, high quality pre-operative images must be registered onto intra-operative configuration of the brain. Therefore evaluation of the degree to which structures may remain misaligned after registration is critically important. We consider two Hausdorff Distance (HD)-based evaluation approaches: the edge-based HD (EBHD) metric and the Robust HD (RHD) metric as well as various commonly used intensity-based similarity metrics such as Mutual Information (MI), Normalised Mutual Information (NMI), Entropy Correlation Coefficient (ECC), Kullback–Leibler Distance (KLD) and Correlation Ratio (CR). We conducted the evaluation by applying known deformations to simple sample images and real cases of brain shift. We conclude that the intensity-based similarity metrics such as MI, NMI, ECC, KLD and CR do not correlate well with actual alignment errors, and hence are not useful for assessing misalignment. On the contrary, the EBHD and the RHD metrics correlated well with actual alignment errors; however, they have been found to underestimate the actual misalignment. We also note that it is beneficial to present HD results as a percentile-HD curve rather than a single number such as the 95-percentile HD. Percentile-HD curves present the full range of alignment errors and also facilitate the comparison of results obtained using different approaches. Furthermore, the qualities that should be possessed by an ideal evaluation metric were highlighted. Future studies could focus on developing such an evaluation metric.

Automatic and accurate registration of VHR optical and SAR images using a quadtree structure

In this article, we combined intensity- and feature-based similarity measures to co-register very-high-resolution (VHR) optical and SAR images. The global translation difference between optical and SAR images is minimized by applying a mutual information (MI) intensity-based similarity measure from coarsest to finest pyramid images constructed from the images in order. Matching points are then extracted considering the spatial distance and gradient orientation of linear features extracted from each image. To increase the reliability of the registration result, a quadtree-based structure is constructed (1) to mask out regions from the similarity measurement such as dense urban or heterogeneous areas, which can cause large differences in geometric and radiometric properties in two images; and (2) to extract evenly distributed and precise matching points by considering regional properties of the study site. To evaluate the proposed method’s generalization, various VHR optical and SAR sensors are used and compounded to construct the study sites. Evenly distributed matching points across the whole image were extracted, and reliable registration results by a non-linear transformation constructed from the points were derived from the proposed method.

Sparse‐induced similarity measure: mono‐modal image registration via sparse‐induced similarity measure

Similarity measure is an important key in image registration. Most traditional intensity-based similarity measures (e.g. sum-of-squared-differences, correlation coefficient, mutual information and correlation ratio) assume a stationary image and pixel-by-pixel independence. These similarity measures ignore the correlation among pixel intensities; hence, a perfect image registration cannot be achieved especially in the presence of spatially varying intensity distortions and outlier objects that appear in one image but not in the other. It is supposed here that non-stationary intensity distortion (such as bias field) has a sparse representation in the transformation domain. Based on this assumption, a novel similarity measure is proposed here based on sparse representation in a mono-modal setting. The zero norm (l 0 ) in the transform domain is introduced as a new similarity measure in the presence of non-stationary intensity distortion. The present study defines a sparsity similarity measure that indicates the complexity of the residual image between two registered images in the transform domain such as discrete cosine transform or wavelet. It is attempted here to analytically and statistically illustrate that the proposed similarity measure has important properties such as metric properties in vector space, from correntropy perspective. This measure produces accurate registration results on both artificial and real-world problems which are examined in the present study.

2D-3D registration for brain radiation therapy using a 3D CBCT and a single limited field-of-view 2D kV radiograph

We report results of an intensity-based 2D-3D rigid registration framework for patient positioning and monitoring during brain radiotherapy. We evaluated two intensity-based similarity measures, the Pearson Correlation Coefficient (ICC) and Maximum Likelihood with Gaussian noise (MLG) derived from the statistics of transmission images. A useful image frequency band was identified from the bone-to-no-bone ratio. Validation was performed on gold-standard data consisting of 3D kV CBCT scans and 2D kV radiographs of an anthropomorphic head phantom acquired at 23 different poses with parameter variations along six degrees of freedom. At each pose, a single limited field of view kV radiograph was registered to the reference CBCT. The ground truth was determined from markers affixed to the phantom and visible in the CBCT images. The mean (and standard deviation) of the absolute errors in recovering each of the six transformation parameters along the x, y and z axes for ICC were φx: 0.08(0.04)°, φy: 0.10(0.09)°, φz: 0.03(0.03)°, tx: 0.13(0.11) mm, ty: 0.08(0.06) mm and tz: 0.44(0.23) mm. For MLG, the corresponding results were φx: 0.10(0.04)°, φy: 0.10(0.09)°, φz: 0.05(0.07)°, tx: 0.11(0.13) mm, ty: 0.05(0.05) mm and tz: 0.44(0.31) mm. It is feasible to accurately estimate all six transformation parameters from a 3D CBCT of the head and a single 2D kV radiograph within an intensity-based registration framework that incorporates the physics of transmission images.

Intensity-based similarity measures evaluation for CT to ultrasound 2D registration

Ultrasound is a non-invasive image modality which allows for real time acquisition. Nevertheless, the low quality of the acquired images makes this a difficult-to-interpret modality during surgical procedures. To overcome this, the registration of ultrasound images with preoperative CT or MR images has been routinely used to fuse complementary information. This work presents the evaluation of eight similarity measures used in the registration of ultrasound and CT images of the left atrium and the pulmonary veins. Each intensity-based similarity measure was evaluated computing its accuracy, capture range, distinctiveness of the optimum, risk and non-convergence and number of minima. The results show that the Woods criterion presents a globally better performance than the other similarity measures. This is especially true for the accuracy and distinctiveness of the optimum indicators. Preprocessing US images does not improve the performance of all similarity measures, except for Woods criterion that shows the optimal accuracy.

Intensity based image registration by minimizing exponential function weighted residual complexity

In this paper, we propose a novel intensity-based similarity measure for medical image registration. Traditional intensity-based methods are sensitive to intensity distortions, contrast agent and noise. Although residual complexity can solve this problem in certain situations, relative modification of the parameter can generate dramatically different results. By introducing a specifically designed exponential weighting function to the residual term in residual complexity, the proposed similarity measure performed well due to automatically weighting the residual image between the reference image and the warped floating image. We utilized local variance of the reference image to model the exponential weighting function. The proposed technique was applied to brain magnetic resonance images, dynamic contrast enhanced magnetic resonance images (DCE-MRI) of breasts and contrast enhanced 3D CT liver images. The experimental results clearly indicated that the proposed approach has achieved more accurate and robust performance than mutual information, residual complexity and Jensen–Tsallis.

3D navigation and monitoring for spinal milling operation based on registration between multiplanar fluoroscopy and CT images

Milling operations in spinal surgery demand much experience and skill for the surgeon to perform the procedure safely. A 3D navigation method is introduced aiming at providing a monitoring system with enhanced safety and minimal intraoperative interaction. An automatic registration method is presented to establish the 3D–3D transformation between the preoperative CT images and a common reference system in the surgical space, and an intensity-based similarity metric adapted for the multi-planar configuration is introduced in the registration procedure. A critical region is defined for real-time monitoring in order to prevent penetration of the lamina and avoid violation of nerve structures. The contour of the spinal canal is reconstructed as the critical region, and different levels of warning limits are defined. During the milling procedure, the position of the surgical instrument relative to the critical region is provided with augmented display and audio warnings. Timely alarm is provided for surgeons to prevent surgical failure when the mill approaches the critical region. Our validation experiment shows that real-time 3D navigation and monitoring is advantageous for improving the safety of the milling operation.

Biomechanically constrained groupwise ultrasound to CT registration of the lumbar spine

We present a groupwise US to CT registration algorithm for guiding percutaneous spinal interventions. In addition, we introduce a comprehensive validation scheme that accounts for changes in the curvature of the spine between preoperative and intraoperative imaging. In our registration methodology, each vertebra in CT is treated as a sub-volume and transformed individually. A biomechanical model is used to constrain the displacement of the vertebrae relative to one another. The sub-volumes are then reconstructed into a single volume. During each iteration of registration, an US image is simulated from the reconstructed CT volume and an intensity-based similarity metric is calculated with the real US image. Validation studies are performed on CT and US images from a sheep cadaver, five patient-based phantoms designed to preserve realistic curvatures of the spine and a sixth patient-based phantom where the curvature of the spine is changed between preoperative and intraoperative imaging. For datasets where the spine curve between two imaging modalities was artificially perturbed, the proposed methodology was able to register initial misalignments of up to 20mm with a success rate of 95%. For the phantom with a physical change in the curvature of the spine introduced between the US and CT datasets, the registration success rate was 98.5%. Finally, the registration success rate for the sheep cadaver with soft-tissue information was 87%. The results demonstrate that our algorithm allows for robust registration of US and CT datasets, regardless of a change in the patients pose between preoperative and intraoperative image acquisitions.

Intensity-Based Image Registration by Minimizing Residual Complexity

Accurate definition of the similarity measure is a key component in image registration. Most commonly used intensity-based similarity measures rely on the assumptions of independence and stationarity of the intensities from pixel to pixel. Such measures cannot capture the complex interactions among the pixel intensities, and often result in less satisfactory registration performances, especially in the presence of spatially-varying intensity distortions. We propose a novel similarity measure that accounts for intensity nonstationarities and complex spatially-varying intensity distortions in mono-modal settings. We derive the similarity measure by analytically solving for the intensity correction field and its adaptive regularization. The final measure can be interpreted as one that favors a registration with minimum compression complexity of the residual image between the two registered images. One of the key advantages of the new similarity measure is its simplicity in terms of both computational complexity and implementation. This measure produces accurate registration results on both artificial and real-world problems that we have tested, and outperforms other state-of-the-art similarity measures in these cases.

Computerized method for nonrigid MR-to-PET breast-image registration

We have developed and tested a new simple computerized finite element method (FEM) approach to MR-to-PET nonrigid breast-image registration. The method requires five–nine fiducial skin markers (FSMs) visible in MRI and PET that need to be located in the same spots on the breast and two on the flanks during both scans. Patients need to be similarly positioned prone during MRI and PET scans. This is accomplished by means of a low gamma-ray attenuation breast coil replica used as the breast support during the PET scan. We demonstrate that, under such conditions, the observed FSM displacement vectors between MR and PET images, distributed piecewise linearly over the breast volume, produce a deformed FEM mesh that reasonably approximates nonrigid deformation of the breast tissue between the MRI and PET scans. This method, which does not require a biomechanical breast tissue model, is robust and fast. Contrary to other approaches utilizing voxel intensity-based similarity measures or surface matching, our method works for matching MR with pure molecular images (i.e. PET or SPECT only). Our method does not require a good initialization and would not be trapped by local minima during registration process. All processing including FSMs detection and matching, and mesh generation can be fully automated. We tested our method on MR and PET breast images acquired for 15 subjects. The procedure yielded good quality images with an average target registration error below 4 mm (i.e. well below PET spatial resolution of 6–7 mm). Based on the results obtained for 15 subjects studied to date, we conclude that this is a very fast and a well-performing method for MR-to-PET breast-image nonrigid registration. Therefore, it is a promising approach in clinical practice. This method can be easily applied to nonrigid registration of MRI or CT of any type of soft-tissue images to their molecular counterparts such as obtained using PET and SPECT.