Multimodal Deformable Image Registration Research Articles

SOLID: a novel similarity metric for mono-modal and multi-modal deformable image registration

Medical image registration is an integral part of various clinical applications including image guidance, motion tracking, therapy assessment and diagnosis. We present a robust approach for mono-modal and multi-modal medical image registration. To this end, we propose the novel shape operator based local image distance (SOLID) which estimates the similarity of images by comparing their second-order curvature information. Our similarity metric is rigorously tailored to be suitable for comparing images from different medical imaging modalities or image contrasts. A critical element of our method is the extraction of local features using higher-order shape information, enabling the accurate identification and registration of smaller structures. In order to assess the efficacy of the proposed similarity metric, we have implemented a variational image registration algorithm that relies on the principle of matching the curvature information of the given images. The performance of the proposed algorithm has been evaluated against various alternative state-of-the-art variational registration algorithms. Our experiments involve mono-modal as well as multi-modal and cross-contrast co-registration tasks in a broad variety of anatomical regions. Compared to the evaluated alternative registration methods, the results indicate a very favorable accuracy, precision and robustness of the proposed SOLID method in various highly challenging registration tasks.

Evaluating the Hounsfield unit assignment and dose differences between CT-based standard and deep learning-based synthetic CT images for MRI-only radiation therapy of the head and neck.

Magnetic resonance image only (MRI-only) simulation for head and neck (H&N) radiotherapy (RT) could allow for single-image modality planning with excellent soft tissue contrast. In the MRI-only simulation workflow, synthetic computed tomography (sCT) is generated from MRI to provide electron density information for dose calculation. Bone/air regions produce little MRI signal which could lead to electron density misclassification in sCT. Establishing the dosimetric impact of this error could inform quality assurance (QA) procedures using MRI-only RT planning or compensatory methods for accurate dosimetric calculation. The aim of this study was to investigate if Hounsfield unit (HU) voxel misassignments from sCT images result in dosimetric errors in clinical treatment plans. Fourteen H&N cancer patients undergoing same-day CT and 3T MRI simulation were retrospectively identified. MRI was deformed to the CT using multimodal deformable image registration. sCTs were generated from T1w DIXON MRIs using a commercially available deep learning-based generator (MRIplanner, Spectronic Medical AB, Helsingborg, Sweden). Tissue voxel assignment was quantified by creating a CT-derived HU threshold contour. CT/sCT HU differences for anatomical/target contours and tissue classification regions including air (<250 HU), adipose tissue (-250 HU to -51 HU), soft tissue (-50 HU to 199 HU), spongy (200 HU to 499 HU) and cortical bone(>500 HU) were quantified. t-test was used to determine if sCT/CT HU differences were significant. The frequency of structures that had a HU difference>80 HU (the CT window-width setting for intra-cranial structures) was computed to establish structure classification accuracy. Clinical intensity modulated radiation therapy (IMRT) treatment plans created on CT were retrospectively recalculated on sCT images and compared using the gamma metric. The mean ratio of sCT HUs relative to CT for air, adipose tissue, soft tissue, spongy and cortical bone were 1.7±0.3, 1.1±0.1, 1.0±0.1, 0.9±0.1 and 0.8±0.1 (value of 1 indicates perfect agreement). T-tests (significance set at t=0.05) identified differences in HU values for air, spongy and cortical bone in sCT images compared to CT. The structures with sCT/CT HU differences>80 HU of note were the left and right (L/R) cochlea and mandible (>79% of the tested cohort), the oral cavity (for 57% of the tested cohort), the epiglottis (for 43% of the tested cohort) and the L/R TM joints (occurring>29% of the cohort). In the case of the cochlea and TM joints, these structures contain dense bone/air interfaces. In the case of the oral cavity and mandible, these structures suffer the additional challenge of being positionally altered in CT versus MRI simulation (due to a non-MR safe immobilizing bite block requiring absence of bite block in MR). Finally, the epiglottis HU assignment suffers from its small size and unstable positionality. Plans recalculated on sCT yielded global/local gamma pass rates of 95.5%±2% (3mm, 3%) and 92.7%±2.1% (2mm, 2%). The largest mean differences in D95, Dmean , D50 dose volume histogram (DVH) metrics for organ-at-risk (OAR) and planning tumor volumes (PTVs) were 2.3%±3.0% and 0.7%±1.9% respectively. In this cohort, HU differences of CT and sCT were observed but did not translate into a reduction in gamma pass rates or differences in average PTV/OAR dose metrics greater than 3%. For sites such as the H&N where there are many tissue interfaces we did not observe large scale dose deviations but further studies using larger retrospective cohorts are merited to establish the variation in sCT dosimetric accuracy which could help to inform QA limits on clinical sCT usage.

Quantifying the Dosimetric Impact of Tissue Hounsfield Unit Assignment in Deep Learning-based Synthetic CT Images For MRI-Only Radiation Therapy of The Head and Neck

MRI-only simulation for head and neck (HN) radiotherapy (RT) could allow for single-image modality planning with excellent soft tissue contrast. In the MRI-only simulation workflow, synthetic CTs (sCTs) are generated from MRI to provide electron density information for dose calculation. Bone/air regions produce little MRI signal which could lead to electron density misclassification in sCTs. Establishing the dosimetric impact of this error could inform quality assurance (QA) procedures using MRI-only RT planning. In this study we quantify differences in Hounsfield Unit (HU) values between paired CT/ sCTs of HN cancer patients and investigate the dosimetric impact on clinical treatment plans. Fourteen patients with head and neck cancer undergoing same-day CT and 3T MRI simulation were retrospectively identified. MRIs were deformed to the CT using multimodal deformable image registration. SCTs were generated from T1w DIXON MRI using a commercially available deep learning-based generator (MRIplanner, Spectronics). Tissue voxel assignment was quantified by creating a CT-derived HU threshold contour. CT/sCT HU differences for anatomical/target contours and tissue classification regions including air (<-250HU), adipose tissue (-250HU to -51HU), soft tissue (-50HU to 199HU), spongy (200HU to 499HU) and cortical bone (> 500HU) were quantified. T-test was used to determine if sCT/CT HU differences were significant. The frequency of structures that had a HU difference >70HU (the CT window-width setting for intra-cranial structures) was computed to establish structure classification accuracy. Clinical IMRT treatment plans created on CTs were retrospectively recalculated on sCT images using compared using the gamma metric. The mean ratio of sCT HUs relative to CT for air, adipose tissue, soft tissue, spongy and cortical bone were 1.7±0.3, 1.1±0.1, 1.0±0.1, 0.9±0.1 and 0.8±0.1 (value of 1 indicates perfect agreement). T-tests (significance set at t = 0.05) identified differences in HU values for air, spongy and cortical bone in sCT images compared to CT. The structures with sCT/CT HU differences > 70HU were the L/R cochlea and mandible, occurring in >78% of the tested cohort. These structures contain dense bone/air interfaces. Plans recalculated on sCTs yielded global/local gamma pass rates of 98.7%±1.2% (3mm,3%) and 95.5%±2.5% (1mm,1%). Mean differences in D95, D50, D10 and D2 dose volume histogram (DVH) metrics for organ-at-risk (OAR) and planning tumor volumes (PTV) were 0.8%±1.5% and 2.1% ± 1.2% respectively. In this cohort, HU differences in sCTs were observed but did not translate into a reduction in gamma pass rates and OAR/PTV DVH metrics. The acquisition of additional training data such as ultrashort echo time MRI could improve bone/air contrast and reduce bone/air sCT misclassifications. Further studies will establish the variation in sCT dosimetric accuracy using a larger retrospective cohort to inform QA limits on clinical sCT usage.

A transformer-based hierarchical registration framework for multimodality deformable image registration.

Deformable image registration (DIR) between daily and reference images is fundamentally important for adaptive radiotherapy. In the last decade, deep learning-based image registration methods have been developed with faster computation time and improved robustness compared to traditional methods. However, the registration performance is often degraded in extra-cranial sites with large volume containing multiple anatomic regions, such as Computed Tomography (CT)/Magnetic Resonance (MR) images used in head and neck (HN) radiotherapy. In this study, we developed a hierarchical deformable image registration (DIR) framework, Patch-based Registration Network (Patch-RegNet), to improve the accuracy and speed of CT-MR and MR-MR registration for head-and-neck MR-Linac treatments. Patch-RegNet includes three steps: a whole volume global registration, a patch-based local registration, and a patch-based deformable registration. Following a whole-volume rigid registration, the input images were divided into overlapping patches. Then a patch-based rigid registration was applied to achieve accurate local alignment for subsequent DIR. We developed a ViT-Morph model, a combination of a convolutional neural network (CNN) and the Vision Transformer (ViT), for the patch-based DIR. A modality independent neighborhood descriptor was adopted in our model as the similarity metric to account for both inter-modality and intra-modality registration. The CT-MR and MR-MR DIR models were trained with 242 CT-MR and 213 MR-MR image pairs from 36 patients, respectively, and both tested with 24 image pairs (CT-MR and MR-MR) from 6 other patients. The registration performance was evaluated with 7 manually contoured organs (brainstem, spinal cord, mandible, left/right parotids, left/right submandibular glands) by comparing with the traditional registration methods in Monaco treatment planning system and the popular deep learning-based DIR framework, Voxelmorph. Evaluation results show that our method outperformed VoxelMorph by 6 % for CT-MR registration, and 4 % for MR-MR registration based on DSC measurements. Our hierarchical registration framework has been demonstrated achieving significantly improved DIR accuracy of both CT-MR and MR-MR registration for head-and-neck MR-guided adaptive radiotherapy.

Quantitative assessment of intra- and inter-modality deformable image registration of the heart, left ventricle, and thoracic aorta on longitudinal 4D-CT and MR images.

PurposeMagnetic resonance imaging (MRI)‐based investigations into radiotherapy (RT)‐induced cardiotoxicity require reliable registrations of magnetic resonance (MR) imaging to planning computed tomography (CT) for correlation to regional dose. In this study, the accuracy of intra‐ and inter‐modality deformable image registration (DIR) of longitudinal four‐dimensional CT (4D‐CT) and MR images were evaluated for heart, left ventricle (LV), and thoracic aorta (TA).Methods and materialsNon‐cardiac‐gated 4D‐CT and T1 volumetric interpolated breath‐hold examination (T1‐VIBE) MRI datasets from five lung cancer patients were obtained at two breathing phases (inspiration/expiration) and two time points (before treatment and 5 weeks after initiating RT). Heart, LV, and TA were manually contoured. Each organ underwent three intramodal DIRs ((A) CT modality over time, (B) MR modality over time, and (C) MR contrast effect at the same time) and two intermodal DIRs ((D) CT/MR multimodality at same time and (E) CT/MR multimodality over time). Hausdorff distance (HD), mean distance to agreement (MDA), and Dice were evaluated and assessed for compliance with American Association of Physicists in Medicine (AAPM) Task Group (TG)‐132 recommendations.ResultsMean values of HD, MDA, and Dice under all registration scenarios for each region of interest ranged between 8.7 and 16.8 mm, 1.0 and 2.6 mm, and 0.85 and 0.95, respectively, and were within the TG‐132 recommended range (MDA < 3 mm, Dice > 0.8). Intramodal DIR showed slightly better results compared to intermodal DIR. Heart and TA demonstrated higher registration accuracy compared to LV for all scenarios except for HD and Dice values in Group A. Significant differences for each metric and tissue of interest were noted between Groups B and D and between Groups B and E. MDA and Dice significantly differed between LV and heart in all registrations except for MDA in Group E.ConclusionsDIR of the heart, LV, and TA between non‐cardiac‐gated longitudinal 4D‐CT and MRI across two modalities, breathing phases, and pre/post‐contrast is acceptably accurate per AAPM TG‐132 guidelines. This study paves the way for future evaluation of RT‐induced cardiotoxicity and its related factors using multimodality DIR.

Combined Mass Spectrometry and Histopathology Imaging for Perioperative Tissue Assessment in Cancer Surgery.

Mass spectrometry is an effective imaging tool for evaluating biological tissue to detect cancer. With the assistance of deep learning, this technology can be used as a perioperative tissue assessment tool that will facilitate informed surgical decisions. To achieve such a system requires the development of a database of mass spectrometry signals and their corresponding pathology labels. Assigning correct labels, in turn, necessitates precise spatial registration of histopathology and mass spectrometry data. This is a challenging task due to the domain differences and noisy nature of images. In this study, we create a registration framework for mass spectrometry and pathology images as a contribution to the development of perioperative tissue assessment. In doing so, we explore two opportunities in deep learning for medical image registration, namely, unsupervised, multi-modal deformable image registration and evaluation of the registration. We test this system on prostate needle biopsy cores that were imaged with desorption electrospray ionization mass spectrometry (DESI) and show that we can successfully register DESI and histology images to achieve accurate alignment and, consequently, labelling for future training. This automation is expected to improve the efficiency and development of a deep learning architecture that will benefit the use of mass spectrometry imaging for cancer diagnosis.

A framework for voxel-based assessment of biological effect after proton radiotherapy in pediatric brain cancer patients using multi-modal imaging.

The exact dependence of biological effect on dose and linear energy transfer (LET) in human tissue when delivering proton therapy is unknown. In this study, we propose a framework for measuring this dependency using multi-modal image-based assays with deformable registrations within imaging sessions and across time. 3T MRI scans were prospectively collected from 6 pediatric brain cancer patients before they underwent proton therapy treatment, and every 3months for a year after treatment. Scans included T1-weighted with contrast enhancement (T1), T2-FLAIR (T2) and fractional anisotropy (FA) images. In addition, the planning CT, dose distributions and Monte Carlo-calculated LET distributions were collected. A multi-modal deformable image registration framework was used to create a dataset of dose, LET and imaging intensities at baseline and follow-up on a voxel-by-voxel basis. We modelled the biological effect of dose and LET from proton therapy using imaging changes over time as a surrogate for biological effect. We investigated various models to show the feasibility of the framework to model imaging changes. To account for interpatient and intrapatient variations, we used a nested generalized linear mixed regression model. The models were applied to predict imaging changes over time as a function of dose and LET for each modality. Using the nested models to predict imaging changes, we saw a decrease in the FA signal as a function of dose; however, the signal increased with increasing LET. Similarly, we saw an increase in T2 signal as a function of dose, but a decrease in signal with LET. We saw no changes in T1 voxel values as a function of either dose or LET. The imaging changes could successfully model biological effect as a function of dose and LET using our proposed framework. Due to the low number of patients, the imaging changes observed for FA and T2 scans were not marked enough to draw any firm conclusions.

Anatomically-adaptive multi-modal image registration for image-guided external-beam radiotherapy

Image-guided radiotherapy (IGRT) allows observation of the location and shape of the tumor and organs-at-risk (OAR) over the course of a radiation cancer treatment. Such information may in turn be used for reducing geometric uncertainties during therapeutic planning, dose delivery and response assessment. However, given the multiple imaging modalities and/or contrasts potentially included within the imaging protocol over the course of the treatment, the current manual approach to determining tissue displacement may become time-consuming and error prone. In this context, variational multi-modal deformable image registration (DIR) algorithms allow automatic estimation of tumor and OAR deformations across the acquired images. In addition, they require short computational times and a low number of input parameters, which is particularly beneficial for online adaptive applications, which require on-the-fly adaptions with the patient on the treatment table.However, the majority of such DIR algorithms assume that all structures across the entire field-of-view (FOV) undergo a similar deformation pattern. Given that various anatomical structures may behave considerably different, this may lead to the estimation of anatomically implausible deformations at some locations, thus limiting their validity. Therefore, in this paper we propose an anatomically-adaptive variational multi-modal DIR algorithm, which employs a regionalized registration model in accordance with the local underlying anatomy. The algorithm was compared against two existing methods which employ global assumptions on the estimated deformations patterns.Compared to the existing approaches, the proposed method has demonstrated an improved anatomical plausibility of the estimated deformations over the entire FOV as well as displaying overall higher accuracy. Moreover, despite the more complex registration model, the proposed approach is very fast and thus suitable for online scenarios. Therefore, future adaptive IGRT workflows may benefit from an anatomically-adaptive registration model for precise contour propagation and dose accumulation, in areas showcasing considerable variations in anatomical properties.

Patch-based field-of-view matching in multi-modal images for electroporation-based ablations

Various multi-modal imaging sensors are currently involved at different steps of an interventional therapeutic work-flow. Cone beam computed tomography (CBCT), computed tomography (CT) or Magnetic Resonance (MR) images thereby provides complementary functional and/or structural information of the targeted region and organs at risk. Merging this information relies on a correct spatial alignment of the observed anatomy between the acquired images. This can be achieved by the means of multi-modal deformable image registration (DIR), demonstrated to be capable of estimating dense and elastic deformations between images acquired by multiple imaging devices. However, due to the typically different field-of-view (FOV) sampled across the various imaging modalities, such algorithms may severely fail in finding a satisfactory solution.In the current study we propose a new fast method to align the FOV in multi-modal 3D medical images. To this end, a patch-based approach is introduced and combined with a state-of-the-art multi-modal image similarity metric in order to cope with multi-modal medical images. The occurrence of estimated patch shifts is computed for each spatial direction and the shift value with maximum occurrence is selected and used to adjust the image field-of-view. The performance of the proposed method – in terms of both registration accuracy and computational needs – is analyzed in the practical case of on-line irreversible electroporation procedures. In total, 30 pairs of pre-/per-operative IRE images are considered to illustrate the efficiency of our algorithm.We show that a regional registration approach using voxel patches provides a good structural compromise between the voxel-wise and “global shifts” approaches. The method was thereby beneficial for CT to CBCT and MRI to CBCT registration tasks, especially when highly different image FOVs are involved. Besides, the benefit of the method for CT to CBCT and MRI to CBCT image registration is analyzed, including the impact of artifacts generated by percutaneous needle insertions. Additionally, the computational needs using commodity hardware are demonstrated to be compatible with clinical constraints in the practical case of on-line procedures. The proposed patch-based workflow thus represents an attractive asset for DIR at different stages of an interventional procedure.

Automated, reference-free local error assessment of multimodal deformable image registration for radiotherapy in the head and neck

BackgroundHead and neck MR-CT deformable image registration (DIR) for radiotherapy planning is hindered by the lack of both ground-truth and per-patient accuracy assessment methods. This study assesses novel post-registration reference-free error assessment algorithms, based on local rigid re-registration of native and pseudomodality images. MethodsHead and neck MR obtained in and out of the treatment position underwent DIR to planning CT. Block-wise mutual information (b-MI) and pseudomodality mutual information (b-pmMI) algorithms were validated against applied rotations and translations. Inherent registration error detection was compared across 14 patient datasets. ResultsUsing radiotherapy position MR-CT DIR, quantitative comparison of applied rotations and translations revealed that errors between 1 and 4 mm were accurately determined by both algorithms. Using diagnostic position MR-CT DIR, translations of up to 5 mm were accurately detected within the gross tumour volume by both methods. In 14 patient datasets, b-MI and b-pmMI detected similar errors with improved stability in regions of low contrast or CT artefact and a 10-fold speedup for b-pmMI. Conclusionsb-MI and b-pmMI algorithms have been validated as providing accurate reference-free quantitative assessment of DIR accuracy on a per-patient basis. b-pmMI is faster and more robust in the presence of modality-specific information.

Multimodal image registration for the identification of dominant intraprostatic lesion in high-precision radiotherapy treatments.

The integration of CT and multiparametric MRI (mpMRI) is a challenging task in high-precision radiotherapy for prostate cancer. A simple methodology for multimodal deformable image registration (DIR) of prostate cancer patients is presented. CT and mpMRI of 10 patients were considered. Organs at risk and prostate were contoured on both scans. The dominant intraprostatic lesion was additionally delineated on MRI. After a preliminary rigid image registration, the voxel intensity of all the segmented structures in both scans except the prostate was increased by a specific amount (a constant additional value, A), in order to enhance the contrast of the main organs influencing its position and shape. 70 couples of scans were obtained by varying A from 0 to 800 and they were subsequently non-rigidly registered. Quantities derived from image analysis and contour statistics were considered for the tuning of the best performing A. A = 200 resulted the minimum enhancement value required to obtain statistically significant superior registration results. Mean centre of mass distance between corresponding structures decreases from 7.4 mm in rigid registration to 5.3 mm in DIR without enhancement (DIR-0) and to 2.7 mm in DIR with A = 200 (DIR-200). Mean contour distance was 2.5, 1.9 and 0.67 mm in rigid registration, DIR-0 and DIR-200, respectively. In DIR-200 mean contours overlap increases of +13 and +24% with respect to DIR-0 and rigid registration, respectively. Contour propagation according to the vector field resulting from DIR-200 allows the delineation of dominant intraprostatic lesion on CT scan and its use for high-precision radiotherapy treatment planning. Advances in knowledge: We investigated the application of a B-spline, mutual information-based multimodal DIR coupled with a simple, patient-unspecific but efficient contrast enhancement procedure in the pelvic body area, thus obtaining a robust and accurate methodology to transfer the functional information deriving from mpMRI onto a planning CT reference volume.

Fast and robust multimodal image registration using a local derivative pattern.

Deformable multimodal image registration, which can benefit radiotherapy and image guided surgery by providing complementary information, remains a challenging task in the medical image analysis field due to the difficulty of defining a proper similarity measure. This article presents a novel, robust and fast binary descriptor, the discriminative local derivative pattern (dLDP), which is able to encode images of different modalities into similar image representations. dLDP calculates a binary string for each voxel according to the pattern of intensity derivatives in its neighborhood. The descriptor similarity is evaluated using the Hamming distance, which can be efficiently computed, instead of conventional L1 or L2 norms. For the first time, we validated the effectiveness and feasibility of the local derivative pattern for multimodal deformable image registration with several multi-modal registration applications. dLDP was compared with three state-of-the-art methods in artificial image and clinical settings. In the experiments of deformable registration between different magnetic resonance imaging (MRI) modalities from BrainWeb, between computed tomography and MRI images from patient data, and between MRI and ultrasound images from BITE database, we show our method outperforms localized mutual information and entropy images in terms of both accuracy and time efficiency. We have further validated dLDP for the deformable registration of preoperative MRI and three-dimensional intraoperative ultrasound images. Our results indicate that dLDP reduces the average mean target registration error from 4.12mm to 2.30mm. This accuracy is statistically equivalent to the accuracy of the state-of-the-art methods in the study; however, in terms of computational complexity, our method significantly outperforms other methods and is even comparable to the sum of the absolute difference. The results reveal that dLDP can achieve superior performance regarding both accuracy and time efficiency in general multimodal image registration. In addition, dLDP also indicates the potential for clinical ultrasound guided intervention.

Tagged MRI based cardiac motion modeling and toxicity evaluation in breast cancer radiotherapy.

Recent research showed radiation for breast cancer can increase heart risks (1, 2). In Ref. (2), it has been noted that for every Gy of radiation a women’s heart risk rises 7.4%. However, the correlation between radiation dose and heart tissue damage is still an open problem. A more accurate model of heart damage will significantly improve the heart safety for patients underwent radiotherapy. Modern radiation treatment planning systems (TPS) use computed tomography (CT) images for dose calculation and evaluation. For evaluation of heart toxicity from radiotherapy, the dose-volume histogram (DVH), which is generated by overlying radiation dose distribution on heart delineations in CT images, is widely used. However, there are three major factors that deteriorate the accuracy of TPS-calculated heart dose distribution. First conventional CT is a fundamentally static imaging modality without the capability to capture and depict the cardiac motion. Instead, heart is usually blurred in CT images due to the motion artifacts. Second, without special contrast dye, CT provides limited contrast between blood in heart chambers and the surrounding myocardium. The heart region in TPS is actually a mixture of myocardium and blood, although only the radiation dose to the myocardium is accountable for heart risks. Finally, there is significant intra- and inter-fractional heart motion. As heart beats involuntarily during and between radiation treatments, myocardium deforms and moves non-rigidly against the fixed radiation beam so that the static dose distribution calculated in CT based TPS does not reflect the accurate radiation dose distribution in heart. There is also concern on the choice of the heart function for the evaluation of radiation damage. Based on radiation beam geometry, only part of the heart will receive clinically significant level of radiation during breast cancer treatment. It is possible that the global heart function remains stable temporarily while cells in the irradiated part of the myocardium lose part or all of their functions. In this case, regional heart function, which can be derived from regional heart wall motion and strain analysis, is a better indication of heart damage corresponding to radiation dose. Although cardiac MRI is widely used in radiology for the diagnosis of heart disease, its application in radiation treatment planning is limited. For multiple reasons, it is not practical to use MRI directly for radiation treatment planning of breast cancer patients. However, via multimodality deformable image registration (DIR) between MRI and CT, MRI images may play a more critical role in the evaluation of the heart damage from whole breast radiation. Tagged MRI (tMRI) (3) is a relatively new imaging protocol that has been implemented in the detection and diagnosis of regional heart functional loss. tMRI methods record regional heart wall motion information as they create identifiable landmark bands (tags) in the myocardium to establish dense point to point correspondence between images. ECG-gated tMRI image sets can be acquired at different phases of the cardiac cycle using the corresponding pulse sequence. The 4D (3D plus time) cardiac motion model can be retrieved by image registration between tMRIs at different phases. In the following sessions, we use tMRI as an example to explain how additional heart function information in MRI is retrieved. It is our objective to demonstrate the additional information retrieved from MRI can help the evaluation and protection of heart risks for breast cancer patients, and we want to discuss the possibility of using MRI to establish a more accurate correlation between regional heart functional loss and radiation dose.

Validation of a deformable image registration technique for cone beam CT-based dose verification.

As radiation therapy evolves toward more adaptive techniques, image guidance plays an increasingly important role, not only in patient setup but also in monitoring the delivered dose and adapting the treatment to patient changes. This study aimed to validate a method for evaluation of delivered intensity modulated radiotherapy (IMRT) dose based on multimodal deformable image registration (dir) for prostate treatments. A pelvic phantom was scanned with CT and cone-beam computed tomography (CBCT). Both images were digitally deformed using two realistic patient-based deformation fields. The original CT was then registered to the deformed CBCT resulting in a secondary deformed CT. The registration quality was assessed as the ability of the dir method to recover the artificially induced deformations. The primary and secondary deformed CT images as well as vector fields were compared to evaluate the efficacy of the registration method and it's suitability to be used for dose calculation. plastimatch, a free and open source software was used for deformable image registration. A B-spline algorithm with optimized parameters was used to achieve the best registration quality. Geometric image evaluation was performed through voxel-based Hounsfield unit (HU) and vector field comparison. For dosimetric evaluation, IMRT treatment plans were created and optimized on the original CT image and recomputed on the two warped images to be compared. The dose volume histograms were compared for the warped structures that were identical in both warped images. This procedure was repeated for the phantom with full, half full, and empty bladder. The results indicated mean HU differences of up to 120 between registered and ground-truth deformed CT images. However, when the CBCT intensities were calibrated using a region of interest (ROI)-based calibration curve, these differences were reduced by up to 60%. Similarly, the mean differences in average vector field lengths decreased from 10.1 to 2.5 mm when CBCT was calibrated prior to registration. The results showed no dependence on the level of bladder filling. In comparison with the dose calculated on the primary deformed CT, differences in mean dose averaged over all organs were 0.2% and 3.9% for dose calculated on the secondary deformed CT with and without CBCT calibration, respectively, and 0.5% for dose calculated directly on the calibrated CBCT, for the full-bladder scenario. Gamma analysis for the distance to agreement of 2 mm and 2% of prescribed dose indicated a pass rate of 100% for both cases involving calibrated CBCT and on average 86% without CBCT calibration. Using deformable registration on the planning CT images to evaluate the IMRT dose based on daily CBCTs was found feasible. The proposed method will provide an accurate dose distribution using planning CT and pretreatment CBCT data, avoiding the additional uncertainties introduced by CBCT inhomogeneity and artifacts. This is a necessary initial step toward future image-guided adaptive radiotherapy of the prostate.

A three-dimensional head-and-neck phantom for validation of multimodality deformable image registration for adaptive radiotherapy.

To develop a three-dimensional (3D) deformable head-and-neck (H&N) phantom with realistic tissue contrast for both kilovoltage (kV) and megavoltage (MV) imaging modalities and use it to objectively evaluate deformable image registration (DIR) algorithms. The phantom represents H&N patient anatomy. It is constructed from thermoplastic, which becomes pliable in boiling water, and hardened epoxy resin. Using a system of additives, the Hounsfield unit (HU) values of these materials were tuned to mimic anatomy for both kV and MV imaging. The phantom opens along a sagittal midsection to reveal radiotransparent markers, which were used to characterize the phantom deformation. The deformed and undeformed phantoms were scanned with kV and MV imaging modalities. Additionally, a calibration curve was created to change the HUs of the MV scans to be similar to kV HUs, (MC). The extracted ground-truth deformation was then compared to the results of two commercially available DIR algorithms, from Velocity Medical Solutions and mim software. The phantom produced a 3D deformation, representing neck flexion, with a magnitude of up to 8 mm and was able to represent tissue HUs for both kV and MV imaging modalities. The two tested deformation algorithms yielded vastly different results. For kV-kV registration, mim produced mean and maximum errors of 1.8 and 11.5 mm, respectively. These same numbers for Velocity were 2.4 and 7.1 mm, respectively. For MV-MV, kV-MV, and kV-MC Velocity produced similar mean and maximum error values. mim, however, produced gross errors for all three of these scenarios, with maximum errors ranging from 33.4 to 41.6 mm. The application of DIR across different imaging modalities is particularly difficult, due to differences in tissue HUs and the presence of imaging artifacts. For this reason, DIR algorithms must be validated specifically for this purpose. The developed H&N phantom is an effective tool for this purpose.

Diffusion-Weighted MRI for Response Assessment During Radiation Therapy of Lung Cancer

Diffusion-weighted MRI (DW-MRI) has recently been integrated into clinical practice to monitor tumor response to therapy in various tumor sites. However, there is only very limited data available on DW-MRI of lung cancer. The apparent diffusion coefficient (ADC), derived from DW-MRI images, is related to the mobility of water molecules and the tissue composition, including cell size, density, and integrity. In this study, changes of the ADC signal during chemoradiation therapy for lung cancer are assessed to determine the value of ADC as a predictor for response and radioresistance. Five patients with locally advanced non-small cell lung cancer (3 squamous cell cancer, 2 adenocarcinoma) underwent three series of MRI with DW-MRI and 4D CT imaging prior to, after 30 Gy, and after 60 Gy of chemoradiation therapy. Respiratory navigated DW-MRIs were acquired on a 1.5T scanner in axial planes with eight b-values ranging from 0-1000 s/mm2. Following multimodality deformable image registration, tumor (T) and affected lymph nodes (LNs) were contoured on CT, post gadolinium T1-weighted ultrafast gradient echo (VIBE), and ADC images for the purpose of comparing volumetric tumor regression and change of ADC signal with tumor response. Mean absolute ADC values and relative ADC and volume changes during radiation therapy are shown in the Table. At the end of therapy, four of five patients had partial response, and one had stable disease according to RECIST 1.1. There was no significant difference in ADC increase and VIBE or CT volume reduction between responding and non-responding patients or between tumor cell types in this small group.Oral Scientific Abstract 36; Table0 Gy0 Gy30 Gy30 Gy60 Gy60 GyTLNTLNTLNAbsolute ADC value change in m/s2 ADC (SD)1300 (423)1210 (310)1544 (388)1497 (488)1569 (469)1518 (301)ADC increase and volume decrease relative to baseline images in % ADC (SD)24 (14)22 (5)20 (9)27 (8) Volume VIBE (SD)37 (8)21 (23)60 (21)45 (29) Volume CT (SD)41 (7)14 (15)61 (6)47 (14) Open table in a new tab The preliminary results of this study show ADC signal increases early during therapy. While volume reduction continues during therapy, ADC signal does not significantly change between the 30 Gy and 60 Gy time points. Further research needs to be undertaken to investigate the predictive value of ADC signal change for tumor response, local control, and as a basis for adaptive radiation therapy planning.

A strategy for multimodal deformable image registration to integrate PET/MR into radiotherapy treatment planning

Background. Combined positron emission tomography (PET)/magnetic resonance imaging (MRI) is highly promising for biologically individualized radiotherapy (RT). Hence, the purpose of this work was to develop an accurate and robust registration strategy to integrate combined PET/MR data into RT treatment planning. Material and methods. Eight patient datasets consisting of an FDG PET/computed tomography (CT) and a subsequently acquired PET/MR of the head and neck (HN) region were available. Registration strategies were developed based on CT and MR data only, whereas the PET components were fused with the resulting deformation field. Following a rigid registration, deformable registration was performed with a transform parametrized by B-splines. Three different optimization metrics were investigated: global mutual information (GMI), GMI combined with a bending energy penalty (BEP) for regularization (GMI+ BEP) and localized mutual information with BEP (LMI+ BEP). Different quantitative registration quality measures were developed, including volumetric overlap and mean distance measures for structures segmented on CT and MR as well as anatomical landmark distances. Moreover, the local registration quality in the tumor region was assessed by the normalized cross correlation (NCC) of the two PET datasets. Results. LMI+ BEP yielded the most robust and accurate registration results. For GMI, GMI+ BEP and LMI+ BEP, mean landmark distances (standard deviations) were 23.9 mm (15.5 mm), 4.8 mm (4.0 mm) and 3.0 mm (1.0 mm), and mean NCC values (standard deviations) were 0.29 (0.29), 0.84 (0.14) and 0.88 (0.06), respectively. Conclusion. Accurate and robust multimodal deformable image registration of CT and MR in the HN region can be performed using a B-spline parametrized transform and LMI+ BEP as optimization metric. With this strategy, biologically individualized RT based on combined PET/MRI in terms of dose painting is possible.

A GPU-based Implementation of Multimodal Deformable Image Registration Based on Mutual Information or Bhattacharyya Distance

This paper describes a multimodal deformable image registration method on the GPU. It is a CUDA-based implementation of a paper by E. D’Agostino et. al, ‘’A viscous fluid model for multimodal non-rigid image registration using mutual information’’. In addition, we incorporate an alternative metric as opposed to mutual information, called Bhattacharyya Distance, in the recent work of Lou and Tannenbaum. This paper is accompanied with the source code, input data, parameters and output data that the authors used for validating the algorithm.

Symmetric deformable image registration via optimization of information theoretic measures

The use of information theoretic measures (ITMs) has been steadily growing in image processing, bioinformatics, and pattern classification. Although the ITMs have been extensively used in rigid and affine registration of multi-modal images, their computation and accuracy are critical issues in deformable image registration. Three important aspects of using ITMs in multi-modal deformable image registration are considered in this paper: computation, inverse consistency, and accuracy; a symmetric formulation of the deformable image registration problem through the computation of derivatives and resampling on both source and target images, and sufficient criteria for inverse consistency are presented for the purpose of achieving more accurate registration. The techniques of estimating ITMs are examined and analytical derivatives are derived for carrying out the optimization in a computationally efficient manner. ITMs based on Shannon’s and Renyi’s definitions are considered and compared. The obtained evaluation results via registration functions, and controlled deformable registration of multi-modal digital brain phantom and in vivo magnetic resonance brain images show the improved accuracy and efficiency of the developed formulation. The results also indicate that despite the recent favorable studies towards the use of ITMs based on Renyi’s definitions, these measures are seen not to provide improvements in this type of deformable registration as compared to ITMs based on Shannon’s definitions.

Model‐independent, multimodality deformable image registration by local matching of anatomical features and minimization of elastic energy

With respect to the demands of adaptive and 4D-radiotherapy applications, an algorithm is proposed for a fully automatic, multimodality deformable registration that follows the concept of translational relocation of regularly distributed image subvolumes governed by local anatomical features. Thereby, the problem of global deformable registration is broken down to multiple independent local registration steps which allows for straightforward parallelization of the algorithm. In a subsequent step, possible local misregistrations are corrected for by minimization of the elastic energy of the displacement field under consideration of image information. The final displacement field results from interpolation of the subvolume shift vectors. The algorithm can employ as a similarity measure both the correlation coefficient and mutual information. The latter allows the application to intermodality deformable registration problems. The typical calculation time on a modern multiprocessor PC is well below 1 min, which facilitates almost-interactive, "online" usage. CT-to-MRI and CT-to-cone-beam-CT registrations of head-and-neck data sets are presented, as well as inhale-to-exhale registrations of lung CT data sets. For quantitative evaluation of registration accuracy, a virtual thorax phantom was developed; additionally, a landmark-based evaluation on four lung respiratory-correlated CT data sets was performed. This consistently resulted in average registration residuals on the order of the voxel size or less (3D-residuals approximately 1-2 mm). Summarizing, the presented algorithm allows an accurate multimodality deformable registration with calculation times well below 1 min, and thus bears promise as a versatile basic tool in adaptive and 4D-radiotherapy applications.