Deformable Image Registration Algorithm Research Articles

Uncertainty estimation and evaluation of deformation image registration based convolutional neural networks

Objective. Fast and accurate deformable image registration (DIR), including DIR uncertainty estimation, is essential for safe and reliable clinical deployment. While recent deep learning models have shown promise in predicting DIR with its uncertainty, challenges persist in proper uncertainty evaluation and hyperparameter optimization for these methods. This work aims to develop and evaluate a model that can perform fast DIR and predict its uncertainty in seconds. Approach. This study introduces a novel probabilistic multi-resolution image registration model utilizing convolutional neural networks to estimate a multivariate normal distributed dense displacement field (DDF) in a multimodal image registration problem. To assess the quality of the DDF distribution predicted by the model, we propose a new metric based on the Kullback–Leibler divergence. The performance of our approach was evaluated against three other DIR algorithms (VoxelMorph, Monte Carlo dropout, and Monte Carlo B-spline) capable of predicting uncertainty. The evaluation of the models included not only the quality of the deformation but also the reliability of the estimated uncertainty. Our application investigated the registration of a treatment planning computed tomography (CT) to follow-up cone beam CT for daily adaptive radiotherapy. Main results. The hyperparameter tuning of the models showed a trade-off between the estimated uncertainty’s reliability and the deformation’s accuracy. In the optimal trade-off, our model excelled in contour propagation and uncertainty estimation (p <0.05) compared to existing uncertainty estimation models. We obtained an average dice similarity coefficient of 0.89 and a KL-divergence of 0.15. Significance. By addressing challenges in DIR uncertainty estimation and evaluation, our work showed that both the DIR and its uncertainty can be reliably predicted, paving the way for safe deployment in a clinical environment.

A comprehensive lung CT landmark pair dataset for evaluating deformable image registration algorithms.

Deformable image registration (DIR) is a key enabling technology in many diagnostic and therapeutic tasks, but often does not meet the required robustness and accuracy for supporting clinical tasks. This is in large part due to a lack of high-quality benchmark datasets by which new DIR algorithms can be evaluated. Our team was supported by the National Institute of Biomedical Imaging and Bioengineering to develop DIR benchmark dataset libraries for multiple anatomical sites, comprising of large numbers of highly accurate landmark pairs on matching blood vessel bifurcations. Here we introduce our lung CT DIR benchmark dataset library, which was developed to improve upon the number and distribution of landmark pairs in current public lung CT benchmark datasets. Thirty CT image pairs were acquired from several publicly available repositories as well as authors' institution with IRB approval. The data processing workflow included multiple steps: (1) The images were denoised. (2) Lungs, airways, and blood vessels were automatically segmented. (3) Bifurcations were directly detected on the skeleton of the segmented vessel tree. (4) Falsely identified bifurcations were filtered out using manually defined rules. (5) A DIR was used to project landmarks detected on the first image onto the second image of the image pair to form landmark pairs. (6) Landmark pairs were manually verified. This workflow resulted in an average of 1262 landmark pairs per image pair. Estimates of the landmark pair target registration error (TRE) using digital phantoms were 0.4mm±0.3mm. The data is published in Zenodo at https://doi.org/10.5281/zenodo.8200423. Instructions for use can be found at https://github.com/deshanyang/Lung-DIR-QA. The dataset library generated in this work is the largest of its kind to date and will provide researchers with a new and improved set of ground truth benchmarks for quantitatively validating DIR algorithms within the lung.

Evaluation and mitigation of deformable image registration uncertainties for MRI-guided adaptive radiotherapy.

We evaluate the performance of a deformable image registration (DIR) software package in registering abdominal magnetic resonance images (MRIs) and then develop a mechanical modeling method to mitigate detected DIR uncertainties. Three evaluation metrics, namely mean displacement to agreement (MDA), DICE similarity coefficient (DSC), and standard deviation of Jacobian determinants (STD-JD), are used to assess the multi-modality (MM), contour-consistency (CC), and image-intensity (II)-based DIR algorithms in the MIM software package, as well as an in-house developed, contour matching-based finite element method (CM-FEM). Furthermore, we develop a hybrid FEM registration technique to modify the displacement vector field of each MIM registration. The MIM and FEM registrations were evaluated on MRIs obtained from 10 abdominal cancer patients. One-tailed Wilcoxon-Mann-Whitney (WMW) tests were conducted to compare the MIM registrations with their FEM modifications. For the registrations performed with the MIM-CC, MIM-MM, MIM-II, and CM-FEM algorithms, their average MDAs are 0.62±0.27, 2.39±1.30, 3.07±2.42, 1.04±0.72mm, and average DSCs are 0.94±0.03, 0.80±0.12, 0.77±0.15, 0.90±0.11, respectively. The p-values of the WMW tests between the MIM registrations and their FEM modifications are less than 0.0084 for STD-JDs and greater than 0.87 for MDA and DSC. Among the three MIM DIR algorithms, MIM-CC shows the smallest errors in terms of MDA and DSC but exhibits significant Jacobian uncertainties in the interior regions of abdominal organs. The hybrid FEM technique effectively mitigates the Jacobian uncertainties in these regions.

Improving hybrid image and structure-based deformable image registration for large internal deformations

Objective. Deformable image registration (DIR) is a widely used technique in radiotherapy. Complex deformations, resulting from large anatomical changes, are a regular challenge. DIR algorithms generally seek a balance between capturing large deformations and preserving a smooth deformation vector field (DVF). We propose a novel structure-based term that can enhance the registration efficacy while ensuring a smooth DVF. Approach. The proposed novel similarity metric for controlling structures was introduced as a new term into a commercially available algorithm. Its performance was compared to the original algorithm using a dataset of 46 patients who received pelvic re-irradiation, many of which exhibited complex deformations. Main results. The mean Dice Similarity Coefficient (DSC) under the improved algorithm was 0.96, 0.94, 0.76, and 0.91 for bladder, rectum, colon, and bone respectively, compared to 0.69, 0.89, 0.62, and 0.88 for the original algorithm. The improvement was more pronounced for complex deformations. Significance. With this work, we have demonstrated that the proposed term is able to improve registration accuracy for complex cases while maintaining realistic deformations.

Performance Evaluation of Deformable Image Registration Systems – SmartAdapt® and Velocity™

Aim: To commission and validate commercial deformable image registration (DIR) systems (SmartAdapt® and Velocity™) using task group 132 (TG-132) digital phantom datasets. Additionally, the study compares and verifies the DIR algorithms of the two systems. Materials and Methods: TG-132 digital phantoms were obtained from the American Association of Physicists in Medicine website and imported into SmartAdapt® and Velocity™ systems for commissioning and validation. The registration results were compared with known shifts using rigid registrations and deformable registrations. Virtual head and neck phantoms obtained online (DIR Evaluation Project) and some selected clinical data sets from the department were imported into the two DIR systems. For both of these datasets, DIR was carried out between the source and target images, and the contours were then propagated from the source to the target image data set. The dice similarity coefficient (DSC), mean distance to agreement (MDA), and Jacobian determinant measures were utilised to evaluate the registration results. Results: The recommended criteria for commissioning and validation of DIR system from TG-132 was error &lt;0.5*voxel dimension (vd). Translation only registration: Both systems met TG-132 recommendations except computed tomography (CT)-positron emission tomography registration in both systems (Velocity ~1.1*vd, SmartAdapt ~1.6*vd). Translational and rotational registration: Both systems failed the criteria for all modalities (For velocity, error ranged from 0.6*vd [CT-CT registration] to 3.4*vd [CT-cone-beam CT (CBCT) registration]. For SmartAdapt® the range was 0.6*vd [CT-CBCT] to 3.6*vd [CT-CT]). Mean ± standard deviation for DSC, MDA and Jacobian metrics were used to compare the DIR results between SmartAdapt® and Velocity™. Conclusion: The DIR algorithms of SmartAdapt® and Velocity™ were commissioned and their deformation results were compared. Both systems can be used for clinical purpose. While there were only minimal differences between the two systems, Velocity™ provided lower values for parotids, bladder, rectum, and prostate (soft tissue) compared to SmartAdapt. However, for mandible, spinal cord, and femoral heads (rigid structures), both systems showed nearly identical results.

Deformable image registration for composite planned doses during adaptive radiation therapy

IntroductionSome patients have significant anatomic changes during radiotherapy, necessitating an adaptive repeat CT-simulation and re-planning. This yields two unique planning datasets that introduce uncertainty into total dose records. This study explored the impact of using deformable image registration (DIR) to spatially align repeat CT-simulation images and calculate total planned dose distributions. Materials & methodsData from 5 head-and-neck, 5 lung, and 5 sarcoma patients who had unanticipated re-planning during radiotherapy were analyzed in a treatment planning system (RayStation v6.1 RaySearch Laboratories). Total planned doses to normal tissues were calculated using two methods and the previously generated manual contours defined on each CT. The first method, termed ‘parameter addition’, simply sums the relevant DVH metrics from the initial and re-planned distributions without spatially registering the CTs. The second, termed ‘dose accumulation’, uses a validated hybrid contour/intensity-based DIR algorithm to deform initial CT and dose distribution onto the repeat CT and re-planning dose distribution. DVH metrics from the summed distribution on the repeat CT are then calculated. Dose differences for organs-at-risk between parameter addition and dose accumulation ≥100 cGy were assumed to be clinically relevant. To elucidate whether relevant differences were due to registration accuracy or contouring variability between CTs, the analysis was repeated using contours on the first CT and the same contours deformed to the repeat CT with DIR. ResultsFor all patients, high overall DIR accuracy was verified visually (qualitatively) and numerically (quantitatively) using image similarity and contour-based metrics. All head-and-neck and lung patients, and one sarcoma patient (11 of 15 total) had dose differences between parameter addition and dose accumulation ≥100 cGy, with absolute mean differences of 160 cGy (range 101–436 cGy) seen in 41 of 205 total DVH criteria. In 22 of these 41 criteria, these differences were attributed to contouring variability between CTs. After correcting for contouring variations using DIR, the mean absolute differences in 7 of these 22 criteria with a relevant result (across 6 patients) was 146 cGy (range 100–502 cGy). In only 4 DVH criteria, the DIR mapped contours had higher variations than the original contours. One lung patient had a DVH criteria exceeding the clinical dose constraint by 125 cGy with parameter addition, and with accurate DIR and dose accumulation, the criteria was actually 97 cGy lower than the constraint. ConclusionsThe use of DIR to generate total planned dose records revealed substantial dose differences in most cases compared to commonly used clinical methods (i.e. parameter addition), and altered the planned acceptance criteria in a minority. DIR is recommended to be used for future adaptive re-plans to generate total planned dose records and facilitate accurate re-contouring. More accurate dose records may also improve our understanding of clinical outcomes.

Deep learning based uncertainty prediction of deformable image registration for contour propagation and dose accumulation in online adaptive radiotherapy

Objective. Online adaptive radiotherapy aims to fully leverage the advantages of highly conformal therapy by reducing anatomical and set-up uncertainty, thereby alleviating the need for robust treatments. This requires extensive automation, among which is the use of deformable image registration (DIR) for contour propagation and dose accumulation. However, inconsistencies in DIR solutions between different algorithms have caused distrust, hampering its direct clinical use. This work aims to enable the clinical use of DIR by developing deep learning methods to predict DIR uncertainty and propagating it into clinically usable metrics. Approach. Supervised and unsupervised neural networks were trained to predict the Gaussian uncertainty of a given deformable vector field (DVF). Since both methods rely on different assumptions, their predictions differ and were further merged into a combined model. The resulting normally distributed DVFs can be directly sampled to propagate the uncertainty into contour and accumulated dose uncertainty. Main results. The unsupervised and combined models can accurately predict the uncertainty in the manually annotated landmarks on the DIRLAB dataset. Furthermore, for 5 patients with lung cancer, the propagation of the predicted DVF uncertainty into contour uncertainty yielded for both methods an expected calibration error of less than 3%. Additionally, the probabilisticly accumulated dose volume histograms (DVH) encompass well the accumulated proton therapy doses using 5 different DIR algorithms. It was additionally shown that the unsupervised model can be used for different DIR algorithms without the need for retraining. Significance. Our work presents first-of-a-kind deep learning methods to predict the uncertainty of the DIR process. The methods are fast, yield high-quality uncertainty estimates and are useable for different algorithms and applications. This allows clinics to use DIR uncertainty in their workflows without the need to change their DIR implementation.

An open-source nnU-net algorithm for automatic segmentation of MRI scans in the male pelvis for adaptive radiotherapy.

Adaptive MRI-guided radiotherapy (MRIgRT) requires accurate and efficient segmentation of organs and targets on MRI scans. Manual segmentation is time-consuming and variable, while deformable image registration (DIR)-based contour propagation may not account for large anatomical changes. Therefore, we developed and evaluated an automatic segmentation method using the nnU-net framework. The network was trained on 38 patients (76 scans) with localized prostate cancer and tested on 30 patients (60 scans) with localized prostate, metastatic prostate, or bladder cancer treated at a 1.5 T MRI-linac at our institution. The performance of the network was compared with the current clinical workflow based on DIR. The segmentation accuracy was evaluated using the Dice similarity coefficient (DSC), mean surface distance (MSD), and Hausdorff distance (HD) metrics. The trained network successfully segmented all 600 structures in the test set. High similarity was obtained for most structures, with 90% of the contours having a DSC above 0.9 and 86% having an MSD below 1mm. The largest discrepancies were found in the sigmoid and colon structures. Stratified analysis on cancer type showed that the best performance was seen in the same type of patients that the model was trained on (localized prostate). Especially in patients with bladder cancer, the performance was lower for the bladder and the surrounding organs. A complete automatic delineation workflow took approximately 1 minute. Compared with contour transfer based on the clinically used DIR algorithm, the nnU-net performed statistically better across all organs, with the most significant gain in using the nnU-net seen for organs subject to more considerable volumetric changes due to variation in the filling of the rectum, bladder, bowel, and sigmoid. We successfully trained and tested a network for automatically segmenting organs and targets for MRIgRT in the male pelvis region. Good test results were seen for the trained nnU-net, with test results outperforming the current clinical practice using DIR-based contour propagation at the 1.5 T MRI-linac. The trained network is sufficiently fast and accurate for clinical use in an online setting for MRIgRT. The model is provided as open-source.

Intelligent Interactive Deformable Image Registration for Online Adaptive Radiotherapy.

The goal of this study is to streamline the time-consuming contouring process in online adaptive radiotherapy (ART) by utilizing a deep learning-based interactive deformable image registration (DIR) algorithm. The objective is to minimize manual review and editing of automatically generated initial contours of organs-at-risk (OARs) and targets, thereby improving the efficiency and effectiveness of the treatment process. Our proposed method reforms the current DIR-based contour propagation method in clinical practice through the implementation of a deep learning-based interactive approach. The steps include: 1) generation of an initial deformable vector field (DVF) using a DL model, based on fixed and moving image pairs, resulting in the initial contours of OARs and targets; 2) clinician review/edit one the OAR/target contours as needed; 3) updated contour is sent to DL model to update the DVF and the remaining OARs/targets contours. Repeat this process until satisfactory contour qualities are achieved. We used the Open Access Series of Imaging Studies (OASIS) as the testbed, including 394 (train) and 20 (test) brain T1-weighted MRI scans, each containing 35 annotated organs. The U-Net architecture was employed to update the DVF from fixed/moving images, initial contours, and updated contours. We compared our approach to traditional manual editing without interaction and quantified the effort reduction using the added path length (APL) metric which is supposed to be proportional to the absolute time spent on the contour editing. We conducted paired t-test to show the significance. For comparison purpose, we assumed the clinicians edit the contours with the largest APL, i.e., the contours that require the most editing efforts. The editing effort, as measured by APL, was reduced by 18.5% to 25.4% with a mean of 23.3%, median of 23.6%, and standard deviation of 1.9%. The significance of the results was confirmed with a p-value of 1.47e-24. Our study demonstrates a significant reduction in editing effort, as measured by APL, compared to traditional manual contour editing. These results demonstrate the potential of our deep learning-based interactive approach to improve the efficiency and accuracy of the contouring process in clinical practice.

Adaptive Auto-Segmentation for MRI-Guided Online Adaptive Radiotherapy of Cervical Carcinoma

Accurate and efficient delineation of organs and targets on session images is critical in MRI-guided online adaptive radiotherapy (MRgOART). This study proposes a registration-guided deep learning image segmentation framework to assist online delineation of cervical carcinoma. A total of 300 T2-weighted MR images were acquired for patients with cervical carcinoma treated by a 1.5T Unity MR-Linac. The CTV, bladder, rectum, pelvic bone and femoral joints were delineated on each MRI by the same radiation oncologist. To overcome these obstacles to online MRI segmentation, we propose a registration-guided DL (RgDL) segmentation framework that integrates image registration algorithms and DL segmentation models. Firstly, the DL segmentation model was trained using nnU-net. Then, for each treatment fraction, the deformable image registration (DIR) algorithm generates initial contours from previous treatment fraction, which were used as guidance by DL model to obtain the accurate current segmentation. The segmentation accuracy of alone DIR, DL and RgDL were evaluated by dice similarity coefficients (DSC) and other distance-based metrics. Compared to the baseline approaches using the DIR and the DL alone, RgDL achieved a DSC of 91.12% on CTV, higher than DIR and DL alone by 15.54% and 10.13%. The DSC of RgDL were improved to 95.58%, 93.65%, 87.8% and 94.84% for bladder, pelvic bone, rectum and femoral joints, higher than DIR and DL alone by 9.61% on average. The proposed adaptive auto-segmentation method can achieve accurate and efficient segmentation and potentially overcome these obstacles to MRgOART.

Quantifying the Effect of 4-Dimensional Computed Tomography–Based Deformable Dose Accumulation on Representing Radiation Damage for Patients with Locally Advanced Non-Small Cell Lung Cancer Treated with Standard-Fractionated Intensity-Modulated Radiation Therapy

The aim of this study was to investigate the dosimetric and clinical effects of 4-dimensional computed tomography (4DCT)-based longitudinal dose accumulation in patients with locally advanced non-small cell lung cancer treated with standard-fractionated intensity-modulated radiation therapy (IMRT). Sixty-seven patients were retrospectively selected from a randomized clinical trial. Their original IMRT plan, planning and verification 4DCTs, and ∼4-month posttreatment follow-up CTs were imported into a commercial treatment planning system. Two deformable image registration algorithms were implemented for dose accumulation, and their accuracies were assessed. The planned and accumulated doses computed using average-intensity images or phase images were compared. At the organ level, mean lung dose and normal-tissue complication probability (NTCP) for grade ≥2 radiation pneumonitis were compared. At the region level, mean dose in lung subsections and the volumetric overlap between isodose intervals were compared. At the voxel level, the accuracy in estimating the delivered dose was compared by evaluating the fit of a dose versus radiographic image density change (IDC) model. The dose-IDC model fit was also compared for subcohorts based on the magnitude of NTCP difference (|ΔNTCP|) between planned and accumulated doses. Deformable image registration accuracy was quantified, and the uncertainty was considered for the voxel-level analysis. Compared with planned doses, accumulated doses on average resulted in <1-Gy lung dose increase and <2% NTCP increase (up to 8.2 Gy and 18.8% for a patient, respectively). Volumetric overlap of isodose intervals between the planned and accumulated dose distributions ranged from 0.01 to 0.93. Voxel-level dose-IDC models demonstrated a fit improvement from planned dose to accumulated dose (pseudo-R2 increased 0.0023) and a further improvement for patients with ≥2% |ΔNTCP| versus for patients with <2% |ΔNTCP|. With a relatively large cohort, robust image registrations, multilevel metric comparisons, and radiographic image-based evidence, we demonstrated that dose accumulation more accurately represents the delivered dose and can be especially beneficial for patients with greater longitudinal response.

Creating patient-specific digital phantoms with a longitudinal atlas for evaluating deformable CT-CBCT registration in adaptive lung radiotherapy.

Quality assurance of deformable image registration (DIR) is challenging because the ground truth is often unavailable. In addition, current approaches that rely on artificial transformations do not adequately resemble clinical scenarios encountered in adaptive radiotherapy. We developed an atlas-based method to create a variety of patient-specific serial digital phantoms with CBCT-like image quality to assess the DIR performance for longitudinal CBCT imaging data in adaptive lung radiotherapy. A library of deformations was created by extracting the longitudinal changes observed between a planning CT and weekly CBCT from an atlas of lung radiotherapy patients. The planning CT of an inquiry patient was first deformed by mapping the deformation pattern from a matched atlas patient, and subsequently appended with CBCT artifacts to imitate a weekly CBCT. Finally, a group of digital phantoms around an inquiry patient was produced to simulate a series of possible evolutions of tumor and adjacent normal structures. We validated the generated deformation vector fields (DVFs) to ensure numerically and physiologically realistic transformations. The proposed framework was applied to evaluate the performance of the DIR algorithm implemented in the commercial Eclipse treatment planning system in a retrospective study of eight inquiry patients. The generated DVFs were inverse consistent within less than 3mm and did not exhibit unrealistic folding. The deformation patterns adequately mimicked the observed longitudinal anatomical changes of the matched atlas patients. Worse Eclipse DVF accuracy was observed in regions of low image contrast or artifacts. The structure volumes exhibiting a DVF error magnitude of equal or more than 2mm ranged from 24.5% (spinal cord) to 69.2% (heart) and the maximum DVF error exceeded 5mm for all structures except the spinal cord. Contour-based evaluations showed a high degree of alignment with dice similarity coefficients above 0.8 in all cases, which underestimated the overall DVF accuracy within the structures. It is feasible to create and augment digital phantoms based on a particular patient of interest using multiple series of deformation patterns from matched patients in an atlas. This can provide a semi-automated procedure to complement the quality assurance of CT-CBCT DIR and facilitate the clinical implementation of image-guided and adaptive radiotherapy that involve longitudinal CBCT imaging studies.

Integration of operator-validated contours in deformable image registration for dose accumulation in radiotherapy.

Deformable image registration (DIR) is a core element of adaptive radiotherapy workflows, integrating daily contour propagation and/or dose accumulation in their design. Propagated contours are usually manually validated and may be edited, thereby locally invalidating the registration result. This means the registration cannot be used for dose accumulation. In this study we proposed and evaluated a novel multi-modal DIR algorithm that incorporated contour information to guide the registration. This integrates operator-validated contours with the estimated deformation vector field and warped dose. The proposed algorithm consisted of both a normalized gradient field-based data-fidelity term on the images and an optical flow data-fidelity term on the contours. The Helmholtz-Hodge decomposition was incorporated to ensure anatomically plausible deformations. The algorithm was validated for same- and cross-contrast Magnetic Resonance (MR) image registrations, Computed Tomography (CT) registrations, and CT-to-MR registrations for different anatomies, all based on challenging clinical situations. The contour-correspondence, anatomical fidelity, registration error, and dose warping error were evaluated. The proposed contour-guided algorithm considerably and significantly increased contour overlap, decreasing the mean distance to agreement by a factor of 1.3 to 13.7, compared to the best algorithm without contour-guidance. Importantly, the registration error and dose warping error decreased significantly, by a factor of 1.2 to 2.0. Our contour-guided algorithm ensured that the deformation vector field and warped quantitative information were consistent with the operator-validated contours. This provides a feasible semi-automatic strategy for spatially correct warping of quantitative information even in difficult and artefacted cases.

A CARDIAC MOTION MODEL TO EVALUATE INTRA-FRACTION DOSIMETRIC VARIATIONS IN RADIOTHERAPY TREATMENTS

Purpose: A cardiac cycle model was implemented to simulate cardiac motion during radiotherapy to evaluate the intra-fraction dosimetric impact on cardiac sub-structures comparing different planning techniques. Methods: Cardiac sub-structures were automatically contoured in 10 CTs acquired in deep inspiration breath hold (DIBH) by using a recently developed hierarchical-clustering atlas-based algorithm. A deformable image registration algorithm was used to simulate the cardiac motion cycle based on volume variations available in the literature. Two synthetic CTs were created and contoured simulating contraction and expansion during the cardiac cycle. Ninety radiotherapy plans were calculated using three radiotherapy paradigms: tangential fields planned as Linac-3D-CRT with a steep linear dose gradient toward the heart-modulated therapy with an intermediately steep concave gradient of intermediate-to-high doses toward the heart, planned as Linac-VMAT; modulated therapy with a steep concave gradient of intermediate-to-high doses toward the heart, planned as helical tomotherapy. Python scripts were developed for autocontouring, automatic creation of synthetic CTs and data extraction. Results: Comparison between paradigms shows that different constraints (maximal gradient toward heart/lung versus maximal sparing of contralateral breast/axilla) do not necessarily result in preferred or reduced heart sparing, but this depends more on individual anatomy. A planning paradigm with an intermediate-steepness dose gradient showed the best robustness against intra-fraction organ motion. Conclusions: Patient-specific organ motion models may reduce differences between planned and delivered RT and may thus help to refine dose–volume–toxicity models for cardiac sub-structures and, as a consequence, clinical constraints. Automatized plan recalculation on synthetic image sets might be used for robustness optimization and evaluation.

PD22-12 STATISTICAL SPATIAL MAPPING OF KIDNEY STONES BASED ON A KIDNEY ATLAS DERIVED FROM CT SCANS

You have accessJournal of UrologyCME1 Apr 2023PD22-12 STATISTICAL SPATIAL MAPPING OF KIDNEY STONES BASED ON A KIDNEY ATLAS DERIVED FROM CT SCANS Jiong Wu, Katherine Fischer, Yuemeng Li, Benjamin Schurhamer, Axel Largent, Joey Logan, Abhay Singh, Joanie Garratt, Justin Ziemba, Gregory Tasian, and Yong Fan Jiong WuJiong Wu More articles by this author , Katherine FischerKatherine Fischer More articles by this author , Yuemeng LiYuemeng Li More articles by this author , Benjamin SchurhamerBenjamin Schurhamer More articles by this author , Axel LargentAxel Largent More articles by this author , Joey LoganJoey Logan More articles by this author , Abhay SinghAbhay Singh More articles by this author , Joanie GarrattJoanie Garratt More articles by this author , Justin ZiembaJustin Ziemba More articles by this author , Gregory TasianGregory Tasian More articles by this author , and Yong FanYong Fan More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000003295.12AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Kidney stones vary in size and location. Precise characterization of the kidney stones’ size and location may facilitate effective treatment planning, and spatial mapping of the kidney stones of a group of patients offers a quantitative means to understand how kidney stones spatially distribute across subjects within the kidney. This study evaluated the feasibility to create a kidney atlas, assess individual kidney stones’ location and size, and statistically map kidney stones of a group of patients based on their CT scans using image segmentation and registration algorithms. METHODS: Non-contrast CT scans of 122 patients with kidney stones were retrospectively identified. A deep learning (DL) image segmentation model was built to segment kidneys automatically. Kidney stones were automatically identified as outlier voxels with Hounsfield Unit (HU) larger than mean plus 5-sigma of HUs of all voxels within the kidney, and a kidney stone image was generated to have unit intensity value at locations of stone voxels and zero otherwise. Bilateral kidney atlases were created by aligning all kidney images to a manually selected one using affine-transformation and diffeomorphic deformable image registration algorithms subsequently. The kidney stone image of each kidney was spatially transformed to the corresponding left or right kidney atlas with the same image deformation information obtained for creating the atlases. Statistical spatial maps of the kidney stones were generated for the bilateral kidneys separately to quantify frequency of the stones at every voxel of the kidney atlases. RESULTS: The DL image segmentation model segmented the kidneys with an average dice value of 0.96. The automatic stone identification algorithm detected individual kidney stones with 100% sensitivity. Statistical spatial maps of the kidney stones quantified spatial frequency of the kidney stones across subjects, with the highest frequency up to 6%. CONCLUSIONS: The statistical spatial mapping of kidney stones provide a quantitative means to characterize frequency of stones within the kidneys. The method can be used to assess kidney stones’ location for individual subjects with reference to group atlases that encode statistical spatial distribution information at a group level. Source of Funding: NIDDK P20 CHOP/ Penn Center for Machine Learning in Urology (P20DK127488)AUA Care Foundation and SPU Sushil Lacy Research Scholar Award (KMF) © 2023 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 209Issue Supplement 4April 2023Page: e669 Advertisement Copyright & Permissions© 2023 by American Urological Association Education and Research, Inc.MetricsAuthor Information Jiong Wu More articles by this author Katherine Fischer More articles by this author Yuemeng Li More articles by this author Benjamin Schurhamer More articles by this author Axel Largent More articles by this author Joey Logan More articles by this author Abhay Singh More articles by this author Joanie Garratt More articles by this author Justin Ziemba More articles by this author Gregory Tasian More articles by this author Yong Fan More articles by this author Expand All Advertisement PDF downloadLoading ...

Accuracy of deformable image registration-based intra-fraction motion management in Magnetic Resonance-guided radiotherapy

Intra-fraction motion management is key in Stereotactic Ablative Radiotherapy (SABR) gated delivery. This study assessed the accuracy of automatic tumor segmentation in the delivery of MR-guided radiotherapy (MRgRT) by comparing it to manual delineations performed by experienced observers. Twenty patients previously treated with MR-guided SABR for thoracic and abdominal tumors were included. Five observers with at least two years of experience in MRgRT manually delineated the gross tumor volume (GTV) for 20 patients on 240 frames of a cine MRI on a sagittal plane. Deformable Image Registration (DIR) based GTV contours were propagated using four different algorithms from a reference frame to subsequent frames.Geometrical analysis based on the Dice Similarity Coefficient (DSC), centroid distance and Hausdorff Distance (HDD) were performed to assess the inter-observer variability and the accuracy of automatic segmentation. A Confidence Value (CV) metric for the reliability of the tumor auto-contouring was also calculated. Inter-observer delineation variability resulted in mean DSC of 0.89, HDD of 5.8mm and centroid distance of 1.7mm. Tumor auto-contouring by the four DIR algorithms resulted in an excellent agreement with the manual delineations by the experienced observers. Mean DSC for each algorithm across all patients wasgreater than0.90, whereas the HDD and centroid distances were below 4.0mm and 1.5mm, respectively. The CV showed a strong correlation with the DSC. DIR-based auto-contouring in MRgRT exhibited a high level of agreement with the manual contouring performed by experts, allowing accurate gated delivery.

A model for gastrointestinal tract motility in a 4D imaging phantom of human anatomy.

Gastrointestinal (GI) tract motility is one of the main sources for intra/inter-fraction variability and uncertainty in radiation therapy for abdominal targets. Models for GI motility can improve the assessment of delivered dose and contribute to the development, testing, and validation of deformable image registration (DIR) and dose-accumulation algorithms. To implement GI tract motion in the 4D extended cardiac-torso (XCAT) digital phantom of human anatomy. Motility modes that exhibit large amplitude changes in the diameter of the GI tract and may persist over timescales comparable to online adaptive planning and radiotherapy delivery were identified based on literature research. Search criteria included amplitude changes larger than planning risk volume expansions and durations of the order of tens of minutes. The following modes were identified: peristalsis, rhythmic segmentation, high amplitude propagating contractions (HAPCs), and tonic contractions. Peristalsis and rhythmic segmentations were modeled by traveling and standing sinusoidal waves. HAPCs and tonic contractions were modeled by traveling and stationary Gaussian waves. Wave dispersion in the temporal and spatial domain was implemented by linear, exponential, and inverse power law functions. Modeling functions were applied to the control points of the nonuniform rational B-spline surfaces defined in the reference XCAT library. GI motility was combined with the cardiac and respiratory motions available in the standard 4D-XCAT phantom. Default model parameters were estimated based on the analysis of cine MRI acquisitions in 10 patients treated in a 1.5T MR-linac. We demonstrate the ability to generate realistic 4D multimodal images that simulate GI motility combined with respiratory and cardiac motion. All modes of motility, except tonic contractions, were observed in the analysis of our cine MRI acquisitions. Peristalsis was the most common. Default parameters estimated from cine MRI were used as initial values for simulation experiments. It is shown that in patients undergoing stereotactic body radiotherapy for abdominal targets, the effects of GI motility can be comparable or larger than the effects of respiratory motion. The digital phantom provides realistic models to aid in medical imaging and radiation therapy research. The addition of GI motility will further contribute to the development, testing, and validation of DIR and dose accumulation algorithms for MR-guided radiotherapy.

Applying Multi-Metric Deformable Image Registration for Dose Accumulation in Combined Cervical Cancer Radiotherapy

(1) Purpose: Challenges remain in dose accumulation for cervical cancer radiotherapy combined with external beam radiotherapy (EBRT) and brachytherapy (BT) as there are many large and complex organ deformations between different treatments. This study aims to improve deformable image registration (DIR) accuracy with the introduction of multi-metric objectives for dose accumulation of EBRT and BT. (2) Materials and methods: Twenty cervical cancer patients treated with EBRT (45-50 Gy/25 fractions) and high-dose-rate BT (≥20 Gy in 4 fractions) were included for DIR. The multi-metric DIR algorithm included an intensity-based metric, three contour-based metrics, and a penalty term. Nonrigid B-spine transformation was used to transform the planning CT images from EBRT to the first BT, with a six-level resolution registration strategy. To evaluate its performance, the multi-metric DIR was compared with a hybrid DIR provided by commercial software. The DIR accuracy was measured by the Dice similarity coefficient (DSC) and Hausdorff distance (HD) between deformed and reference organ contours. The accumulated maximum dose of 2 cc (D2cc) of the bladder and rectum was calculated and compared to simply addition of D2cc from EBRT and BT (ΔD2cc). (3) Results: The mean DSC of all organ contours for the multi-metric DIR were significantly higher than those for the hybrid DIR (p ≤ 0.011). In total, 70% of patients had DSC > 0.8 using the multi-metric DIR, while 15% of patients had DSC > 0.8 using the commercial hybrid DIR. The mean ΔD2cc of the bladder and rectum for the multi-metric DIR were 3.25 ± 2.29 and 3.54 ± 2.02 GyEQD2, respectively, whereas those for the hybrid DIR were 2.68 ± 2.56 and 2.32 ± 3.25 GyEQD2, respectively. The multi-metric DIR resulted in a much lower proportion of unrealistic D2cc than the hybrid DIR (2.5% vs. 17.5%). (4) Conclusions: Compared with the commercial hybrid DIR, the introduced multi-metric DIR significantly improved the registration accuracy and resulted in a more reasonable accumulated dose distribution.

Deep learning based direct segmentation assisted by deformable image registration for cone-beam CT based auto-segmentation for adaptive radiotherapy

Cone-beam CT (CBCT)-based online adaptive radiotherapy calls for accurate auto-segmentation to reduce the time cost for physicians. However, deep learning (DL)-based direct segmentation of CBCT images is a challenging task, mainly due to the poor image quality and lack of well-labelled large training datasets. Deformable image registration (DIR) is often used to propagate the manual contours on the planning CT (pCT) of the same patient to CBCT. In this work, we undertake solving the problems mentioned above with the assistance of DIR. Our method consists of three main components. First, we use deformed pCT contours derived from multiple DIR methods between pCT and CBCT as pseudo labels for initial training of the DL-based direct segmentation model. Second, we use deformed pCT contours from another DIR algorithm as influencer volumes to define the region of interest for DL-based direct segmentation. Third, the initially trained DL model is further fine-tuned using a smaller set of true labels. Nine patients are used for model evaluation. We found that DL-based direct segmentation on CBCT without influencer volumes has much poorer performance compared to DIR-based segmentation. However, adding deformed pCT contours as influencer volumes in the direct segmentation network dramatically improves segmentation performance, reaching the accuracy level of DIR-based segmentation. The DL model with influencer volumes can be further improved through fine-tuning using a smaller set of true labels, achieving mean Dice similarity coefficient of 0.86, Hausdorff distance at the 95th percentile of 2.34 mm, and average surface distance of 0.56 mm. A DL-based direct CBCT segmentation model can be improved to outperform DIR-based segmentation models by using deformed pCT contours as pseudo labels and influencer volumes for initial training, and by using a smaller set of true labels for model fine tuning.

RMSim: controlled respiratory motion simulation on static patient scans

Objective. This work aims to generate realistic anatomical deformations from static patient scans. Specifically, we present a method to generate these deformations/augmentations via deep learning driven respiratory motion simulation that provides the ground truth for validating deformable image registration (DIR) algorithms and driving more accurate deep learning based DIR. Approach. We present a novel 3D Seq2Seq deep learning respiratory motion simulator (RMSim) that learns from 4D-CT images and predicts future breathing phases given a static CT image. The predicted respiratory patterns, represented by time-varying displacement vector fields (DVFs) at different breathing phases, are modulated through auxiliary inputs of 1D breathing traces so that a larger amplitude in the trace results in more significant predicted deformation. Stacked 3D-ConvLSTMs are used to capture the spatial-temporal respiration patterns. Training loss includes a smoothness loss in the DVF and mean-squared error between the predicted and ground truth phase images. A spatial transformer deforms the static CT with the predicted DVF to generate the predicted phase image. 10-phase 4D-CTs of 140 internal patients were used to train and test RMSim. The trained RMSim was then used to augment a public DIR challenge dataset for training VoxelMorph to show the effectiveness of RMSim-generated deformation augmentation. Main results. We validated our RMSim output with both private and public benchmark datasets (healthy and cancer patients). The structure similarity index measure (SSIM) for predicted breathing phases and ground truth 4D CT images was 0.92 ± 0.04, demonstrating RMSim’s potential to generate realistic respiratory motion. Moreover, the landmark registration error in a public DIR dataset was improved from 8.12 ± 5.78 mm to 6.58mm ± 6.38 mm using RMSim-augmented training data. Significance. The proposed approach can be used for validating DIR algorithms as well as for patient-specific augmentations to improve deep learning DIR algorithms. The code, pretrained models, and augmented DIR validation datasets will be released at https://github.com/nadeemlab/SeqX2Y.