Deformable Registration Algorithm Research Articles

DiffuseRT: predicting likely anatomical deformations of patients undergoing radiotherapy

Objective. Predicting potential deformations of patients can improve radiotherapy treatment planning. Here, we introduce new deep-learning models that predict likely anatomical changes during radiotherapy for head and neck cancer patients. Approach. Denoising diffusion probabilistic models (DDPMs) were developed to generate fraction-specific anatomical changes based on a reference cone-beam CT (CBCT), the fraction number and the dose distribution delivered. Three distinct DDPMs were developed: (1) the image model was trained to directly generate likely future CBCTs, (2) the deformable vector field (DVF) model was trained to generate DVFs that deform a reference CBCT and (3) the hybrid model was trained similarly to the DVF model, but without relying on an external deformable registration algorithm. The models were trained on 9 patients with longitudinal CBCT images (224 CBCTs) and evaluated on 5 patients (152 CBCTs). Results. The generated images mainly exhibited random positioning shifts and small anatomical changes for early fractions. For later fractions, all models predicted weight losses in accordance with the training data. The distributions of volume and position changes of the body, esophagus, and parotids generated with the image and hybrid models were more similar to the ground truth distribution than the DVF model, evident from the lower Wasserstein distance achieved with the image (0.33) and hybrid model (0.30) compared to the DVF model (0.36). Generating several images for the same fraction did not yield the expected variability since the ground truth anatomical changes were only in 76% of the fractions within the 95% bounds predicted with the best model. Using the generated images for robust optimization of simplified proton therapy plans improved the worst-case clinical target volume V95 with 7% compared to optimizing with 3 mm set-up robustness while maintaining a similar integral dose. Significance. The newly developed DDPMs generate distributions similar to the real anatomical changes and have the potential to be used for robust anatomical optimization.

Deformable registration of magnetic resonance images using unsupervised deep learning in neuro-/radiation oncology

PurposeAccurate deformable registration of magnetic resonance imaging (MRI) scans containing pathologies is challenging due to changes in tissue appearance. In this paper, we developed a novel automated three-dimensional (3D) convolutional U-Net based deformable image registration (ConvUNet-DIR) method using unsupervised learning to establish correspondence between baseline pre-operative and follow-up MRI scans of patients with brain glioma.MethodsThis study involved multi-parametric brain MRI scans (T1, T1-contrast enhanced, T2, FLAIR) acquired at pre-operative and follow-up time for 160 patients diagnosed with glioma, representing the BraTS-Reg 2022 challenge dataset. ConvUNet-DIR, a deep learning-based deformable registration workflow using 3D U-Net style architecture as a core, was developed to establish correspondence between the MRI scans. The workflow consists of three components: (1) the U-Net learns features from pairs of MRI scans and estimates a mapping between them, (2) the grid generator computes the sampling grid based on the derived transformation parameters, and (3) the spatial transformation layer generates a warped image by applying the sampling operation using interpolation. A similarity measure was used as a loss function for the network with a regularization parameter limiting the deformation. The model was trained via unsupervised learning using pairs of MRI scans on a training data set (n = 102) and validated on a validation data set (n = 26) to assess its generalizability. Its performance was evaluated on a test set (n = 32) by computing the Dice score and structural similarity index (SSIM) quantitative metrics. The model’s performance also was compared with the baseline state-of-the-art VoxelMorph (VM1 and VM2) learning-based algorithms.ResultsThe ConvUNet-DIR model showed promising competency in performing accurate 3D deformable registration. It achieved a mean Dice score of 0.975 ± 0.003 and SSIM of 0.908 ± 0.011 on the test set (n = 32). Experimental results also demonstrated that ConvUNet-DIR outperformed the VoxelMorph algorithms concerning Dice (VM1: 0.969 ± 0.006 and VM2: 0.957 ± 0.008) and SSIM (VM1: 0.893 ± 0.012 and VM2: 0.857 ± 0.017) metrics. The time required to perform a registration for a pair of MRI scans is about 1 s on the CPU.ConclusionsThe developed deep learning-based model can perform an end-to-end deformable registration of a pair of 3D MRI scans for glioma patients without human intervention. The model could provide accurate, efficient, and robust deformable registration without needing pre-alignment and labeling. It outperformed the state-of-the-art VoxelMorph learning-based deformable registration algorithms and other supervised/unsupervised deep learning-based methods reported in the literature.

C2FResMorph: A high-performance framework for unsupervised 2D medical image registration

Deformable medical image registration is an important precursor task for surgical automation, while enhancing the registration performance of 2D medical images remains a challenging work. Existing methods primarily minimize the similarity loss between image pairs as the main optimization objective, leading to limited registration accuracy and a lack of pixel matching. Moreover, the scarcity of informative features in 2D images often results in overfitting on the training set, hampering generalization. To address these issues, we propose C2FResMorph, a learning-based deformable registration algorithm specifically designed for 2D medical images. C2FResMorph employs a two-stage framework that improves registration accuracy and preserves topology during deformation in a coarse-to-fine manner. Inside the framework, by leveraging the convolutional neural network’s locality and the multi-head self-attention mechanism’s globality, a ResMorph registration network is designed. Additionally, the integration of residual image knowledge addresses deformation folding in 2D image registration, enhancing the preservation of local structures and improving generalization. Experimental evaluations on three datasets demonstrate that C2FResMorph outperforms existing learning-based methods in terms of accuracy, generalization ability for 2D medical image registration, and also retains the efficiency advantages.

Utilizing Artificial Intelligence-Based Deformable Registration for Global and Layer-Specific Cardiac MRI Strain Analysis in Healthy Children and Young Adults

The absence of published reference values for multilayer-specific strain measurement using cardiac magnetic resonance (CMR) in young healthy individuals limits its use. This study aimed to establish normal global and layer-specific strain values in healthy children and young adults using a deformable registration algorithm (DRA). A retrospective study included 131 healthy children and young adults (62 males and 69 females) with a mean age of 16.6±3.9 years. CMR examinations were conducted using 1.5T scanners, and strain analysis was performed using TrufiStrain research prototype software (Siemens Healthineers, Erlangen, Germany). Global and layer-specific strain parameters were extracted from balanced Steady-state free precession cine images. Statistical analyses were conducted to evaluate the impact of demographic variables on strain measurements. The peak global longitudinal strain (LS) was -16.0±3.0%, peak global radial strain (RS) was 29.9±6.3%, and peak global circumferential strain (CS) was -17.0±1.8%. Global LS differed significantly between males and females. Transmural strain analysis showed a consistent pattern of decreasing LS and CS from endocardium to epicardium, while radial strain increased. Basal-to-apical strain distribution exhibited decreasing LS and increasing CS in both global and layer-specific analysis. This study uses DRA to provide reference values for global and layer-specific strain in healthy children and young adults. The study highlights the impact of sex and age on LS and body mass index on RS. These insights are vital for future cardiac assessments in children, particularly for early detection of heart diseases.

Intelligent Interactive Deformable Image Registration for Online Adaptive Radiotherapy.

The goal of this study is to streamline the time-consuming contouring process in online adaptive radiotherapy (ART) by utilizing a deep learning-based interactive deformable image registration (DIR) algorithm. The objective is to minimize manual review and editing of automatically generated initial contours of organs-at-risk (OARs) and targets, thereby improving the efficiency and effectiveness of the treatment process. Our proposed method reforms the current DIR-based contour propagation method in clinical practice through the implementation of a deep learning-based interactive approach. The steps include: 1) generation of an initial deformable vector field (DVF) using a DL model, based on fixed and moving image pairs, resulting in the initial contours of OARs and targets; 2) clinician review/edit one the OAR/target contours as needed; 3) updated contour is sent to DL model to update the DVF and the remaining OARs/targets contours. Repeat this process until satisfactory contour qualities are achieved. We used the Open Access Series of Imaging Studies (OASIS) as the testbed, including 394 (train) and 20 (test) brain T1-weighted MRI scans, each containing 35 annotated organs. The U-Net architecture was employed to update the DVF from fixed/moving images, initial contours, and updated contours. We compared our approach to traditional manual editing without interaction and quantified the effort reduction using the added path length (APL) metric which is supposed to be proportional to the absolute time spent on the contour editing. We conducted paired t-test to show the significance. For comparison purpose, we assumed the clinicians edit the contours with the largest APL, i.e., the contours that require the most editing efforts. The editing effort, as measured by APL, was reduced by 18.5% to 25.4% with a mean of 23.3%, median of 23.6%, and standard deviation of 1.9%. The significance of the results was confirmed with a p-value of 1.47e-24. Our study demonstrates a significant reduction in editing effort, as measured by APL, compared to traditional manual contour editing. These results demonstrate the potential of our deep learning-based interactive approach to improve the efficiency and accuracy of the contouring process in clinical practice.

Robustness Analysis of Reference Surfaces for Surface Guided Radiation Therapy of the Breast

Surface guided radiation therapy (SGRT) implements an optical imaging system in radiation therapy for positioning and motion management. This system projects visible light onto a patient and the reflected light is used to generate 3D positional information so that clinicians can accurately reproduce body positions. Patient setup shifts are calculated with six degrees of freedom by a registration algorithm comparing a reference surface (RS) of the patient to a live surface map of the patient on treatment day. SGRT has been an effective tool in daily localization for the treatment of breast cancer patients. It is common for patients to have multiple RS throughout the course of their treatment to account for anatomical variation between fractions. We sought to evaluate the robustness of reference surfaces and vendor specific algorithms used for SGRT. At our institution, positional shift data for five patients treated for right-sided breast cancer were retrospectively analyzed. SGRT performance was compared between RS using bilateral breasts or a single ipsilateral breast. Shift parameters were calculated over the entire treatment course for all patients with a vendor supplied software tool that offers rigid and deformable registration algorithms. The deformable algorithm was used for treatment setups, with the treatment RS encompassing both breasts plus a margin. Two robustness tests were carried out: 1) a trimmed down RS encompassing just the ipsilateral breast and 2) a comparison of deformable vs rigid registration of the clinically used RS. After obtaining translational and angular shift data, the absolute mean differences between shifts were calculated to compare differences between RS size and algorithm performance. On average, 1.4 new RS were created per patient guided by weekly radiographic imaging to adjust for anatomical changes. The absolute value of the average of the discrepancies between shifts using the clinical RS subtracted from the trimmed external (89 fractions) were <1mm and 1° and the maximum differences were: Lateral: 2.6mm, Longitudinal: 1.4mm, Vertical: 1.1mm, Yaw: 1.1°, Roll: 1.5°, Pitch: 1.7°. Discrepancies between tracking algorithms (83 fractions) were <1.5mm and 1° and the maximum differences were: Lateral: 3.4mm, Longitudinal: 3.5mm, Vertical: 2.0mm, Yaw: 2.4°, Roll: 2.7°, Pitch: 1.9°. Clinically negligible mean discrepancies were observed for both robustness tests showing that neither the reference surface size nor the algorithms investigated caused systematic variations in the shifts for this group of patients. Maximum discrepancies of up to 3 mm and 3° were found between the algorithms, which indicate some variation, but within clinical tolerance. Overall, different selection of reference surfaces and algorithms had a minor effect on clinical shifts for SGRT of the breast.

Adaptive Auto-Segmentation for MRI-Guided Online Adaptive Radiotherapy of Cervical Carcinoma

Accurate and efficient delineation of organs and targets on session images is critical in MRI-guided online adaptive radiotherapy (MRgOART). This study proposes a registration-guided deep learning image segmentation framework to assist online delineation of cervical carcinoma. A total of 300 T2-weighted MR images were acquired for patients with cervical carcinoma treated by a 1.5T Unity MR-Linac. The CTV, bladder, rectum, pelvic bone and femoral joints were delineated on each MRI by the same radiation oncologist. To overcome these obstacles to online MRI segmentation, we propose a registration-guided DL (RgDL) segmentation framework that integrates image registration algorithms and DL segmentation models. Firstly, the DL segmentation model was trained using nnU-net. Then, for each treatment fraction, the deformable image registration (DIR) algorithm generates initial contours from previous treatment fraction, which were used as guidance by DL model to obtain the accurate current segmentation. The segmentation accuracy of alone DIR, DL and RgDL were evaluated by dice similarity coefficients (DSC) and other distance-based metrics. Compared to the baseline approaches using the DIR and the DL alone, RgDL achieved a DSC of 91.12% on CTV, higher than DIR and DL alone by 15.54% and 10.13%. The DSC of RgDL were improved to 95.58%, 93.65%, 87.8% and 94.84% for bladder, pelvic bone, rectum and femoral joints, higher than DIR and DL alone by 9.61% on average. The proposed adaptive auto-segmentation method can achieve accurate and efficient segmentation and potentially overcome these obstacles to MRgOART.

Deep Learning-Based Multi-Modality Segmentation of Primary Gross Tumor Volume in CT and MRI for Nasopharyngeal Carcinoma

The delineation of primary gross tumor volume (GTV) of nasopharyngeal carcinoma (NPC) is an essential step for radiotherapy planning. In clinical practice, radiation oncologists manually delineate the GTV in planning CT with the help of diagnostic MRI. This is because NPC tumors are closely adjacent to many important anatomic structures, and CT and MRI provide complementary strength to accurately determine the tumor extension boundary. Manual delineation is time-consuming with the potential registration errors between MRI and CT decreasing the delineation accuracy. In this study, we propose a fully automated GTV segmentation method based on CT and MRI by first aligning MRI to CT, and then, segmenting the GTV using a multi-modality deep learning model. We collected 104 nasopharyngeal carcinoma patients with both planning CT and diagnostic MRI scans (T1 & T2 phases). An experienced radiation oncologists manually delineated the GTV, which was further examined by another senior radiation oncologist. Then, a coarse to fine cross-modality registration from MRI to CT was conducted as follows: (1) A rigid transformation was performed on MRI to roughly align MRI to CT with similar anatomic position. (2) Then, the region of interest (RoI) on both CT and rigid-transformed MRI were cropped. (3) A leading cross-modality deformable registration algorithm, named DEEDS, was applied on the cropped MRI and CT RoIs to find an accurate local alignment. Next, using CT and registered MRI as the combined input, a multi-modality deep segmentation network based on nnUNet was trained to generate the GTV prediction. 20% patients were randomly selected as the unseen testing set to quantitatively evaluate the performance. The quantitative NPC GTV segmentation performance is summarized in Table 1. The deep segmentation model using CT alone achieved reasonable high performance with 76.6% Dice score and 1.34mm average surface distance (ASD). When both CT and registered MRI were used, the segmentation model further improved the performance by 0.9% Dice score increase and 11% relative ASD error reduction, demonstrating the complementary strength of CT and MRI in determining NPC GTV. Notably, the achieved 77.5% Dice score and 1.19mm ASD by the multimodality model is among the top performing results reported in recent automatic NPC GTV segmentation using either CT or MRI modality. We developed a fully automated multi-modal deep-learning model for NPC GTV segmentation. The developed model can segment the NPC GTV in high accuracy. With further optimization and validation, this automated model has potential to standardize the NPC GTV segmentation and significantly decrease the workload of radiation oncologists in clinical practice.

Quantifying the Effect of 4-Dimensional Computed Tomography–Based Deformable Dose Accumulation on Representing Radiation Damage for Patients with Locally Advanced Non-Small Cell Lung Cancer Treated with Standard-Fractionated Intensity-Modulated Radiation Therapy

The aim of this study was to investigate the dosimetric and clinical effects of 4-dimensional computed tomography (4DCT)-based longitudinal dose accumulation in patients with locally advanced non-small cell lung cancer treated with standard-fractionated intensity-modulated radiation therapy (IMRT). Sixty-seven patients were retrospectively selected from a randomized clinical trial. Their original IMRT plan, planning and verification 4DCTs, and ∼4-month posttreatment follow-up CTs were imported into a commercial treatment planning system. Two deformable image registration algorithms were implemented for dose accumulation, and their accuracies were assessed. The planned and accumulated doses computed using average-intensity images or phase images were compared. At the organ level, mean lung dose and normal-tissue complication probability (NTCP) for grade ≥2 radiation pneumonitis were compared. At the region level, mean dose in lung subsections and the volumetric overlap between isodose intervals were compared. At the voxel level, the accuracy in estimating the delivered dose was compared by evaluating the fit of a dose versus radiographic image density change (IDC) model. The dose-IDC model fit was also compared for subcohorts based on the magnitude of NTCP difference (|ΔNTCP|) between planned and accumulated doses. Deformable image registration accuracy was quantified, and the uncertainty was considered for the voxel-level analysis. Compared with planned doses, accumulated doses on average resulted in <1-Gy lung dose increase and <2% NTCP increase (up to 8.2 Gy and 18.8% for a patient, respectively). Volumetric overlap of isodose intervals between the planned and accumulated dose distributions ranged from 0.01 to 0.93. Voxel-level dose-IDC models demonstrated a fit improvement from planned dose to accumulated dose (pseudo-R2 increased 0.0023) and a further improvement for patients with ≥2% |ΔNTCP| versus for patients with <2% |ΔNTCP|. With a relatively large cohort, robust image registrations, multilevel metric comparisons, and radiographic image-based evidence, we demonstrated that dose accumulation more accurately represents the delivered dose and can be especially beneficial for patients with greater longitudinal response.

Influence of air mapping errors on the dosimetric accuracy of prostate CBCT-guided online adaptive radiation therapy.

CBCT-guided online adaptive radiotherapy (oART) plans presently utilize daily synthetic CTs (sCT) that are automatically generated using deformable registration algorithms. These algorithms may have poor performance at reproducing variable volumes of gas present during treatment. Therefore, we have analyzed the air mapping error between the daily CBCTs and the corresponding sCT and explored its dosimetric effect on oART plan calculation. Abdominopelvic air volume was contoured on both the daily CBCT images and the corresponding synthetic images for 207 online adaptive pelvic treatments. Air mapping errors were tracked over all fractions. For two case studies representing worst case scenarios, dosimetric effects of air mapping errors were corrected in the sCT images using the daily CBCT air contours, then recalculating dose. Dose volume histogram statistics and 3D gamma passing rates were used to compare the original and air-corrected sCT-based dose calculations. All analyzed patients showed observable air pocket contour differences between the sCT and the CBCT images. The largest air volume difference observed in daily CBCT images for a given patient was 276.3 cc, a difference of more than 386% compared to the sCT. For the two case studies, the largest observed change in DVH metrics was a 2.6% reduction in minimum PTV dose, with all other metrics varying by less than 1.5%. 3D gamma passing rates using 1%/1mm criteria were above 90% when comparing the uncorrected and corrected dose distributions. Current CBCT-based oART workflow can lead to inaccuracies in the mapping of abdominopelvic air pockets from daily CBCT to the sCT images used for the optimization and calculation of the adaptive plan. Despite the large observed mapping errors, the dosimetric effects of such differences on the accuracy of the adapted plan dose calculation are unlikely to cause differences greater than 3% for prostate treatments.

A CARDIAC MOTION MODEL TO EVALUATE INTRA-FRACTION DOSIMETRIC VARIATIONS IN RADIOTHERAPY TREATMENTS

Purpose: A cardiac cycle model was implemented to simulate cardiac motion during radiotherapy to evaluate the intra-fraction dosimetric impact on cardiac sub-structures comparing different planning techniques. Methods: Cardiac sub-structures were automatically contoured in 10 CTs acquired in deep inspiration breath hold (DIBH) by using a recently developed hierarchical-clustering atlas-based algorithm. A deformable image registration algorithm was used to simulate the cardiac motion cycle based on volume variations available in the literature. Two synthetic CTs were created and contoured simulating contraction and expansion during the cardiac cycle. Ninety radiotherapy plans were calculated using three radiotherapy paradigms: tangential fields planned as Linac-3D-CRT with a steep linear dose gradient toward the heart-modulated therapy with an intermediately steep concave gradient of intermediate-to-high doses toward the heart, planned as Linac-VMAT; modulated therapy with a steep concave gradient of intermediate-to-high doses toward the heart, planned as helical tomotherapy. Python scripts were developed for autocontouring, automatic creation of synthetic CTs and data extraction. Results: Comparison between paradigms shows that different constraints (maximal gradient toward heart/lung versus maximal sparing of contralateral breast/axilla) do not necessarily result in preferred or reduced heart sparing, but this depends more on individual anatomy. A planning paradigm with an intermediate-steepness dose gradient showed the best robustness against intra-fraction organ motion. Conclusions: Patient-specific organ motion models may reduce differences between planned and delivered RT and may thus help to refine dose–volume–toxicity models for cardiac sub-structures and, as a consequence, clinical constraints. Automatized plan recalculation on synthetic image sets might be used for robustness optimization and evaluation.

PD22-12 STATISTICAL SPATIAL MAPPING OF KIDNEY STONES BASED ON A KIDNEY ATLAS DERIVED FROM CT SCANS

You have accessJournal of UrologyCME1 Apr 2023PD22-12 STATISTICAL SPATIAL MAPPING OF KIDNEY STONES BASED ON A KIDNEY ATLAS DERIVED FROM CT SCANS Jiong Wu, Katherine Fischer, Yuemeng Li, Benjamin Schurhamer, Axel Largent, Joey Logan, Abhay Singh, Joanie Garratt, Justin Ziemba, Gregory Tasian, and Yong Fan Jiong WuJiong Wu More articles by this author , Katherine FischerKatherine Fischer More articles by this author , Yuemeng LiYuemeng Li More articles by this author , Benjamin SchurhamerBenjamin Schurhamer More articles by this author , Axel LargentAxel Largent More articles by this author , Joey LoganJoey Logan More articles by this author , Abhay SinghAbhay Singh More articles by this author , Joanie GarrattJoanie Garratt More articles by this author , Justin ZiembaJustin Ziemba More articles by this author , Gregory TasianGregory Tasian More articles by this author , and Yong FanYong Fan More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000003295.12AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Kidney stones vary in size and location. Precise characterization of the kidney stones’ size and location may facilitate effective treatment planning, and spatial mapping of the kidney stones of a group of patients offers a quantitative means to understand how kidney stones spatially distribute across subjects within the kidney. This study evaluated the feasibility to create a kidney atlas, assess individual kidney stones’ location and size, and statistically map kidney stones of a group of patients based on their CT scans using image segmentation and registration algorithms. METHODS: Non-contrast CT scans of 122 patients with kidney stones were retrospectively identified. A deep learning (DL) image segmentation model was built to segment kidneys automatically. Kidney stones were automatically identified as outlier voxels with Hounsfield Unit (HU) larger than mean plus 5-sigma of HUs of all voxels within the kidney, and a kidney stone image was generated to have unit intensity value at locations of stone voxels and zero otherwise. Bilateral kidney atlases were created by aligning all kidney images to a manually selected one using affine-transformation and diffeomorphic deformable image registration algorithms subsequently. The kidney stone image of each kidney was spatially transformed to the corresponding left or right kidney atlas with the same image deformation information obtained for creating the atlases. Statistical spatial maps of the kidney stones were generated for the bilateral kidneys separately to quantify frequency of the stones at every voxel of the kidney atlases. RESULTS: The DL image segmentation model segmented the kidneys with an average dice value of 0.96. The automatic stone identification algorithm detected individual kidney stones with 100% sensitivity. Statistical spatial maps of the kidney stones quantified spatial frequency of the kidney stones across subjects, with the highest frequency up to 6%. CONCLUSIONS: The statistical spatial mapping of kidney stones provide a quantitative means to characterize frequency of stones within the kidneys. The method can be used to assess kidney stones’ location for individual subjects with reference to group atlases that encode statistical spatial distribution information at a group level. Source of Funding: NIDDK P20 CHOP/ Penn Center for Machine Learning in Urology (P20DK127488)AUA Care Foundation and SPU Sushil Lacy Research Scholar Award (KMF) © 2023 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 209Issue Supplement 4April 2023Page: e669 Advertisement Copyright & Permissions© 2023 by American Urological Association Education and Research, Inc.MetricsAuthor Information Jiong Wu More articles by this author Katherine Fischer More articles by this author Yuemeng Li More articles by this author Benjamin Schurhamer More articles by this author Axel Largent More articles by this author Joey Logan More articles by this author Abhay Singh More articles by this author Joanie Garratt More articles by this author Justin Ziemba More articles by this author Gregory Tasian More articles by this author Yong Fan More articles by this author Expand All Advertisement PDF downloadLoading ...

Autostereoscopic 3D Augmented Reality Navigation for Laparoscopic Surgery: A Preliminary Assessment.

Since Augmented Reality (AR) and 3D visualization have proven to be of great significance to the safety and effectiveness of surgical outcomes, will autostereoscopic 3D AR without glasses bring new opportunities for surgical navigation of laparoscopic surgery? We used the CPD-based deformation registration algorithm and the proposed virtual view generation algorithm, realizing a deformable autostereoscopic 3D AR navigation framework for laparoscopic surgery. The depth perception and user experience of the 3D AR navigation were evaluated compared with the 2D AR display using an in-vitro porcine heart and offline clinical partial nephrectomy laparoscopic images. The autostereoscopic 3D AR allowed participants to have a more consistent spatial perception as well as a shorter measuring time than 2D AR with significant difference of p < 0.05. It can also improve relative depth perception for smaller distance separation of objects < = 3.28 mm. However, the autostereoscopic 3D AR perceived a worse experience compared to 2D AR in the user experience. Autostereoscopic 3D AR shows a more efficient and robust sense of spatial scale than 2D AR with better potential to shorten the operating time and improve surgical outcomes than 2D AR, but image blur and distortion are issues that must be solved to improve the perception effect. High precise registration and high fluency visualization requirements could make autostereoscopic 3D AR navigation for soft tissue more challenging. Our work lays a theoretical foundation for the further development of laparoscopic surgical navigation.

The spatial-temporal dynamics of pulmonary blood flow are altered in pulmonary arterial hypertension.

Global fluctuation dispersion (FDglobal), a spatial-temporal metric derived from serial images of the pulmonary perfusion obtained with MRI-arterial spin labeling, describes temporal fluctuations in the spatial distribution of perfusion. In healthy subjects, FDglobal is increased by hyperoxia, hypoxia, and inhaled nitric oxide. We evaluated patients with pulmonary arterial hypertension (PAH, 4F, aged 47 ± 15, mean pulmonary artery pressure 48 ± 7 mmHg) and healthy controls (CON, 7F, aged 47 ± 12) to test the hypothesis that FDglobal is increased in PAH. Images were acquired at ∼4-5 s intervals during voluntary respiratory gating, inspected for quality, registered using a deformable registration algorithm, and normalized. Spatial relative dispersion (RD = SD/mean) and the percent of the lung image with no measurable perfusion signal (%NMP) were also assessed. FDglobal was significantly increased in PAH (PAH = 0.40 ± 0.17, CON = 0.17 ± 0.02, P = 0.006, a 135% increase) with no overlap in values between the two groups, consistent with altered vascular regulation. Both spatial RD and %NMP were also markedly greater in PAH vs. CON (PAH RD = 1.46 ± 0.24, CON = 0.90 ± 0.10, P = 0.0004; PAH NMP = 13.4 ± 6.1%; CON = 2.3 ± 1.4%, P = 0.001 respectively) consistent with vascular remodeling resulting in poorly perfused regions of lung and increased spatial heterogeneity. The difference in FDglobal between normal subjects and patients with PAH in this small cohort suggests that spatial-temporal imaging of perfusion may be useful in the evaluation of patients with PAH. Since this MR imaging technique uses no injected contrast agents and has no ionizing radiation it may be suitable for use in diverse patient populations.NEW & NOTEWORTHY Using proton MRI-arterial spin labeling to obtain serial images of pulmonary perfusion, we show that global fluctuation dispersion (FDglobal), a metric of temporal fluctuations in the spatial distribution of perfusion, was significantly increased in female patients with pulmonary arterial hypertension (PAH) compared with healthy controls. This potentially indicates pulmonary vascular dysregulation. Dynamic measures using proton MRI may provide new tools for evaluating individuals at risk of PAH or for monitoring therapy in patients with PAH.

A model for gastrointestinal tract motility in a 4D imaging phantom of human anatomy.

Gastrointestinal (GI) tract motility is one of the main sources for intra/inter-fraction variability and uncertainty in radiation therapy for abdominal targets. Models for GI motility can improve the assessment of delivered dose and contribute to the development, testing, and validation of deformable image registration (DIR) and dose-accumulation algorithms. To implement GI tract motion in the 4D extended cardiac-torso (XCAT) digital phantom of human anatomy. Motility modes that exhibit large amplitude changes in the diameter of the GI tract and may persist over timescales comparable to online adaptive planning and radiotherapy delivery were identified based on literature research. Search criteria included amplitude changes larger than planning risk volume expansions and durations of the order of tens of minutes. The following modes were identified: peristalsis, rhythmic segmentation, high amplitude propagating contractions (HAPCs), and tonic contractions. Peristalsis and rhythmic segmentations were modeled by traveling and standing sinusoidal waves. HAPCs and tonic contractions were modeled by traveling and stationary Gaussian waves. Wave dispersion in the temporal and spatial domain was implemented by linear, exponential, and inverse power law functions. Modeling functions were applied to the control points of the nonuniform rational B-spline surfaces defined in the reference XCAT library. GI motility was combined with the cardiac and respiratory motions available in the standard 4D-XCAT phantom. Default model parameters were estimated based on the analysis of cine MRI acquisitions in 10 patients treated in a 1.5T MR-linac. We demonstrate the ability to generate realistic 4D multimodal images that simulate GI motility combined with respiratory and cardiac motion. All modes of motility, except tonic contractions, were observed in the analysis of our cine MRI acquisitions. Peristalsis was the most common. Default parameters estimated from cine MRI were used as initial values for simulation experiments. It is shown that in patients undergoing stereotactic body radiotherapy for abdominal targets, the effects of GI motility can be comparable or larger than the effects of respiratory motion. The digital phantom provides realistic models to aid in medical imaging and radiation therapy research. The addition of GI motility will further contribute to the development, testing, and validation of DIR and dose accumulation algorithms for MR-guided radiotherapy.

RMSim: controlled respiratory motion simulation on static patient scans

Objective. This work aims to generate realistic anatomical deformations from static patient scans. Specifically, we present a method to generate these deformations/augmentations via deep learning driven respiratory motion simulation that provides the ground truth for validating deformable image registration (DIR) algorithms and driving more accurate deep learning based DIR. Approach. We present a novel 3D Seq2Seq deep learning respiratory motion simulator (RMSim) that learns from 4D-CT images and predicts future breathing phases given a static CT image. The predicted respiratory patterns, represented by time-varying displacement vector fields (DVFs) at different breathing phases, are modulated through auxiliary inputs of 1D breathing traces so that a larger amplitude in the trace results in more significant predicted deformation. Stacked 3D-ConvLSTMs are used to capture the spatial-temporal respiration patterns. Training loss includes a smoothness loss in the DVF and mean-squared error between the predicted and ground truth phase images. A spatial transformer deforms the static CT with the predicted DVF to generate the predicted phase image. 10-phase 4D-CTs of 140 internal patients were used to train and test RMSim. The trained RMSim was then used to augment a public DIR challenge dataset for training VoxelMorph to show the effectiveness of RMSim-generated deformation augmentation. Main results. We validated our RMSim output with both private and public benchmark datasets (healthy and cancer patients). The structure similarity index measure (SSIM) for predicted breathing phases and ground truth 4D CT images was 0.92 ± 0.04, demonstrating RMSim’s potential to generate realistic respiratory motion. Moreover, the landmark registration error in a public DIR dataset was improved from 8.12 ± 5.78 mm to 6.58mm ± 6.38 mm using RMSim-augmented training data. Significance. The proposed approach can be used for validating DIR algorithms as well as for patient-specific augmentations to improve deep learning DIR algorithms. The code, pretrained models, and augmented DIR validation datasets will be released at https://github.com/nadeemlab/SeqX2Y.

Detecting and quantifying spatial misalignment between longitudinal kilovoltage computed tomography (kVCT) scans of the head and neck by using convolutional neural networks (CNNs).

Adaptive radiotherapy (ART) aims to address anatomical modifications appearing during the treatment of patients by modifying the planning treatment according to the daily positioning image. Clinical implementation of ART relies on the quality of the deformable image registration (DIR) algorithms included in the ART workflow. To translate ART into clinical practice, automatic DIR assessment is needed. This article aims to estimate spatial misalignment between two head and neck kilovoltage computed tomography (kVCT) images by using two convolutional neural networks (CNNs). The first CNN quantifies misalignments between 0 mm and 15 mm and the second CNN detects and classifies misalignments into two classes (poor alignment and good alignment). Both networks take pairs of patches of 33x33x33 mm3 as inputs and use only the image intensity information. The training dataset was built by deforming kVCT images with basis splines (B-splines) to simulate DIR error maps. The test dataset was built using 2500 landmarks, consisting of hard and soft landmark tissues annotated by 6 clinicians at 10 locations. The quantification CNN reaches a mean error of 1.26 mm (± 1.75 mm) on the landmark set which, depending on the location, has annotation errors between 1 mm and 2 mm. The errors obtained for the quantification network fit the computed interoperator error. The classification network achieves an overall accuracy of 79.32%, and although the classification network overdetects poor alignments, it performs well (i.e., it achieves a rate of 90.4%) in detecting poor alignments when given one. The performances of the networks indicate the feasibility of using CNNs for an agnostic and generic approach to misalignment quantification and detection.

Inter- and intrafractional 4D dose accumulation for evaluating ΔNTCP robustness in lung cancer

Background and purposeModel-based selection of proton therapy patients relies on a predefined reduction in normal tissue complication probability (NTCP) with respect to photon therapy. The decision is necessarily made based on the treatment plan, but NTCP can be affected when the delivered treatment deviates from the plan due to delivery inaccuracies. Especially for proton therapy of lung cancer, this can be important because of tissue density changes and, with pencil beam scanning, the interplay effect between the proton beam and breathing motion. Materials and methodsIn this work, we verified whether the expected benefit of proton therapy is retained despite delivery inaccuracies by reconstructing the delivered treatment using log-file based dose reconstruction and inter- and intrafractional accumulation. Additionally, the importance of two uncertain parameters for treatment reconstruction, namely deformable image registration (DIR) algorithm and α/β ratio, was assessed. ResultsThe expected benefit or proton therapy was confirmed in 97% of all studied cases, despite regular differences up to 2 percent point (p.p.) NTCP between the delivered and planned treatments. The choice of DIR algorithm affected NTCP up to 1.6 p.p., an order of magnitude higher than the effect of α/β ratio. ConclusionFor the patient population and treatment technique employed, the predicted clinical benefit for patients selected for proton therapy was confirmed for 97.0% percent of all cases, although the NTCP based proton selection was subject to 2 p.p. variations due to delivery inaccuracies.

Validation of a surface-based deformable MRI-3D ultrasound image registration algorithm toward clinical implementation for interstitial prostate brachytherapy

PURPOSEThe purpose of this study was to evaluate and clinically implement a deformable surface-based magnetic resonance imaging (MRI) to three-dimensional ultrasound (US) image registration algorithm for prostate brachytherapy (BT) with the aim to reduce operator dependence and facilitate dose escalation to an MRI-defined target. METHODS AND MATERIALSOur surface-based deformable image registration (DIR) algorithm first translates and scales to align the US- and MR-defined prostate surfaces, followed by deformation of the MR-defined prostate surface to match the US-defined prostate surface. The algorithm performance was assessed in a phantom using three deformation levels, followed by validation in three retrospective high-dose-rate BT clinical cases. For comparison, manual rigid registration and cognitive fusion by physician were also employed. Registration accuracy was assessed using the Dice similarity coefficient (DSC) and target registration error (TRE) for embedded spherical landmarks. The algorithm was then implemented intraoperatively in a prospective clinical case. RESULTSIn the phantom, our DIR algorithm demonstrated a mean DSC and TRE of 0.74 ± 0.08 and 0.94 ± 0.49 mm, respectively, significantly improving the performance compared to manual rigid registration with 0.64 ± 0.16 and 1.88 ± 1.24 mm, respectively. Clinical results demonstrated reduced variability compared to the current standard of cognitive fusion by physicians. CONCLUSIONSWe successfully validated a DIR algorithm allowing for translation of MR-defined target and organ-at-risk contours into the intraoperative environment. Prospective clinical implementation demonstrated the intraoperative feasibility of our algorithm, facilitating targeted biopsies and dose escalation to the MR-defined lesion. This method provides the potential to standardize the registration procedure between physicians, reducing operator dependence.

Geometric Deep Learning for Unsupervised Registration of Diffusion Magnetic Resonance Images.

Deep learning based models for registration predict a transformation directly from moving and fixed image appearances. These models have revolutionized the field of medical image registration, achieving accuracy on-par with classical registration methods at a fraction of the computation time. Unfortunately, most deep learning based registration methods have focused on scalar imaging modalities such as T1/T2 MRI and CT, with less attention given to more complex modalities such as diffusion MRI. In this paper, to the best of our knowledge, we present the first end-to-end geometric deep learning based model for the non-rigid registration of fiber orientation distribution fields (fODF) derived from diffusion MRI (dMRI). Our method can be trained in a fully-unsupervised fashion using only input fODF image pairs, i.e. without ground truth deformation fields. Our model introduces several novel differentiable layers for local Jacobian estimation and reorientation that can be seamlessly integrated into the recently introduced manifold-valued convolutional network in literature. The results of this work are accurate deformable registration algorithms for dMRI data that can execute in the order of seconds, as opposed to dozens of minutes to hours consumed by their classical counterparts.