Deformable Image Research Articles

An In Silico study of a One-Day One-Machine Workflow for Definitive Radiotherapy Cases on a Novel Simulation and Treatment Platform

The workflow in Radiotherapy (RT) has largely unchanged for the past three decades, despite increasing evidence suggesting that delayed access to RT, including the wait time between consultation, simulation, and treatment appointments, can negatively impact clinical outcomes. In this pilot study, we present preliminary results of an in silico study that demonstrate the feasibility of a novel RT platform, which integrates simulation into the treatment process and enables patients to receive immediate RT after their initial RT consultation. A prospective clinical study has been approved to assess the capabilities of a novel RT platform with a high quality CBCT system for imaging guidance as well as planning. This new platform enables a novel clinical workflow that allows clinicians to review contours and plans created on diagnostic CT images prior to the initial RT consultation and allow them to approve new plans adapted on the actual simulation dataset acquired on the first treatment fraction. Four patients receiving standard of care RT (three abdomen and one thorax) consented for this study and underwent additional experimental CBCT simulation on the new platform in addition to their standard CT simulation. The CBCT simulation was taken in two setups: with a specific mold on a flat couch and without a mold on a curved couch. To demonstrate the equivalence of the new workflow to the current standard of care, the plan created on the most recent diagnostic CT images was compared to the plans adapted on the experimental simulation images and the standard CT simulation images, using a knowledge-based model. Contours were propagated from approved datasets to the new datasets through deformable image registration. All experimental simulations were completed between 14 and 21 minutes with the assistance of two therapists. The contouring, editing, and replanning process took less than one hour in all cases, in line with our experience and peer-reviewed literature. Despite notable anatomical changes observed, the dose-volume histograms (DVH) were consistent, as shown in Table 1. The novel workflow presented herein was feasible and demonstrates that the integration of simulation with image-guided RT on one single platform may unlock the potential of accelerating the RT workflow and reducing the wait time for treatment from weeks to hours.

Clinical prior Knowledge-Based One-Shot Learning for Automatic Delineation of Clinical Target Volumes in Adaptation Radiotherapy of Cervical Cancer

Rapid and accurate delineation of clinical target volumes (CTV) of cervical cancer is the crux to ensure the efficiency and benefits of adaptation radiotherapy (ART). However, contour propagation using deformation image registration (DIR) is difficult to ensure the accuracy of CTV contours due to the significant tumor recession in next fraction, and the tumor progress in each fraction is not considered by conventional automatic delineation methods based on deep learning (DL). Currently, one-shot learning (OSL) is feasible to learn the tumor progress from former fractions to improve the accuracy of automatically delineating CTV. We retrospectively collected 45 patients with cervical cancer from January 2021 to May 2022 in our department. All patients consist of a pair of planning CT and daily CT in ART. A personalized automatic delineation method based on one-shot learning was developed to delineate CTV in daily CT by learning the clinical prior knowledge from the CTV contours and images of planning CT. The performance of our proposed method was evaluated by dice similarity coefficient (DSC), 95% Harsdorff distance (95HD) and average surface distance (ASD) with human experts, and its automatic delineation performance were compared with DIR and DL in daily CT. Our automatic delineation method OSL performed the best results in all evaluation metrics (denoted by mean ± standard deviation) as shown in Table 1, it is superior to method DL: 0.92 & 0.90 of DSC, 2.33 mm & 2.68 mm of HD95, 0.68 mm & 0.82 mm of ASD, P < 0.05 for DSC and ASD. Specifically, our method is significantly superior to the automatic delineation results by method DIR: 0.92 & 0.84 of DSC, 2.33 mm & 4.11 mm of HD95, 0.68 mm & 1.52 mm of ASD, P < 0.05 for all. In addition, OSL can significantly overcome the delineation problems in fuzzy boundary and delineation missing and perform better generalization for some unusual images, compared with DIR and DL. We proposed an automatic delineation method based on one-shot learning for CTV of cervical cancer in ART, the results demonstrated that the proposed method could improve the precision and generalization of automatically delineating CTV compared against current popular methods. Therefore, it is potential to improve the quality and efficiency of ART for personalized patients and have a positive impact on tumor control and patient survival.

Evaluation of Brain Atrophy after One Year of Whole Brain Irradiation Using Voxel Based Morphometry

Whole-brain irradiation has often been used for metastatic brain tumors and in prophylactic irradiation, but there are few reports on the frequency of cognitive decline and brain atrophy. We attempted to adapt an objective method of measuring brain atrophy to patients after whole-brain irradiation, Using Voxel based morphometry (VBM) measured from MRI to determine if brain atrophy has occurred one year after whole brain irradiation. VBM is a method to measure the change in signal intensity when an atrophic brain is transformed into a normal brain by Deformable Image Registration. We use VBM to measure brain atrophy before and one year after irradiation in patients who received whole brain irradiation at our institution. The MRI used for the analysis was a contrast-enhanced T1-weighted image, Sagittal cross-sectional images of 1 mm thickness were obtained from the whole brain using the 3D variable flip angle turbo spin echo sequence, which is clinically used at our hospital for the cancer patient's management. The subjects were 28 patients who had undergone whole-brain irradiation at our hospital and for whom MRIs were available before treatment and after one year. The cases analyzed included patients who received prophylactic Cranial irradiation (PCI) in cases of small cell lung cancer obtained complete remission. Z-scores obtained from the medial temporal lobe were compared before and after whole brain irradiation treatment to quantitatively assess brain atrophy. The Z-score is the normalized difference between the voxel values obtained from the normal group average and the individual case. There were 15 cases of lung cancer, 5 cases of malignant lymphoma, 5 cases of breast cancer, and 3 cases of various types of cancer, with an average age of 65 years. Cases with tumors or brain edema in the vicinity of the hippocampus were excluded. Various irradiation doses were used, but the commonly used radiation doses were 23.4 Gy/13Fr (lymphoma), 25 Gy/10 Fr (PCI), 35 Gy/14 Fr (multiple metastasis). A Paired-samples one sided t-test was performed to Mean z-score of positive values within the medial temporal lobe before and one year after whole brain irradiation. This number reflects the degree of atrophy within the region of interest and is determined as mild atrophy if it is greater than 1.0. The mean Z-score was found to increase at a risk of 0.01 after whole brain irradiation. This means that brain atrophy is progressing after whole brain irradiation. Hippocampal atrophy becomes apparent about a year after whole brain irradiation. It suggests that brain atrophy occurs after radiotherapy. The problem with this study is that the effects of chemotherapy-induced brain atrophy and aging cannot be ruled out.

Feasibility Study of Robust Treatment Planning in VMAT for Head and Neck Cancer

In general, patient positional uncertainty is considered by adding a geometrically expanded margin to clinical target volume (CTV) in photon radiation therapy. However, this method may not be suitable because image-guided radiotherapy is available. In intensity modulated proton beam therapy, robust treatment planning is currently common to take patient positional uncertainty into account in optimization rather than in margins. The purpose of this study is to assess the feasibility of clinical implementation of the method in volumetric modulated arc therapy (VMAT) for head and neck cancer. We quantitatively evaluated whether the plans with the robust optimization (Robust plans) can adequately cover CTV against patients' positional uncertainties and body shape change throughout a treatment course. Ten head and neck cancer patients were chosen, who were treated with PTV-based VMAT plans in our hospital between 2021.5-2022.4. RayStation V10A (RaySearch Laboratories, Stockholm, Sweden) was used for the robust optimization, which was applied to the CTVs with patient positional uncertainty of 5 mm in the 6-axis direction. Dose prescribed to the high- and low-risk CTVs were to 70 and 56 Gy in 35 fractions, respectively. To create the patients' CT images with residual set-up errors and body shape change at the treatment, pseudo simulation-CT images were created by deformable image registration with CBCT and simulation-CT. Dose distribution at the treatment was re-calculated by applying the plan to the pseudo simulation-CT images. The variation of D98 for the high-risk CTV from the time of treatment planning was evaluated on a weekly basis. For comparison, planning target volume (PTV) -based plans (5 mm margin circumference) were created and a similar evaluation was performed. D98 for the high-risk CTV varied between -3∼2% in the robust plan and between -5∼1% in the PTV-based plan during the treatment course. There was no significant difference in the amount of D98 variation between the two plans by t-test, except for one case with hypopharyngeal cancer. In this case, D98 for the high-risk CTV varied within ±1% with the PTV-based plan, whereas the value decreased up to 3% with the robust plan (p < 0.05). This case often had a residual setup error of approximately 5 mm at the sites related to the pitch rotation of head, suggesting that the dose distribution for the robust plan was affected by non-rigid positional errors. Patient weight loss during the treatment period was -3.5±2.4 kg, showing a weak correlation (r = -0.33) with the variation in D98 for the high-risk CTV. The robust treatment planning exhibits comparable CTV coverage to the conventional PTV-based planning against positional uncertainty and body shape change throughout a treatment period. In order to overcome set-up baseline shift by the non-rigid positional errors, re-planning should be recommended. Further planning studies will be conducted to promote clinical implementation of the method.

Determining the Dosimetric Accuracy of Deep Learning-Based Fully Automated Registration-Segmentation Approach for Thoracic Cancer Organs-at-Risk Contouring

For adaptive radiation therapy (ART), the contours on the planning CT (pCT) are frequently propagated to cone-beam CT (CBCT) via deformable image registration and manually edited, which is observer-dependent and time-consuming. To automate this process, we created a fully automated workflow by combining a deep learning (DL)-based pCT segmentation model with a CT-to-CBCT registration-segmentation DL model. The purpose of our research is to determine how using the proposed workflow's automatically generated contours affects thoracic organs-at-risk sparing (OAR). Seven patients with locally advanced non-small cell lung cancer who underwent treatment with intensity modulated radiation therapy were included in this study. Each patient's pCT was segmented using a published DL model that has been used for generating thoracic OAR segmentation and radiotherapy planning in the clinic since July of 2020. Next, pCT was deformably registered using a published recurrent deep registration-segmentation method. Whereas the original method's segmentation sub-network was only trained to segment esophagus, the registration sub-network was used to propagate contours for heart, esophagus, and the proximal bronchial tree (PBT). Geometric segmentation accuracy using the Dice Similarity Coefficient (DSC) and the 95th percentile Hausdorff Distance (HD) and dose metrics including the mean esophageal dose (MED) and D90% of the heart (D90) were computed from the total accumulated dose for the first two weeks of treatment. The esophagus had a high DSC and a low HD (0.93 and 2.85mm) and conversely, the heart had lower accuracy (DSC = 0.85, HD = 22.06mm). PBT showed relatively high performance as well, with DSC of 0.91 and HD of 2.28mm, owing to its proximity to the esophagus. The accumulated MED for manual contour was slightly lower than AI-contours (11.34 vs 11.83 Gy), suggesting reliability of the proposed workflow. The reverse is seen for the D90 of the heart (manual = 1.74 and AI-contour = 1.56 Gy), likely due to the heart not being included in the original DL framework. This study reported preliminary results on the feasibility of using a fully automated and patient-specific workflow for CBCT auto-segmentation in ART, confirming its role as a geometrically and dosimetrically accurate solution for thoracic OARs. However, because it is currently limited to the esophagus, we believe that re-training the algorithm will increase confidence in other OARs such as the heart and lungs.

Quantifying the Dosimetric Impact of Tissue Hounsfield Unit Assignment in Deep Learning-based Synthetic CT Images For MRI-Only Radiation Therapy of The Head and Neck

MRI-only simulation for head and neck (HN) radiotherapy (RT) could allow for single-image modality planning with excellent soft tissue contrast. In the MRI-only simulation workflow, synthetic CTs (sCTs) are generated from MRI to provide electron density information for dose calculation. Bone/air regions produce little MRI signal which could lead to electron density misclassification in sCTs. Establishing the dosimetric impact of this error could inform quality assurance (QA) procedures using MRI-only RT planning. In this study we quantify differences in Hounsfield Unit (HU) values between paired CT/ sCTs of HN cancer patients and investigate the dosimetric impact on clinical treatment plans. Fourteen patients with head and neck cancer undergoing same-day CT and 3T MRI simulation were retrospectively identified. MRIs were deformed to the CT using multimodal deformable image registration. SCTs were generated from T1w DIXON MRI using a commercially available deep learning-based generator (MRIplanner, Spectronics). Tissue voxel assignment was quantified by creating a CT-derived HU threshold contour. CT/sCT HU differences for anatomical/target contours and tissue classification regions including air (<-250HU), adipose tissue (-250HU to -51HU), soft tissue (-50HU to 199HU), spongy (200HU to 499HU) and cortical bone (> 500HU) were quantified. T-test was used to determine if sCT/CT HU differences were significant. The frequency of structures that had a HU difference >70HU (the CT window-width setting for intra-cranial structures) was computed to establish structure classification accuracy. Clinical IMRT treatment plans created on CTs were retrospectively recalculated on sCT images using compared using the gamma metric. The mean ratio of sCT HUs relative to CT for air, adipose tissue, soft tissue, spongy and cortical bone were 1.7±0.3, 1.1±0.1, 1.0±0.1, 0.9±0.1 and 0.8±0.1 (value of 1 indicates perfect agreement). T-tests (significance set at t = 0.05) identified differences in HU values for air, spongy and cortical bone in sCT images compared to CT. The structures with sCT/CT HU differences > 70HU were the L/R cochlea and mandible, occurring in >78% of the tested cohort. These structures contain dense bone/air interfaces. Plans recalculated on sCTs yielded global/local gamma pass rates of 98.7%±1.2% (3mm,3%) and 95.5%±2.5% (1mm,1%). Mean differences in D95, D50, D10 and D2 dose volume histogram (DVH) metrics for organ-at-risk (OAR) and planning tumor volumes (PTV) were 0.8%±1.5% and 2.1% ± 1.2% respectively. In this cohort, HU differences in sCTs were observed but did not translate into a reduction in gamma pass rates and OAR/PTV DVH metrics. The acquisition of additional training data such as ultrashort echo time MRI could improve bone/air contrast and reduce bone/air sCT misclassifications. Further studies will establish the variation in sCT dosimetric accuracy using a larger retrospective cohort to inform QA limits on clinical sCT usage.

Quantifying Cardiac Chamber Motion for Noninvasive Stereotactic Cardiac Radiosurgery Using Deformable Image Registration

Noninvasive Stereotactic Cardiac Radiosurgery (NSCR) is a novel treatment for drug resistant ventricular tachycardia. One of the major issues with this new treatment style is uncertainty in the target location due to cardiac and respiratory motion. This motion uncertainty results in planning target volumes that are 3 to 4 times the original target volume. Quantification of this motion requires accurate segmentation of the heart chambers throughout cardiac and respiratory phases. Manual segmentation of the heart is incredibly time consuming and not feasible to be done for each patient. This work aimed to develop a semi-automated workflow that takes manual contours of the heart chambers in one phase of a breath-hold ECG cardiac 4DCT and warp these contours onto all other phases. Four chambers of the heart and the aorta were manually contoured on one phase of a patient's breath-hold ECG cardiac 4DCT. A single shot deep learning deformable image registration code called GroupRegNet was used to propagate the contours onto the additional phases. This code uses a global smoothness parameter that does not work well in cardiac cases. To account for the type of motion that is expected in the cardiac 4DCTs, a piecewise smoothing parameter that only enforces smoothness within each of the original contours was added to the code. To determine the accuracy of the registration, a second phase with significant motion from the contoured phase was also manually contoured. The Dice coefficient and surface-to-surface distance between the manual and propagated contours were calculated. The tables below show the improvement of the contour propagation accuracy, measured as the Dice coefficients and the mean surface-to-surface distance using the original global smoothing and piecewise-smoothing only within the original contours. These results show an improvement in registration accuracy due to using a piecewise smoothing factor instead of global smoothness. Along with improvements in registration accuracy, this work shows promise for this semi-automated methodology to be further improved with the hope of fully automating the process and eventually implemented into the workflow for NSCR treatment to quantify cardiac chamber motion more accurately and eventually reduce the margins during radiation treatment.

Adaptive Auto-Segmentation for MRI-Guided Online Adaptive Radiotherapy of Cervical Carcinoma

Accurate and efficient delineation of organs and targets on session images is critical in MRI-guided online adaptive radiotherapy (MRgOART). This study proposes a registration-guided deep learning image segmentation framework to assist online delineation of cervical carcinoma. A total of 300 T2-weighted MR images were acquired for patients with cervical carcinoma treated by a 1.5T Unity MR-Linac. The CTV, bladder, rectum, pelvic bone and femoral joints were delineated on each MRI by the same radiation oncologist. To overcome these obstacles to online MRI segmentation, we propose a registration-guided DL (RgDL) segmentation framework that integrates image registration algorithms and DL segmentation models. Firstly, the DL segmentation model was trained using nnU-net. Then, for each treatment fraction, the deformable image registration (DIR) algorithm generates initial contours from previous treatment fraction, which were used as guidance by DL model to obtain the accurate current segmentation. The segmentation accuracy of alone DIR, DL and RgDL were evaluated by dice similarity coefficients (DSC) and other distance-based metrics. Compared to the baseline approaches using the DIR and the DL alone, RgDL achieved a DSC of 91.12% on CTV, higher than DIR and DL alone by 15.54% and 10.13%. The DSC of RgDL were improved to 95.58%, 93.65%, 87.8% and 94.84% for bladder, pelvic bone, rectum and femoral joints, higher than DIR and DL alone by 9.61% on average. The proposed adaptive auto-segmentation method can achieve accurate and efficient segmentation and potentially overcome these obstacles to MRgOART.

Heatmaps to Assess Tumor Motion Probability with MRcine in Head and Neck Cancer

Intrafraction motion affects tumor position during radiation for head and neck cancers (HNC) and can be assessed using magnetic resonance cine (MR-cine). Heatmaps - visual representation of patient specific temporal tumor location akin to internal target volume (ITV) - were analyzed to demonstrate the variation of population-based motion margins in primary oropharyngeal (OP) and laryngeal/hypopharyngeal (LH) sites with the goal to provide insights towards personalized margins. MR-cine were performed for LH and OP HNC patients as part of simulation protocol. Images (900-1500 slices) were acquired across 3-5 minutes per patient. Gross target volumes were propagated on T1 contrast sagittal sequence using deformable image registration then adjusted manually. Tumor locations were integrated across all frames and displayed as heatmaps representing location probability. To determine individualized motion, a baseline contour representing average tumor rest position was expanded both isotropically and directionally in 1 mm increments in a novel analysis to define the contribution of each increment on target coverage. To assess directional dependence, displacements from rest position were evaluated in 4 planes from centroid: (A) 12 o'clock (OC) to 6OC (capturing volumetric shift in a portion of superior, posterior and inferior directions), B) 3OC-9OC, C) 6OC-12OC, D) 9OC-3OC. Histograms demonstrating the proportion of scan time the tumor was within expansions were generated, with expansion margins for 25%-95% coverage analyzed by patient. Wilcoxon-rank-sum test was performed to compare motion by site. Motion was evaluated in 66 patients (LH n = 27, OP n = 39). In LH, a median (med) isometric expansion of 8.5mm to achieve 95% target was required, with large variations observed for minimum (min): 2.8mm and maximum (max):23.5mm. Smaller differences were observed for OP with med of 5.3mm, (min:1.9mm, max:14.6mm). At 75%, 50% and 25% coverage, expansions for LH (med: 4.8 min: 1.9 max: 17.3; med: 3.8 min: 1.2 max: 12.9; med: 3.3, min: 0.6, max: 9.3) were not significantly different than OP (med: 3.8, min: 1.7, max: 10.9; med: 3.1 min: 1.3, max: 9.4; med: 2.1, min: 0.7, max: 8.4), suggesting larger but infrequent shifts in LH. Directional tumor displacement varied widely (Table 1), and by site and points describing the need for personalized margins. Current literature as well as the analysis in this study indicate a wide range of motion - both in magnitude and duty cycle - between and within HN sub-sites not captured on CT simulation. Moving forward, motion based heatmaps based on MR-cine may serve as visualization tool for ITV contouring or to generate personalized motion margins.

Target Volumes Comparison between Postoperative Simulation MRI and Preoperative Diagnostic MRI for Prone Breast Radiotherapy after Breast-Conserving Surgery

To investigate the correlation and difference between target volumes determined by postoperative prone positioning magnetic resonance imaging (MRI) and preoperative prone diagnostic MRI for breast radiotherapy after breast-conserving surgery (BCS) based on deformable image registration (DIR). A total of 17 patients undergoing prone whole breast-irradiation after BCS were enrolled, all these patients were undergoing preoperative and postoperative prone MRI scans. The gross tumor volume (GTV) was delineated on the preoperative MRI images, then expanding 10 mm to simulate the preoperative lumpectomy cavity (LC). The LC were expended to clinical target volume (CTV), and planning target volume (PTV) on preoperative and postoperative MRI sequences, respectively. Preoperative and postoperative MRI was registered based on DIR by a commercially available software system. The target volume, the distance between the centers of mass (dCOM), conformity index (CI), degree of inclusion (DI) were compared to explore the target volume variation for pre- and post-surgery delineation. And evaluated the relationship between CILC/CIPTV and clinical factors. The volume of GTVpre was 3.05cm3(range, 1-23.63cm3). The median volume of LCpre were 26.84 cm3 (range, 11.75-102.28 cm3). 0.85cm3 smaller than the LCpre, the LCpost were 25.99 cm3 (range, 5.98-41.95) (P = 0.04). The dCOM, CI, and DI between LCpre and LCpost were 1.371cm (range, 0.533-5.447), 0.221 (range, 0.041-0.516), and 0.472 (range, 0.108-0.804). There was no significant statistically difference between PTVpre and PTVpost (120.73cm3 vs 131.67cm3, p = 0.723). Compared with LC, the CI and DI between PTVpre and PTVpost were increased to 0.446 (range, 0.233-0.700), 0.636 (range, 0.362-0.834), respectively. Meanwhile, the dCOM was reduced to 1.239cm (range, 0.413-3.642). There was no obvious linear correlation between the CI with the GTV volume, the ratio of the primary tumor volume to the breast volume, the distance from the primary tumor to the nipple and chest wall, and the Body Mass Index (BMI), respectively. For prone breast irradiation, even based on DIR, the spatial correspondence of the target volumes between preoperative and postoperative MRI images were still poor. Therefore, it is not reasonable to delineate postoperative LC refer to preoperative diagnostic MRI based on DIR. Preoperative MRI also cannot provide individualized guidance for LC delineation according to tailored patient.

PC-Reg: A pyramidal prediction–correction approach for large deformation image registration

Deformable image registration plays an important role in medical image analysis. Deep neural networks such as VoxelMorph and TransMorph are fast, but limited to small deformations and face challenges in the presence of large deformations. To tackle large deformations in medical image registration, we propose PC-Reg, a pyramidal Prediction and Correction method for deformable registration, which treats multi-scale registration akin to solving an ordinary differential equation (ODE) across scales. Starting with a zero-initialized deformation at the coarse level, PC-Reg follows the predictor–corrector regime and progressively predicts a residual flow and a correction flow to update the deformation vector field through different scales. The prediction in each scale can be regarded as a single step of ODE integration. PC-Reg can be easily extended to diffeomorphic registration and is able to alleviate the multiscale accumulated upsampling and diffeomorphic integration error. Further, to transfer details from full resolution to low scale, we introduce a distillation loss, where the output is used as the target label for intermediate outputs. Experiments on inter-patient deformable registration show that the proposed method significantly improves registration not only for large but also for small deformations.

Extracting lung contour deformation features with deep learning for internal target motion tracking: a preliminary study

Objective. To propose lung contour deformation features (LCDFs) as a surrogate to estimate the thoracic internal target motion, and to report their performance by correlating with the changing body using a cascade ensemble model (CEM). LCDFs, correlated to the respiration driver, are employed without patient-specific motion data sampling and additional training before treatment. Approach. LCDFs are extracted by matching lung contours via an encoder–decoder deep learning model. CEM estimates LCDFs from the currently captured body, and then uses the estimated LCDFs to track internal target motion. The accuracy of the proposed LCDFs and CEM were evaluated using 48 targets’ motion data, and compared with other published methods. Main results. LCDFs estimated the internal targets with a localization error of 2.6 ± 1.0 mm (average ± standard deviation). CEM reached a localization error of 4.7 ± 0.9 mm and a real-time performance of 256.9 ± 6.0 ms. With no internal anatomy knowledge, they achieved a small accuracy difference (of 0.34∼1.10 mm for LCDFs and of 0.43∼1.75 mm for CEM at the 95% confidence level) with a patient-specific lung biomechanical model and the deformable image registration models. Significance. The results demonstrated the effectiveness of LCDFs and CEM on tracking target motion. LCDFs and CEM are non-invasive, and require no patient-specific training before treatment. They show potential for broad applications.

MacJNet: weakly-supervised multimodal image deformable registration using joint learning framework and multi-sampling cascaded MIND

Deformable multimodal image registration plays a key role in medical image analysis. It remains a challenge to find accurate dense correspondences between multimodal images due to the significant intensity distortion and the large deformation. macJNet is proposed to align the multimodal medical images, which is a weakly-supervised multimodal image deformable registration method using a joint learning framework and multi-sampling cascaded modality independent neighborhood descriptor (macMIND). The joint learning framework consists of a multimodal image registration network and two segmentation networks. The proposed macMIND is a modality-independent image structure descriptor to provide dense correspondence for registration, which incorporates multi-orientation and multi-scale sampling patterns to build self-similarity context. It greatly enhances the representation ability of cross-modal features in the registration network. The semi-supervised segmentation networks generate anatomical labels to provide semantics correspondence for registration, and the registration network helps to improve the performance of multimodal image segmentation by providing the consistency of anatomical labels. 3D CT-MR liver image dataset with 118 samples is built for evaluation, and comprehensive experiments have been conducted to demonstrate that macJNet achieves superior performance over state-of-the-art multi-modality medical image registration methods.

The image-based ultrasonic cell shaking test.

Mechanical signals play a vital role in cell biology and is a vast area of research. Thus, there is motivation to understand cell deformation and mechanobiological responses. However, the ability to controllably deform cells in the ultrasonic regime and test their response is a noted challenge throughout the literature. Quantifying and eliciting an appropriate stimulus has proven to be difficult, resulting in methods that are either too aggressive or oversimplified. Furthermore, the ability to gain a real-time insight into cell deformation and link this with the biological response is yet to be achieved. One application of this understanding is in ultrasonic surgical cutting, which is a promising alternative to traditional methods, but with little understanding of its effect on cells. Here we present the image based ultrasonic cell shaking test, a novel method that enables controllable loading of cells and quantification of their response to ultrasonic vibrations. Practically, this involves seeding cells on a substrate that resonates at ultrasonic frequencies and transfers the deformation to the cells. This is then incorporated into microscopic imaging techniques to obtain high-speed images of ultrasonic cell deformation that can be analysed using digital image correlation techniques. Cells can then be extracted after excitation to undergo analysis to understand the biological response to the deformation. This method could aid in understanding the effects of ultrasonic stimulation on cells and how activated mechanobiological pathways result in physical and biochemical responses.

Evaluating the Necessity of Adaptive RT and the Role of Deformable Image Registration in Lung Cancer with Different Pathologic Classifications.

This study aimed to analyze differential radiotherapy (RT) responses according to the pathological type of lung cancer to see the possibility of applying adaptive radiotherapy (ART). ART planning with resampled-computed tomography was conducted for a total of 30 patients (20 non-small-cell lung cancer patients and 10 small-cell lung cancer patients) using a deformable image registration technique to reveal gross tumor volume (GTV) changes according to the duration of RT. The small-cell lung cancer group demonstrated an average GTV reduction of 20.95% after the first week of initial treatment (p = 0.001), whereas the adenocarcinoma and squamous cell carcinoma groups showed an average volume reduction of 20.47% (p = 0.015) and 12.68% in the second week. The application of ART according to the timing of GTV reduction has been shown to affect changes in radiation dose irradiated to normal tissues. This suggests that ART applications may have to be different depending on pathological differences in lung cancer. Through these results, the present study proposes the possibility of personalized treatment options for individual patients by individualizing ART based on specific radiation responses by pathologic types of lung cancer.

Deformable lung 4DCT image registration via landmark-driven cycle network.

An automated, accurate, and efficient lung four-dimensional computed tomography (4DCT) image registration method is clinically important to quantify respiratory motion for optimal motion management. The purpose of this work is to develop a weakly supervised deep learning method for 4DCT lung deformable image registration (DIR). The landmark-driven cycle network is proposed as a deep learning platform that performs DIR of individual phase datasets in a simulation 4DCT. This proposed network comprises a generator and a discriminator. The generator accepts moving and target CTs as input and outputs the deformation vector fields (DVFs) to match the two CTs. It is optimized during both forward and backward paths to enhance the bi-directionality of DVF generation. Further, the landmarks are used to weakly supervise the generator network. Landmark-driven loss is used to guide the generator's training. The discriminator then judges the realism of the deformed CT to provide extra DVF regularization. We performed four-fold cross-validation on 10 4DCT datasets from the public DIR-Lab dataset and a hold-out test on our clinic dataset, which included 50 4DCT datasets. The DIR-Lab dataset was used to evaluate the performance of the proposed method against other methods in the literature by calculating the DIR-Lab Target Registration Error (TRE). The proposed method outperformed other deep learning-based methods on the DIR-Lab datasets in terms of TRE. Bi-directional and landmark-driven loss were shown to be effective for obtaining high registration accuracy. The mean and standard deviation of TRE for the DIR-Lab datasets was 1.20±0.72mm and the mean absolute error (MAE) and structural similarity index (SSIM) for our datasets were 32.1±11.6 HU and 0.979±0.011, respectively. The landmark-driven cycle network has been validated and tested for automatic deformable image registration of patients' lung 4DCTs with results comparable to or better than competing methods.

A whole-body diffusion MRI normal atlas: development, evaluation and initial use

BackgroundStatistical atlases can provide population-based descriptions of healthy volunteers and/or patients and can be used for region- and voxel-based analysis. This work aims to develop whole-body diffusion atlases of healthy volunteers scanned at 1.5T and 3T. Further aims include evaluating the atlases by establishing whole-body Apparent Diffusion Coefficient (ADC) values of healthy tissues and including healthy tissue deviations in an automated tumour segmentation task.MethodsMulti-station whole-body Diffusion Weighted Imaging (DWI) and water-fat Magnetic Resonance Imaging (MRI) of healthy volunteers (n = 45) were acquired at 1.5T (n = 38) and/or 3T (n = 29), with test-retest imaging for five subjects per scanner. Using deformable image registration, whole-body MRI data was registered and composed into normal atlases. Healthy tissue ADCmean was manually measured for ten tissues, with test-retest percentage Repeatability Coefficient (%RC), and effect of age, sex and scanner assessed. Voxel-wise whole-body analyses using the normal atlases were studied with ADC correlation analyses and an automated tumour segmentation task. For the latter, lymphoma patient MRI scans (n = 40) with and without information about healthy tissue deviations were entered into a 3D U-Net architecture.ResultsSex- and Body Mass Index (BMI)-stratified whole-body high b-value DWI and ADC normal atlases were created at 1.5T and 3T. %RC of healthy tissue ADCmean varied depending on tissue assessed (4–48% at 1.5T, 6–70% at 3T). Scanner differences in ADCmean were visualised in Bland-Altman analyses of dually scanned subjects. Sex differences were measurable for liver, muscle and bone at 1.5T, and muscle at 3T. Volume of Interest (VOI)-based multiple linear regression, and voxel-based correlations in normal atlas space, showed that age and ADC were negatively associated for liver and bone at 1.5T, and positively associated with brain tissue at 1.5T and 3T. Adding voxel-wise information about healthy tissue deviations in an automated tumour segmentation task gave numerical improvements in the segmentation metrics Dice score, sensitivity and precision.ConclusionsWhole-body DWI and ADC normal atlases were created at 1.5T and 3T, and applied in whole-body voxel-wise analyses.

Consistency in contouring of organs at risk by artificial intelligence vs oncologists in head and neck cancer patients

Background In the Danish Head and Neck Cancer Group (DAHANCA) 35 trial, patients are selected for proton treatment based on simulated reductions of Normal Tissue Complication Probability (NTCP) for proton compared to photon treatment at the referring departments. After inclusion in the trial, immobilization, scanning, contouring and planning are repeated at the national proton centre. The new contours could result in reduced expected NTCP gain of the proton plan, resulting in a loss of validity in the selection process. The present study evaluates if contour consistency can be improved by having access to AI (Artificial Intelligence) based contours. Materials and Methods The 63 patients in the DAHANCA 35 pilot trial had a CT from the local DAHANCA centre and one from the proton centre. A nationally validated convolutional neural network, based on nnU-Net, was used to contour OARs on both scans for each patient. Using deformable image registration, local AI and oncologist contours were transferred to the proton centre scans for comparison. Consistency was calculated with the Dice Similarity Coefficient (DSC) and Mean Surface Distance (MSD), comparing contours from AI to AI and oncologist to oncologist, respectively. Two NTCP models were applied to calculate NTCP for xerostomia and dysphagia. Results The AI contours showed significantly better consistency than the contours by oncologists. The median and interquartile range of DSC was 0.85 [0.78 − 0.90] and 0.68 [0.51 − 0.80] for AI and oncologist contours, respectively. The median and interquartile range of MSD was 0.9 mm [0.7 − 1.1] mm and 1.9 mm [1.5 − 2.6] mm for AI and oncologist contours, respectively. There was no significant difference in NTCP. Conclusions The study showed that OAR contours made by the AI algorithm were more consistent than those made by oncologists. No significant impact on the NTCP calculations could be discerned.

Diegraph: dual-branch information exchange graph convolutional network for deformable medical image registration

Diegraph: dual-branch information exchange graph convolutional network for deformable medical image registration

P07.04.A ELASTIC IMAGE FUSION OF INTRAOPERATIVE ULTRASOUND AND PREOPERATIVE MRI FOR IMPROVING NAVIGATION ACCURACY IN BRAIN TUMOR SURGERY

Abstract BACKGROUND In brain surgery, neuronavigation based on preoperative MRI is limited by various physical and operational factors, such as surgery-induced brain shift. Using navigated intraoperative ultrasound (iUS) imaging and 3D iUS image reconstruction may help improve neuronavigation. However, the registration of MRI and US scans can lead to spatial inconsistencies between the two modalities. This study aims to evaluate the clinical integration of navigated iUS imagining and its potential to improve the accuracy of resection during brain tumor surgery by comparing the performance of rigid and elastic MRI-iUS image fusion algorithms during retrospective evaluation. MATERIAL AND METHODS N = 50 patients with brain tumors were prospectively enrolled in this trial. The neuronavigation system (BrainLab AG) used in the study allowed for the reconstruction and integration of 3D iUS imaging, which was acquired multiple times throughout the surgery using a tracked ultrasound probe (BK Medical). Prototypes of automatic rigid and elastic MRI-iUS image fusion algorithms (Brainlab Elements Image Fusion) were retrospectively applied to improve co-registration accuracy between the MRI and iUS data. To evaluate the image co-registration accuracy, anatomical landmarks were defined in the MRI and iUS scans and used to measure Euclidean distances between each landmark pair. RESULTS Improvements in the image quality over the course of the study indicated that skills in US handling are essential. Contrast behavior and tumor structure influence the visibility of the tumor in the US. Artifacts caused by hemostatic agents challenge the US image quality and the fluid-filled resection cavity requires experience in probe handling. In terms of image fusion, significant mismatches of the iUS and MRI data were reported. For scans acquired before resection, rigid fusion using a novel co-registration algorithm may enhance the registration accuracy with mean target registration errors for two representative patients of 6.5/8.9 mm and 3.7/3.4 mm before and after registration update, respectively. For iUS scans acquired after tumor resection, rigid fusion is inherently limited due to non-linear brain shifts. Therefore, elastic image fusion has been applied to provide deformable MR-iUS image co-registration. CONCLUSION Navigated iUS imaging can be integrated into clinical routine and the use of MR-iUS image co-registration for updating navigation is a promising tool to compensate brainshift during surgical resection. However, further studies are needed to quantify the accuracy and assess whether iUS could potentially replace the use of iMRI in resection control of selected cases.