Autosegmental Model Research Articles

Automatic segmentation and radiomic texture analysis for osteoporosis screening using chest low-dose computed tomography.

This study developed a diagnostic tool combining machine learning (ML) segmentation and radiomic texture analysis (RTA) for bone density screening using chest low-dose computed tomography (LDCT). A total of 197 patients who underwent LDCT followed by dual-energy X-ray absorptiometry were analyzed. First, an autosegmentation model was trained using LDCT to delineate the thoracic vertebral body (VB). Second, a two-level classifier was developed using radiomic features extracted from VBs for the hierarchical pairwise classification of each patient's bone status. All the patients were initially classified as either normal or abnormal, and all patients with abnormal bone density were then subdivided into an osteopenia group and an osteoporosis group. The performance of the classifier was evaluated through fivefold cross-validation. The model for automated VB segmentation achieved a Sorenson-Dice coefficient of 0.87 ± 0.01. Furthermore, the area under the receiver operating characteristic curve scores for the two-level classifier were 0.96 ± 0.01 for detecting abnormal bone density (accuracy = 0.91 ± 0.02; sensitivity = 0.93 ± 0.03; specificity = 0.89 ± 0.03) and 0.98 ± 0.01 for distinguishing osteoporosis (accuracy = 0.94 ± 0.02; sensitivity = 0.95 ± 0.03; specificity = 0.93 ± 0.03). The testing prediction accuracy levels for the first- and second-level classifiers were 0.92 ± 0.04 and 0.94 ± 0.05, respectively. The overall testing prediction accuracy of our method was 0.90 ± 0.05. The combination of ML segmentation and RTA for automated bone density prediction based on LDCT scans is a feasible approach that could be valuable for osteoporosis screening during lung cancer screening. • This study developed an automatic diagnostic tool combining machine learning-based segmentation and radiomic texture analysis for bone density screening using chest low-dose computed tomography. • The developed method enables opportunistic screening without quantitative computed tomography or a dedicated phantom. • The developed method could be integrated into the current clinical workflow and used as an adjunct for opportunistic screening or for patients who are ineligible for screening with dual-energy X-ray absorptiometry.

Deep-learning-based generation of synthetic 6-minute MRI from 2-minute MRI for use in head and neck cancer radiotherapy.

Quick magnetic resonance imaging (MRI) scans with low contrast-to-noise ratio are typically acquired for daily MRI-guided radiotherapy setup. However, for patients with head and neck (HN) cancer, these images are often insufficient for discriminating target volumes and organs at risk (OARs). In this study, we investigated a deep learning (DL) approach to generate high-quality synthetic images from low-quality images. We used 108 unique HN image sets of paired 2-minute T2-weighted scans (2mMRI) and 6-minute T2-weighted scans (6mMRI). 90 image sets (~20,000 slices) were used to train a 2-dimensional generative adversarial DL model that utilized 2mMRI as input and 6mMRI as output. Eighteen image sets were used to test model performance. Similarity metrics, including the mean squared error (MSE), structural similarity index (SSIM), and peak signal-to-noise ratio (PSNR) were calculated between normalized synthetic 6mMRI and ground-truth 6mMRI for all test cases. In addition, a previously trained OAR DL auto-segmentation model was used to segment the right parotid gland, left parotid gland, and mandible on all test case images. Dice similarity coefficients (DSC) were calculated between 2mMRI and either ground-truth 6mMRI or synthetic 6mMRI for each OAR; two one-sided t-tests were applied between the ground-truth and synthetic 6mMRI to determine equivalence. Finally, a visual Turing test using paired ground-truth and synthetic 6mMRI was performed using three clinician observers; the percentage of images that were correctly identified was compared to random chance using proportion equivalence tests. The median similarity metrics across the whole images were 0.19, 0.93, and 33.14 for MSE, SSIM, and PSNR, respectively. The median of DSCs comparing ground-truth vs. synthetic 6mMRI auto-segmented OARs were 0.86 vs. 0.85, 0.84 vs. 0.84, and 0.82 vs. 0.85 for the right parotid gland, left parotid gland, and mandible, respectively (equivalence p<0.05 for all OARs). The percent of images correctly identified was equivalent to chance (p<0.05 for all observers). Using 2mMRI inputs, we demonstrate that DL-generated synthetic 6mMRI outputs have high similarity to ground-truth 6mMRI, but further improvements can be made. Our study facilitates the clinical incorporation of synthetic MRI in MRI-guided radiotherapy.

Comparison of an Auto-Segmentation Model Using a Dual-Path Convolutional Neural Network to Intra- and Inter-Operator High-Risk Clinical Target Volumes for Tandem-and-Ovoid Brachytherapy

<h3>Purpose/Objective(s)</h3> To compare an auto-segmentation model for defining high-risk clinical target volumes (HR-CTVs) for tandem-and-ovoid (T&O) brachytherapy to intra- and inter-operator volumes. <h3>Materials/Methods</h3> Pre-implant T2-weighted MRI and post-implant CT images for 65 (48 for training, 8 for validation, and 9 for testing) patients who underwent T&O brachytherapy for FIGO IB-IIIC cervical cancer were included. The post-implant CT was aligned to the MRI using rigid registration and HR-CTVs were manually contoured or reviewed by a fellowship trained radiation oncologist (FRO) on both the CT and MRI. A dual-path 3D asymmetric convolutional neural network (CNN) architecture was built to extract CT (entire CT volume) and MRI features (masked HR-CTV_MR volume), and was optimized with an asymmetric weighting of 18:6 for CT:MRI. Voxel-based dice similarity coefficients (DSC_V) were used to evaluate the model performance and cross-validation was performed to assess model stability. The study cohort was divided into a small (< 20 cc), medium (20-40 cc), and large (> 40 cc) tumor group based on the HR-CTV_CT for further model evaluation. To determine intra-operator variability, 10 patients were randomly selected from each of the volume groups and contours from the first two fractions of treatment were reviewed. For inter-operator variability, four patients were randomly selected from each of the volume groups and were independently contoured by two additional radiation oncologists (RO1 and RO2). DSC_V were analyzed between the intra-operator contours, inter-operator contours (RO1 vs FRO and RO2 vs FRO), and auto-segmentation contours (auto-segmentation vs FRO). <h3>Results</h3> The dual-path asymmetric auto-segmentation model yielded a favorable contouring result with a mean DSC_V (auto-segmentation vs. FRO) of 0.76 ± 0.06. This result was numerically superior to inter-operator uncertainties for RO1 vs FRO (DSC_inter=0.68 ± 0.11) and RO2 vs FRO (DSC_inter=0.69 ± 0.07), as well as the mean intra-operator agreement of FRO (DSC_intra=0.74 ± 0.07). However, the differences were statistically insignificant with p values of 0.14, 0.08, 0.72, respectively. DSC_V of volumetric subgroups of small, medium, and large tumors are shown in Table 1<b>.</b> <h3>Conclusion</h3> An asymmetric dual-path CNN model utilizing pre-implant MRI (masked by HR-CTV_MR) and post-implant CT images was successfully developed for auto-segmentation of HR-CTV_CT for tandem-and-ovoid brachytherapy patients with a similar DSC_V compared to intra-operator and inter-operator contours.

Generation of Synthetic 6-Minute MRI Scans from 2-Minute MRI Scans for Use in Head and Neck Cancer Adaptive Radiotherapy

<h3>Purpose/Objective(s)</h3> In clinical workflows for radiation therapy (RT) using integrated MRI-linear accelerator devices, quick, low-resolution images are often acquired for on-board setup imaging to minimize treatment times. However, for patients with head and neck cancer (HNC), these quick setup images are not always sufficient for discrimination of target structures and organs at risk (OAR). In this study, we investigate a deep learning (DL) approach to synthetically generate high resolution scans from low resolution scans, which we hypothesize are not significantly different from ground-truth high resolution scans. <h3>Materials/Methods</h3> Paired T2-weighted 2-minute MRI scans (2mMRI) and 6-minute MRI scans (6mMRI) acquired during the same imaging session were collected for 39 cases (36 patients with HNC; 3 volunteers). 21 cases were used to train a 2D DL generative adversarial network that utilized 2mMRI as input and 6mMRI as output. 18 cases were used to test the performance of the model. Global intensity metrics (normalized mean squared error [NMSE], structural similarity index [SSIM], and peak signal to noise ratio [PSNR]) were calculated between synthetic 6mMRI and ground-truth 6mMRI for all test cases. Additionally, a HNC OAR DL auto-segmentation model previously trained on independent 2mMRI was used to segment the right parotid gland, left parotid gland, and mandible on synthetic 6mMRI, ground-truth 6mMRI, and ground-truth 2mMRI for all test cases. Dice similarity coefficient (DSC) values were calculated between ground-truth 2mMRI and either synthetic 6mMRI or ground-truth 6mMRI for each OAR; Wilcoxon signed rank tests were applied between DSC values of the synthetic 6mMRI and the ground-truth 6mMRI for each OAR to determine any significant differences. <h3>Results</h3> Each synthetic 6mMRI took approximately 5-10 seconds to generate. Mean (standard deviation) NMSE, SSIM, and PSNR values for test cases were 0.44 (0.06), 0.90 (0.03), and 33.35 (1.60), respectively. The DSC values comparing synthetic vs. ground-truth 6mMRI auto-segmented OARs were 0.80 (0.07) vs. 0.81 (0.11), 0.83 (0.07) vs. 0.82 (0.14), and 0.82 (0.08) vs. 0.84 (0.08) for the right parotid gland, left parotid gland, and mandible, respectively; DSC values were not significantly different (p>0.05 for all OARs). <h3>Conclusion</h3> We demonstrate DL-generated synthetic 6mMRI that have low global intensity value differences compared with ground-truth 6mMRI. Moreover, auto-segmented OARs demonstrate non-significant differences when evaluated using paired synthetic or ground-truth 6mMRI. Future analysis will include the incorporation of additional training data, utilization of bias field correction, and application of human Turing test evaluation. Our study facilitates the clinical incorporation of MR-guided HNC adaptive RT using reduced acquisition times.

Prediction of Patients at Risk of Pelvic Insufficiency Fractures Following Pelvic Radiotherapy

<h3>Purpose/Objective(s)</h3> Radiotherapy-induced pelvic insufficiency fractures (PIFs), historically under-reported, can lead to significant morbidity. Using deep-learning techniques, this study develops a fully automated predictive tool to identify patients who are at high-risk of developing PIFs following radiotherapy for gynecological cancer. <h3>Materials/Methods</h3> Retrospective clinical, dosimetry and imaging data from 330 patients receiving pelvic radiotherapy for any primary gynecological cancer between January 2012 and January 2021 was collected (study approved by local ethical committee). Patient electronic records were reviewed; PIFs were diagnosed radiologically using computed tomography (CT) and magnetic resonance imaging (MRI). An auto-segmentation model for the sacrum and whole pelvis on CT-imaging was developed using a UNETR architecture to aid evaluation of pelvic radiotherapy dose and CT density. The model was validated by comparing the derived dose-volume histograms (DVHs) with those derived from manual contours. Chi-squared and Mann-Whitney tests were performed to determine associations between derived potential risk factors and PIF. Using Elastic Net Logistic Regression modelling a predictive tool to identify patients at risk of developing PIF was developed and internally validated. <h3>Results</h3> Among the 330 patients, the median follow-up was 36.6 months, and median age was 65 years. The PIF incidence was 10.9% and median interval from radiotherapy to PIF was 16 months. 86% of fractures occurred in the sacrum, 56% of patients had multiple fractures and 64% of patients were symptomatic. The auto-contouring tool of the sacrum and whole pelvis was clinically validated and, against gold-standard contours, achieved a mean DICE coefficient of 0.94, Average Surface Distance (ASD) of 0.65, and maximum absolute error for DVH curves of 3.1%. Univariant analysis found that site of primary disease, body mass index, prior diagnosis of osteopenia or osteoporosis, use of concurrent chemotherapy, V_55Gy (volume of the sacrum and whole pelvis receiving 55 Gy), and lower CT density were statistically associated with the development of PIF (p<0.05). The fully automated predictive tool to identify patients at risk of PIF using solely imaging and dosimetry data had similar accuracy to models including additional clinical features; both had a receiver-operating-characteristic (ROC) area-under-the-curve (AUC) of 0.761. The prediction tool has a sensitivity of 66%, specificity of 72% and a negative predictive value of 95%. <h3>Conclusion</h3> PIF is not uncommon in this setting, often symptomatic and strongly associated with low bone density on CT. Prediction and prevention of PIF would have an immense impact on patients and healthcare systems. We propose an automated predictive tool to identify patients at high risk, for whom effective prophylactic treatments are needed.

Association Between Cardiovascular Substructure Dose and Death Without Progression after Chemoradiation for Locally Advanced Non-Small Cell Lung Cancer

<h3>Purpose/Objective(s)</h3> Prior work has focused on associations between radiotherapy dose to cardiovascular (CV) substructures and either overall survival (OS) or cardiac events. Death without progression (DWP) is a potentially more specific endpoint than OS for understanding radiation toxicity and easier to record than cardiac events. We assessed associations of CV substructure dose with DWP and OS after chemoradiation (CRT) for locally advanced NSCLC (LA-NSCLC). <h3>Materials/Methods</h3> We retrospectively reviewed records of 187 patients with LA-NSCLC who received definitive CRT (median 66.6 Gy) from 2008-2016 at a single institution. CV substructures, including both atria (BA), right atrium, left atrium (LA), both ventricles (BV), right ventricle (RV), left ventricle, pericardium, aorta, superior vena cava, and pulmonary artery, were defined on radiation simulation scans using a deep learning auto-segmentation model. Dose-volume histogram (DVH) metrics to CV substructures (mean dose, volume receiving ≥5 Gy [V5], V30, V50), whole heart, lung, and esophagus were extracted. DWP was modelled with Fine-Gray method; disease progression was a competing event. OS was modelled with Cox regression. For each CV substructure, associations between each DVH metric and DWP and OS were assessed, and the candidate DVH metric with the lowest p value was promoted to multivariable analysis. Multivariable models with the same baseline covariates but different DVH metrics were generated and ranked by Akaike information criterion (AIC) to select "best" models. Cutpoint analyses were done with Contal and O'Quigley method. <h3>Results</h3> 98 patients (52%) received proton therapy and 89 (48%) received photon therapy (68/89 intensity-modulated radiation therapy). At median follow-up of 28.8 months, 143 (76%) patients had died; overall, 25 (13%) experienced DWP. Causes of DWP included respiratory failure due to heart failure, COPD, pneumonia, or aspiration (n=10), out-of-hospital cardiopulmonary arrest (n=4), undifferentiated sepsis (n=2), probable radiation pneumonitis (n=1), esophagopleural fistula (n=1), and unknown (n=7). On multivariable analysis for DWP, best model included RV mean dose (AIC 222); 2^nd best included BV V5 (AIC 231). 3-year cumulative incidence of DWP was 21.3% vs 5.6% for RV mean dose ≥5.5 Gy vs <5.5 Gy (p=0.001). On multivariable analysis for OS, best model included LA V5 (AIC 1277.8); 2^nd best included BA V5 (AIC 1278.5). 3-year OS was 25.6% vs 46.3% for LA V5 ≥74.5% vs <74.5% (p=0.002). Associations were stronger among and driven by the photon subgroup. Proton therapy delivered lower dose to multiple CV substructures (e.g., RV mean dose, median 0.03 Gy vs 7.7 Gy; and LA V5, median 46.1% vs 78.1%; both p<0.0001). <h3>Conclusion</h3> Radiation dose to CV substructures showed different associations with DWP and OS. These results suggest DWP may be a meaningful endpoint for understanding CRT toxicity in LA-NSCLC.

Explainable Boosting Machine for Choosing Radiation Dose-Volume Constraints on Cardio-Pulmonary Substructures Associated with Overall Survival in the NRG Oncology RTOG 0617 Clinical Trial

<h3>Purpose/Objective(s)</h3> Black box (BB) machine learning models are frequently used for decision making in healthcare. Although posthoc model-agnostic explanations can improve the BB model's interpretability, they are often unreliable. Explainable boosting machine (EBM) is a glass box model that allows for model-specific interpretation of each feature's contribution to outcome prediction. We used EBM for choosing radiation dose-volume constraints (DVCs) on cardio-pulmonary substructures (CPSs) associated with overall survival (OS) in locally advanced non-small cell lung cancer (NSCLC) radiotherapy. <h3>Materials/Methods</h3> A total of 379 NSCLC patients enrolled in RTOG 0617 were randomly split into training/validation subsets (n=230/149). A deep learning autosegmentation model was used for segmenting 12 CPSs: heart, atria, ventricles, aorta, left ventricle/atrium (LV/LA), right ventricle/atrium (RV/RA), inferior/superior vena cava, pulmonary artery, and pericardium. Contours of peri (=pericardium-heart), rest [=heart-(LV|RV|LA|RA)], clinical target volume and lungs were also obtained. From each structure, dose-volume histogram (DVH) features including min/mean dose to the hottest x% volume (Dx%[Gy]/MOHx%[Gy]; x was 5-100 in 5% increments) and min/mean/max dose as well as percent volume receiving at least xGy (VxGy[%]; x was 5-75 in 5Gy increments) were extracted. Unsupervised multivariate multiblock regression was used to select DVH features in each structure that were canonically correlated with 38 clinical/dosimetric/outcome variables taken from the NTCN/NCORP data archive for RTOG 0617. Additional feature selection for OS analysis was performed using stability selection supervised principal component analysis and Boruta. Algorithm for survival stacking was implemented for training EBM by casting survival analysis as a classification problem. Discrimination performance of EBM for OS in the validation set was evaluated by Harrell's c-index and compared to that of Cox proportional hazards (CPH) and 3 BB models: random survival forest (RSF), deep neural network (DeepSurv) and extreme gradient boosting (XGB). Shapley additive explanations (SHAP) were used to explain their predictions. <h3>Results</h3> Selected features included RA_V55Gy[%], aorta_V15Gy[%], LV_D5%[Gy] and pericardium_D20%[Gy]. EBM and all 3 BB models ranked LV_D5%[Gy] as the most important feature, while CPH model ranked second. EBM achieved the best performance for predicting OS, followed by DeepSurv, XGB, RSF and CPH (c-index=0.657, 0.646, 0.643, 0.632 and 0.630). Global explanations of EBM suggested that LV_D5%[Gy]<17.2Gy, pericardium_D20%[Gy]<52.6Gy, RA_V55Gy[%]<12.9% and aorta_V15Gy[%]<49.4% for improved OS. Although SHAP values of CPH and all 3 BB models showed similar trends, they were unable to assist in the precise selection of DVCs. <h3>Conclusion</h3> EBM can be used for modeling OS while also providing guidance on CPS DVC selection for optimal cardiac toxicity management in lung cancer radiotherapy.

Auto-segmentation of important centers of growth in the pediatric skeleton to consider during radiation therapy based on deep learning.

Routinely delineating of important skeletal growth centers is imperative to mitigate radiation-induced growth abnormalities for pediatric cancer patients treated with radiotherapy. However, it is hindered by several practical problems, including difficult identification, time consumption, and inter-practitioner variability. The goal of this study was to construct and evaluate a novel Triplet-Attention U-Net (TAU-Net)-based auto-segmentation model for important skeletal growth centers in childhood cancer radiotherapy, concentrating on the accuracy and time efficiency. A total of 107 childhood cancer patients fulfilled the eligibility criteria were enrolled in the training cohort (N=80) and test cohort (N=27). The craniofacial growth plates, shoulder growth centers, and pelvic ossification centers, with a total of 19 structures in the three groups, were manually delineated by two experienced radiation oncologists on axial, coronal, and sagittal computed tomography images. Modified from U-Net, the proposed TAU-Net has one main branch and two bypass branches, receiving semantic information of three adjacent slices to predict the target structure. With supervised deep learning, the skeletal growth centers contouring of each group was generated by three different auto-segmentation models: U-Net, V-Net, and the proposed TAU-Net. Dice similarity coefficient (DSC) and Hausdorff distance 95% (HD95) were used to evaluate the accuracy of three auto-segmentation models. The time spent on performing manual tasks and manually correcting auto-contouring generated by TAU-Net was recorded. The paired t-test was used to compare the statistical differences in delineation quality and time efficiency. Among the three groups, including craniofacial growth plates, shoulder growth centers, and pelvic ossification centers groups, TAU-Net had demonstrated highly acceptable performance (the average DSC=0.77, 0.87, and 0.83 for each group; the average HD95=2.28, 2.07, and 2.86mm for each group). In the overall evaluation of 19 regions of interest (ROIs) in the test cohort, TAU-Net had an overwhelming advantage over U-Net (63.2% ROIs in DSC and 31.6% ROIs in HD95, p=0.001-0.042) and V-Net (94.7% ROIs in DSC and 36.8% ROIs in HD95, p=0.001-0.040). With an average time of 52.2min for manual delineation, the average time saved to adjust TAU-Net-generated contours was 37.6min (p<0.001), a 72% reduction. Deep learning-based models have presented enormous potential for the auto-segmentation of important growth centers in pediatric skeleton, where the proposed TAU-Net outperformed the U-Net and V-Net in geometrical precision for the majority status.

Evaluation of auto-segmentation for EBRT planning structures using deep learning-based workflow on cervical cancer

Deep learning (DL) based approach aims to construct a full workflow solution for cervical cancer with external beam radiation therapy (EBRT) and brachytherapy (BT). The purpose of this study was to evaluate the accuracy of EBRT planning structures derived from DL based auto-segmentation compared with standard manual delineation. Auto-segmentation model based on convolutional neural networks (CNN) was developed to delineate clinical target volumes (CTVs) and organs at risk (OARs) in cervical cancer radiotherapy. A total of 300 retrospective patients from multiple cancer centers were used to train and validate the model, and 75 independent cases were selected as testing data. The accuracy of auto-segmented contours were evaluated using geometric and dosimetric metrics including dice similarity coefficient (DSC), 95% hausdorff distance (95%HD), jaccard coefficient (JC) and dose-volume index (DVI). The correlation between geometric metrics and dosimetric difference was performed by Spearman’s correlation analysis. The right and left kidney, bladder, right and left femoral head showed superior geometric accuracy (DSC: 0.88–0.93; 95%HD: 1.03 mm–2.96 mm; JC: 0.78–0.88), and the Bland–Altman test obtained dose agreement for these contours (P > 0.05) between manual and DL based methods. Wilcoxon’s signed-rank test indicated significant dosimetric differences in CTV, spinal cord and pelvic bone (P < 0.001). A strong correlation between the mean dose of pelvic bone and its 95%HD (R = 0.843, P < 0.001) was found in Spearman’s correlation analysis, and the remaining structures showed weak link between dosimetric difference and all of geometric metrics. Our auto-segmentation achieved a satisfied agreement for most EBRT planning structures, although the clinical acceptance of CTV was a concern. DL based auto-segmentation was an essential component in cervical cancer workflow which would generate the accurate contouring.

A Prior Knowledge-Guided, Deep Learning-Based Semiautomatic Segmentation for Complex Anatomy on Magnetic Resonance Imaging

Despite recent substantial improvement in autosegmentation using deep learning (DL) methods, labor-intensive and time-consuming slice-by-slice manual editing is often needed, particularly for complex anatomy (eg, abdominal organs). This work aimed to develop a fast, prior knowledge-guided DL semiautomatic segmentation (DL-SAS) method for complex structures on abdominal magnetic resonance imaging (MRI) scans. A novel application using contours on an adjacent slice as a prior knowledge informant in a 2-dimensional UNet DL model to guide autosegmentation for a subsequent slice was implemented for DL-SAS. A generalized, instead of organ-specific, DL-SAS model was trained and tested for abdominal organs on T2-weighted MRI scans collected from 75 patients (65 for training and 10 for testing). The DL-SAS model performance was compared with 3 common autocontouring methods (linear interpolation, rigid propagation, and a full 3-dimensional DL autosegmentation model trained with the same training data set) based on various quantitative metrics including the Dice similarity coefficient (DSC) and ratio of acceptable slices (ROA) using paired t tests. For the 10 testing cases, the DL-SAS model performed best with the slice interval (SI) of 1, resulting in an average DSC of 0.93 ± 0.02, 0.92 ± 0.02, 0.91 ± 0.02, 0.88 ± 0.03, and 0.87 ± 0.02 for the large bowel, stomach, small bowel, duodenum, and pancreas, respectively. The performance decreased with increased SIs from the guidance slice. The DL-SAS method performed significantly better (P < .05) than the other 3 methods. The ROA values were in the range of 48% to 66% for all the organs with an SI of 1 for DL-SAS, higher than those for linear interpolation (31%-57% for an SI of 1) and DL auto-segmentation (16%-51%). The developed DL-SAS model segmented complex abdominal structures on MRI with high accuracy and efficiency and may be implemented as an interactive manual contouring tool or a contour editing tool in conjunction with a full autosegmentation process, facilitating fast and accurate segmentation for MRI-guided online adaptive radiation therapy.

Automatic delineation of organs at risk in non-small cell lung cancer radiotherapy based on deep learning networks*

Abstract Objective To introduce an end-to-end automatic segmentation method for organs at risk (OARs) in chest computed tomography (CT) images based on dense connection deep learning and to provide an accurate auto-segmentation model to reduce the workload on radiation oncologists. Methods CT images of 36 lung cancer cases were included in this study. Of these, 27 cases were randomly selected as the training set, six cases as the validation set, and nine cases as the testing set. The left and right lungs, cord, and heart were auto-segmented, and the training time was set to approximately 5 h. The testing set was evaluated using geometric metrics including the Dice similarity coefficient (DSC), 95% Hausdorff distance (HD95), and average surface distance (ASD). Thereafter, two sets of treatment plans were optimized based on manually contoured OARs and automatically contoured OARs, respectively. Dosimetric parameters including Dmax and Vx of the OARs were obtained and compared. Results The proposed model was superior to U-Net in terms of the DSC, HD95, and ASD, although there was no significant difference in the segmentation results yielded by both networks (P &gt; 0.05). Compared to manual segmentation, auto-segmentation significantly reduced the segmentation time by nearly 40.7% (P &lt; 0.05). Moreover, the differences in dose-volume parameters between the two sets of plans were not statistically significant (P &gt; 0.05). Conclusion The bilateral lung, cord, and heart could be accurately delineated using the DenseNet-based deep learning method. Thus, feature map reuse can be a novel approach to medical image auto-segmentation.

Stepwise deep neural network (stepwise-net) for head and neck auto-segmentation on CT images

ObjectiveThe current study aims to propose the auto-segmentation model on CT images of head and neck cancer using a stepwise deep neural network (stepwise-net). Material and methodsSix normal tissue structures in the head and neck region of 3D CT images: Brainstem, optic nerve, parotid glands (left and right), and submandibular glands (left and right) were segmented with deep learning. In addition to a conventional convolutional neural network (CNN) on U-net, a stepwise neural network (stepwise-network) was developed. The stepwise-network was based on 3D FCN. We designed two networks in the stepwise-network. One is identifying the target region for the segmentation with the low-resolution images. Then, the target region is cropped, which used for the input image for the prediction of the segmentation. These were compared with a clinical used atlas-based segmentation. ResultsThe DSCs of the stepwise-net was significantly higher than the atlas-based method for all organ at risk structures. Similarly, the JSCs of the stepwise-net was significantly higher than the atlas-based methods for all organ at risk structures. The Hausdorff distance (HD) was significantly smaller than the atlas-based method for all organ at-risk structures. For the comparison of the stepwise-net and U-net, the stepwise-net had a higher DSC and JSC and a smaller HD than the conventional U-net. ConclusionsWe found that the stepwise-network plays a role is superior to conventional U-net-based and atlas-based segmentation. Our proposed model that is a potentially valuable method for improving the efficiency of head and neck radiotherapy treatment planning.

Deep learning-based two-step organs at risk auto-segmentation model for brachytherapy planning in parotid gland carcinoma.

Delineation of organs at risk (OARs) represents a crucial step for both tailored delivery of radiation doses and prevention of radiation-induced toxicity in brachytherapy. Due to lack of studies on auto-segmentation methods in head and neck cancers, our study proposed a deep learning-based two-step approach for auto-segmentation of organs at risk in parotid carcinoma brachytherapy. Computed tomography images of 200 patients with parotid gland carcinoma were used to train and evaluate our in-house developed two-step 3D nnU-Net-based model for OARs auto-segmentation. OARs during brachytherapy were defined as the auricula, condyle process, skin, mastoid process, external auditory canal, and mandibular ramus. Auto-segmentation results were compared to those of manual segmentation by expert oncologists. Accuracy was quantitatively evaluated in terms of dice similarity coefficient (DSC), Jaccard index, 95th-percentile Hausdorff distance (95HD), and precision and recall. Qualitative evaluation of auto-segmentation results was also performed. The mean DSC values of each OAR were 0.88, 0.91, 0.75, 0.89, 0.74, and 0.93, respectively, indicating close resemblance of auto-segmentation results to those of manual contouring. In addition, auto-segmentation could be completed within a minute, as compared with manual segmentation, which required over 20 minutes. All generated results were deemed clinically acceptable. Our proposed deep learning-based two-step OARs auto-segmentation model demonstrated high efficiency and good agreement with gold standard manual contours. Thereby, this novel approach carries the potential in expediting the treatment planning process of brachytherapy for parotid gland cancers, while allowing for more accurate radiation delivery to minimize toxicity.

Deep Learning Auto-Segmentation Model for HN005 Contour Quality Assurance

Ensuring the quality of radiotherapy is essential for the successful conduct of clinical trials. The accurate delineation of organs at risks and targets compliance with protocol definitions is critical and historically time-consuming. We established a workflow utilizing deep learning-based AI auto-segmentation models for automated quality assurances (QA) for a head and neck multi-center clinical trial. Both in-house developed and commercially available AI models are tested. We report on the preliminary results.Radiotherapy digital data of one-hundred-eighty-four patients enrolled in NRG-HN002 were used for this study. One-hundred-eighty-four CT and RT structures were reviewed and used for model training. Auto-segmentation models were trained for the mandible, brain stem, spinal cord, oral cavity, parotids, LarynxGSL, lips, esophagus submandibular glands, and pharynx with cascaded astrous convolution (CAC) and spatial pyramid pooling (SPP) module enhance convolutional neural networks (CNN). Twenty patients submitted to HN005 were selected for model testing to establish the QA workflow. AI (CNN) segmentation tools trained with separate sets of data from Mirada (DLCExpertTM) were also included in the testing process. Dice coefficient thresholds were established for detecting submitted segmentations with major variations with a 95% confidence interval.Both in-house and commercial auto-segmentations demonstrated high consensus; and, when compared with submitted segmentations, achieved a higher than 0.7 Dice similarity coefficient for the mandible, brain stem, spinal cord, parotid, and submandibular glands; higher than 0.5 dice for esophagus, oral cavity, larynx, and lips. Submitted segmentation with the dice coefficient lower than the established threshold often has missing slices, redundant slices, or erroneous overlapping with other structures. The detection specificity was found to be higher than 0.9. Due to the irregular shape of the pharynx and its overlapping with the target volume, the auto-segmentations have not achieved sufficient accuracy (with dice below 0.2).Both commercial and in-house models demonstrate high specificity for submitted contour error detection for all the HN005 required OARs except for the pharynx. Comparison between AI and expert reviews will be included in future studies.

A Double-Blind Study to Evaluate the Feasibility of Using AI-Powered Auto-Segmentation in Prostate Cancer Treatment

To evaluate whether the contours generated by an AI-powered auto-segmentation software can be used directly for prostate cancer radiation therapy.A total of 23 prostate cases treated between 2018 and 2020 were retrospectively randomly selected. The prostate and surrounding organs at risk (OARs) were delineated on computed tomography (CT) scans by a radiation oncologist (RO) as ground-truth (GT) contour and by Artificial Intelligence (AI) model as AI contour. The GT and AI contour sets were anonymized randomly to be evaluated by a different RO. To eliminate the planner's bias, RapidPlans were used to generate treatment plans using AI and GT contours. The OAR dose on both plans were evaluated on GT contour sets to determine whether the AI contour can produce similar results as the GT contour.In subjective double-blind test, AI contours show that 95.7% (22 out of 23) are scored as either "Great" (34.8%) or "Acceptable without changes" (60.9%), and only one case (4.3%) is evaluated as "Acceptable with minor changes". Note that the GT contours also have one case in the "minor change" category. Totally 69.6% (16 of 23) AI contours were considered equal or better than the GT counterparts. Paired t-tests showed no significant differences in dosimetric plan quality yielded by the AI and GT for the prostate, rectum, femoral heads, and penile bulb (P > 0.05). Statistically significant difference (P < 0.05) was observed on Bladder and Seminal Vesicles. However, no plan fails the RTOG-0815 organ tolerances.The investigated auto-segmentation model for prostate anatomy provides compatible performance to manually delineated contours by a Radiation Oncologist.

Cardiac Substructure Delineation Based on Synthetic Contrast-Enhanced CT Generation Using Deep Convolutional Neural Network in Breast Cancer Radiation Therapy

Although radiation-induced cardiac toxicity is an important issue in breast radiation therapy (RT), the dose relationship between cardiac structures and its toxicity has not been fully elucidated, partially because many centers do not routinely administer intravenous (IV) contrast for breast RT, which preclude substructure delineation and detection of potential correlations. In this study, we attempted to generate the synthetic contrast-enhanced CT (CECTsyn) from the non-contrast CT (NCTreal) using deep convolutional neural network (DCNN) and to investigate whether CECTsyn can take a supportive role in case the studies for the cardiac toxicity induced by radiation is needed for the patient whose CECTreal cannot be obtained.For this study, 22 NCTreal-CECTreal cardiac scan-pairs have been prepared of which the volume size was ∼512 × 512 × 400 and the resolution was ∼0.75 × 0.75 × 1 mm3 on average. Of the 22 datasets, 13/2/7 were used for training, validation, and testing, respectively. After matching the structure of paired scans using the deformable image registration (DIR), the area near the heart was cropped and used for training the deep learning model. We adopted the modified 2D fully convolution DenseNet (FC-DenseNet) as our backbone and trained it in the conditional generative adversarial network (cGAN) framework. The similarity between CECTsyn and CECTreal was evaluated first, and all NCTreal,CECTsyn, and CECTreal were applied to pre-trained cardiac auto-segmentation models to obtain the substructures of the heart, which is used for dose assessments in each of them. Finally, dose evaluations were conducted on CECTsyn and CECTreal with the dose distributions which are from clinical treatment plans and the manually modified contours of cardiac substructures.Firstly, the peak signal-to-noise ratio (PSNR) and the structural similarity index measure (SSIM) between CECTsyn and CECTreal were 23.96 and 0.772, whereas those of NCTreal and CECTreal were 22.25and 0.748, respectively. Secondly, the results of applying the pre-trained cardiac auto-segmentation model were obtained, and the dice similarity coefficient (DSC) of CECTsyn was 0.762 on average which is superior to 0.267 of NCTreal and comparable to 0.843 of CECTreal. We could distinguish the L/R ventricles, L/R atriums, left anterior descending artery (LAD), and right coronary artery (RCA) clearly on CECTsyn. Finally, the dose differences between CECTsyn and CECTreal were on average 0.4/2.0 Gy in Dmean and Dmax, and 2.1/1.3/0.6/0.4/0.3/0.1/0.0% in V5 Gy, V10 Gy, V20 Gy, V30 Gy, V40 Gy, V50 Gy, and V60 Gy, respectively.DCNN model proposed here showed the feasibility of CECTsyn generation from NCTreal and potential for cardiac substructure delineation such as ventricles, atriums, LAD, and RCA on NCTreal for breast RT. Furthermore, it is shown that CECTsyn can play a supportive role when the studies for the radiation-induced cardiac toxicity is needed.

Head and Neck Oropharyngeal GTV Autosegmentation: Combining nnU-Net With Shape Representation Loss Driven by a Variational Autoencoder Model

Autosegmentation models for tumors are becoming increasingly important across several domains. GTV autosegmentations can facilitate faster radiation treatment planning, standardize delineation for radiomics, and provide contour consistency across centers. Oropharyngeal squamous cell carcinomas (SCC) of the head and neck serve as a challenging tumor to segment given the underlying anatomy and imaging characteristics. We build upon nnU-Net, a new medical image segmentation framework that uses a set of heuristics based on the training data type (modality, size etc.) and available resources (GPU RAM etc.) to optimally select pre-processing steps, model parameters and model ensembling techniques. Previous work has shown that latent-space based shape representation (obtained from stacked auto-encoder) of organs at risk within the head and neck, when used in loss function, improves OAR segmentation accuracy. Here, we apply nnU-Net for autosegmentation of oropharyngeal SCCs GTVs. And we adapted nnU-Net to utilize a shape representation loss function to further constrain the GTV segmentations.The 2020 HEad and NeCK TumOR (HECKTOR) Challenge at the MICCAI 2020 conference provided a dataset of 201 patients with oropharyngeal caner with pre-radiation treatment CT and PET images from 4 different medical centers. We applied autosegmentation to this HECKTOR dataset, and we tested on an independent dataset consisting of 53 patients from a fifth and separate institution. We trained a model with the original nnU-Net framework on the HECKTOR CT and PET training data. To generate a shape representation model, we built a variational autoencoder with input as the cropped and centered ground truth GTV binary masks. We performed hyperparameter optimization and found an optimal latent space size of 50, a network depth of 5 convolutional layers, and a KL loss weight of 0.001. To constrain the output of nnU-Net, we computed the L1-loss in the latent space between the ground truth and the autosegmentation, generating a shape representation model (SRM) loss function.The latent space of the variational autoencoder encodes effectively shape information of GTVs, including size and T-staging of the tumor. The original nnU-Net model achieved a 74.7% Dice score on the test set, which placed 3rd in the post-submission HECKTOR challenge. Our SRM model constrained the segmentation and achieved a Dice of 75.9%, which placed first in the post-submission HECKTOR challenge as of February 2021.These results show that nnU-Net can be used to generate a well-performing autosegmentation model for head and neck GTV segmentation using multimodal input. We also demonstrate that the PyTorch framework can be readily altered and optimized for particular tasks. Our shape representation loss was able to improve performance and achieve the top segmentation result in the HECKTOR challenge. Further optimizations into this framework are actively being explored.

Deep Learning Auto-Segmentation on Multi-Sequence MRI for MR-Guided Adaptive Radiation Therapy

<h3>Purpose/Objective(s)</h3> Robust auto-segmentation based on daily MRI is highly desirable for MR-guided adaptive radiation therapy (MRgART). However, it is generally challenging particularly if the daily MRI is acquired with multiple sequences or if it is for imaging complex anatomy, e.g., abdomen. To overcome these issues, we developed a deep learning algorithm to segment abdominal organs quickly and automatically on multi-sequence MRI. <h3>Materials/Methods</h3> The deep learning auto-segmentation model was developed using a modified U-net, ResUnet3D, with end-to-end mapping, and was trained with 85 daily MRI sets acquired during MRgART on a 1.5T MR-Linac for 37 patients with abdominal tumors. The daily MRI sets were motion averaged images derived from 4D MRI acquired with multiple sequences, including Fast Field Echo (FFE), Balanced Turbo Field Echo (BTFE) and TFE. Each daily MRI was paired with a CT. All MRIs were pre-processed using bias field correction and intensity standardization. The ground truth contours were created manually based on existing guidelines. For the training datasets, field-of-view extension via online random 3D elastic transform/deformation was utilized to augment and improve the quality of datasets. During the model training, a patient-based z-score normalization method was employed to standardize the multi-contrast images. Two models were trained, one for 11 organs: aorta, duodenum, kidneys, liver, pancreas, spinal cord, spleen, stomach, large bowel, and small bowel, and another for 10 organs with combined large and small bowel. Performance of the obtained models was evaluated using 11 test datasets in terms of dice similarity coefficient (DSC), mean distance to agreement (MDA) and time efficiency using an Intel Xeon CPU (2.4 GHz, 64GB RAM) hardware. <h3>Results</h3> For the challenging bowels, the auto-segmentation accuracy for the model trained with the combined bowel was better, with MDA/DSC of 2.1/0.86 versus 6.8/0.76 (large bowel) and 6.1/0.68 (small bowel) for the separate-bowel model. Average DSC for the obtained contours of aorta, left kidney, right kidney, liver, spinal cord, spleen, and stomach was 0.93, 0.93, 0.94, 0.94, 0.83, 0.92 and 0.88, respectively from both models. While no significant difference in model performance was observed with BTFE versus TFE testing datasets, a slightly improved segmentation was seen on the BTFE images for pancreas and duodenum, DSC 0.71 and 0.72. This improvement may be attributed to the enhanced contrast seen in BTFE images when compared to the TFE and FFE images. The time required to generate all organs on an MRI was 20-24 seconds. <h3>Conclusion</h3> We have developed deep learning auto-segmentation models based on multi-sequence MRI sets for abdominal organs. The models performed reasonably well. With further development using large datasets, the models may be integrated in MRgART for fast, fully automated, and accurate contouring of abdominal organs on daily MRI.

A Generalizable Guided Deep Learning Auto-Segmentation Method of Pancreatic GTV on Multi-Protocol Daily MRIs for MR-Guided Adaptive Radiotherapy

Fast and consistent delineation of gross-tumor volume (GTV) on non-contrast enhanced daily MRI is critical but can be challenging in MR-guided Online Adaptive Radiation Therapy (MRgOART). Our previous works have demonstrated the potential of using deep learning (DL) method for pancreatic GTV segmentation on MRI of a single sequence. However, the application of the obtained model is limited by the variations of MRIs from different scan protocols. To overcome the protocol dependence, we propose a generalizable prior knowledge guided DL method for pancreatic GTV auto-segmentation using the guidance of patient-specific contour from the verification MRI (obtained in the treatment room before the first fraction) on subsequent multi-protocol daily MRIs.The daily MRIs acquired from 21 pancreatic cancer patients on a 1.5T MR-Linac during MRgOART were used to demonstrate the method. Two different scan protocols were used: T2 TSE sequence for nine patients and 4D Vane sequence for the remaining twelve patients. Manual contours carefully delineated on each MRI by treating physicians were utilized as ground truth. All the MRIs were pre-processed (e.g., bias corrected, normalized, cropped, and resampled). Each daily MRI was rigidly registered to the verification MRI and a total of 99 image pairs were generated. The DL network backbone used was UNet with 3 encoders interconnected with 3 decoders and a classification block. Model input was the composite of verification MRI, verification GTV mask, and daily MRI. Data augmentation methods such as translation, rotation, flip, guidance-target inversion, and rigid registration error introduction were applied for robust model training. Patient-based leave-one-out-cross-validation was employed and the model performance was evaluated using Dice similarity coefficient (DSC) compared with the ground truth.The average DSC was 0.63 ± 0.18 for all the cases studied. For the cases with GTV > 5cc, the average DSC was 0.76 ± 0.13 and 0.69 ± 0.08 for T2 TSE and 4D Vane MRIs, respectively. The results are reasonably comparable to the inter-observer variation of DSC = 0.71 estimated in a previous study. The time required to generate GTV contour on a daily MRI set was less than 1s on a hardware with an I7 CPU and a GTX 1060 GPU.The proposed prior-knowledge guided deep leaning auto-segmentation method for pancreatic GTV on daily multi-protocol non-contrast MRIs performed reasonably well even with the relatively small dataset. This proof-of-principle study suggests that, with patient-specific prior knowledge guidance, it is feasible to implement a single DL auto-segmentation model for multiple MRI sequences in MRgOART. The method can be generalized for other structures or tumors. (First and second authors contributed equally.).

Clinical application and improvement of a CNN-based autosegmentation model for clinical target volumes in cervical cancer radiotherapy.

ObjectiveClinical target volume (CTV) autosegmentation for cervical cancer is desirable for radiation therapy. Data heterogeneity and interobserver variability (IOV) limit the clinical adaptability of such methods. The adaptive method is proposed to improve the adaptability of CNN‐based autosegmentation of CTV contours in cervical cancer.MethodsThis study included 400 cervical cancer treatment planning cases with CTV delineated by radiation oncologists from three hospitals. The datasets were divided into five subdatasets (80 cases each). The cases in datasets 1, 2, and 3 were delineated by physicians A, B, and C, respectively. The cases in datasets 4 and 5 were delineated by multiple physicians. Dataset 1 was divided into training (50 cases), validation (10 cases), and testing (20 cases) cohorts, and they were used to construct the pretrained model. Datasets 2–5 were regarded as host datasets to evaluate the accuracy of the pretrained model. In the adaptive process, the pretrained model was fine‐tuned to measure improvements by gradually adding more training cases selected from the host datasets. The accuracy of the autosegmentation model on each host dataset was evaluated using the corresponding test cases. The Dice similarity coefficient (DSC) and 95% Hausdorff distance (HD_95) were used to evaluate the accuracy.ResultsBefore and after adaptive improvements, the average DSC values on the host datasets were 0.818 versus 0.882, 0.763 versus 0.810, 0.727 versus 0.772, and 0.679 versus 0.789, which are improvements of 7.82%, 6.16%, 6.19%, and 16.05%, respectively. The average HD_95 values were 11.143 mm versus 6.853 mm, 22.402 mm versus 14.076 mm, 28.145 mm versus 16.437 mm, and 33.034 mm versus 16.441 mm, which are improvements of 37.94%, 37.17%, 41.60%, and 50.23%, respectively.ConclusionThe proposed method improved the adaptability of the CNN‐based autosegmentation model when applied to host datasets.