Deep Learning-based Auto-segmentation Research Articles

Localized fine-tuning and clinical evaluation of deep-learning based auto-segmentation (DLAS) model for clinical target volume (CTV) and organs-at-risk (OAR) in rectal cancer radiotherapy

Background and purposeVarious deep learning auto-segmentation (DLAS) models have been proposed, some of which have been commercialized. However, the issue of performance degradation is notable when pretrained models are deployed in the clinic. This study aims to enhance precision of a popular commercial DLAS product in rectal cancer radiotherapy by localized fine-tuning, addressing challenges in practicality and generalizability in real-world clinical settings.Materials and methodsA total of 120 Stage II/III mid-low rectal cancer patients were retrospectively enrolled and divided into three datasets: training (n = 60), external validation (ExVal, n = 30), and generalizability evaluation (GenEva, n = 30) datasets respectively. The patients in the training and ExVal dataset were acquired on the same CT simulator, while those in GenEva were on a different CT simulator. The commercial DLAS software was first localized fine-tuned (LFT) for clinical target volume (CTV) and organs-at-risk (OAR) using the training data, and then validated on ExVal and GenEva respectively. Performance evaluation involved comparing the LFT model with the vendor-provided pretrained model (VPM) against ground truth contours, using metrics like Dice similarity coefficient (DSC), 95th Hausdorff distance (95HD), sensitivity and specificity.ResultsLFT significantly improved CTV delineation accuracy (p < 0.05) with LFT outperforming VPM in target volume, DSC, 95HD and specificity. Both models exhibited adequate accuracy for bladder and femoral heads, and LFT demonstrated significant enhancement in segmenting the more complex small intestine. We did not identify performance degradation when LFT and VPM models were applied in the GenEva dataset.ConclusionsThe necessity and potential benefits of LFT DLAS towards institution-specific model adaption is underscored. The commercial DLAS software exhibits superior accuracy once localized fine-tuned, and is highly robust to imaging equipment changes.

Clinical evaluation of a deep learning CBCT auto-segmentation software for prostate adaptive radiation therapy

PurposeAim of the present study is to characterize a deep learning-based auto-segmentation software (DL) for prostate cone beam computed tomography (CBCT) images and to evaluate its applicability in clinical adaptive radiation therapy routine. Materials and methodsTen patients, who received exclusive radiation therapy with definitive intent on the prostate gland and seminal vesicles, were selected. Femoral heads, bladder, rectum, prostate, and seminal vesicles were retrospectively contoured by four different expert radiation oncologists on patients CBCT, acquired during treatment. Consensus contours (CC) were generated starting from these data and compared with those created by DL with different algorithms, trained on CBCT (DL-CBCT) or computed tomography (DL-CT). Dice similarity coefficient (DSC), centre of mass (COM) shift and volume relative variation (VRV) were chosen as comparison metrics. Since no tolerance limit can be defined, results were also compared with the inter-operator variability (IOV), using the same metrics. ResultsThe best agreement between DL and CC was observed for femoral heads (DSC of 0.96 for both DL-CBCT and DL-CT). Performance worsened for low-contrast soft tissue organs: the worst results were found for seminal vesicles (DSC of 0.70 and 0.59 for DL-CBCT and DL-CT, respectively). The analysis shows that it is appropriate to use algorithms trained on the specific imaging modality. Furthermore, the statistical analysis showed that, for almost all considered structures, there is no significant difference between DL-CBCT and human operator in terms of IOV. ConclusionsThe accuracy of DL-CBCT is in accordance with CC; its use in clinical practice is justified by the comparison with the inter-operator variability.

A deep learning-based framework (Co-ReTr) for auto-segmentation of non-small cell-lung cancer in computed tomography images.

Deep learning-based auto-segmentation algorithms can improve clinical workflow by defining accurate regions of interest while reducing manual labor. Over the past decade, convolutional neural networks (CNNs) have become prominent in medical image segmentation applications. However, CNNs have limitations in learning long-range spatial dependencies due to the locality of the convolutional layers. Transformers were introduced to address this challenge. In transformers with self-attention mechanism, even the first layer of information processing makes connections between distant image locations. Our paper presents a novel framework that bridges these two unique techniques, CNNs and transformers, to segment the gross tumor volume (GTV) accurately and efficiently in computed tomography (CT) images of non-small cell-lung cancer (NSCLC) patients. Under this framework, input of multiple resolution images was used with multi-depth backbones to retain the benefits of high-resolution and low-resolution images in the deep learning architecture. Furthermore, a deformable transformer was utilized to learn the long-range dependency on the extracted features. To reduce computational complexity and to efficiently process multi-scale, multi-depth, high-resolution 3D images, this transformer pays attention to small key positions, which were identified by a self-attention mechanism. We evaluated the performance of the proposed framework on a NSCLC dataset which contains 563 training images and 113 test images. Our novel deep learning algorithm was benchmarked against five other similar deep learning models. The experimental results indicate that our proposed framework outperforms other CNN-based, transformer-based, and hybrid methods in terms of Dice score (0.92) and Hausdorff Distance (1.33). Therefore, our proposed model could potentially improve the efficiency of auto-segmentation of early-stage NSCLC during the clinical workflow. This type of framework may potentially facilitate online adaptive radiotherapy, where an efficient auto-segmentation workflow is required. Our deep learning framework, based on CNN and transformer, performs auto-segmentation efficiently and could potentially assist clinical radiotherapy workflow.

Technical note: Generalizable and promptable artificial intelligence model to augment clinical delineation in radiation oncology

Efficient and accurate delineation of organs at risk (OARs) is a critical procedure for treatment planning and dose evaluation. Deep learning-based auto-segmentation of OARs has shown promising results and is increasingly being used in radiation therapy. However, existing deep learning-based auto-segmentation approaches face two challenges in clinical practice: generalizability and human-AI interaction. A generalizable and promptable auto-segmentation model, which segments OARs of multiple disease sites simultaneously and supports on-the-fly human-AI interaction, can significantly enhance the efficiency of radiation therapy treatment planning. Meta's segment anything model (SAM) was proposed as a generalizable and promptable model for next-generation natural image segmentation. We further evaluated the performance of SAM in radiotherapy segmentation. Computed tomography (CT) images of clinical cases from four disease sites at our institute were collected: prostate, lung, gastrointestinal, and head & neck. For each case, we selected the OARs important in radiotherapy treatment planning. We then compared both the Dice coefficients and Jaccard indices derived from three distinct methods: manual delineation (ground truth), automatic segmentation using SAM's 'segment anything' mode, and automatic segmentation using SAM's 'box prompt' mode that implements manual interaction via live prompts during segmentation. Our results indicate that SAM's segment anything mode can achieve clinically acceptable segmentation results in most OARs with Dice scores higher than 0.7. SAM's box prompt mode further improves Dice scores by 0.1∼0.5. Similar results were observed for Jaccard indices. The results show that SAM performs better for prostate and lung, but worse for gastrointestinal and head & neck. When considering the size of organs and the distinctiveness of their boundaries, SAM shows better performance for large organs with distinct boundaries, such as lung and liver, and worse for smaller organs with less distinct boundaries, like parotid and cochlea. Our results demonstrate SAM's robust generalizability with consistent accuracy in automatic segmentation for radiotherapy. Furthermore, the advanced box-prompt method enables the users to augment auto-segmentation interactively and dynamically, leading to patient-specific auto-segmentation in radiation therapy. SAM's generalizability across different disease sites and different modalities makes it feasible to develop a generic auto-segmentation model in radiotherapy.

Custom-Trained Deep Learning-Based Auto-Segmentation for Male Pelvic Iterative CBCT on C-Arm Linear Accelerators

Purpose: To evaluate the clinical applicability of a commercial artificial intelligence (AI)-driven deep learning auto-segmentation (DLAS) tool on enhanced iterative cone-beam CT (iCBCT) acquisitions for intact prostate and prostate bed treatments. Methods and MaterialsDLAS models were trained using 116 iCBCT datasets with manually delineated organs-at-risk (OARs – bladder, femoral heads, and rectum) and target volumes (intact prostate and prostate bed) adhering to institution-specific contouring guidelines. An additional 25 intact prostate and prostate bed iCBCT datasets were utilized for model testing. Segmentation accuracy relative to a reference structure set was quantified using various geometric comparison metrics and qualitatively evaluated by trained physicists and physicians. These results were compared to those obtained for an additional DLAS-based model trained on planning CT (pCT) datasets and for a deformable image registration (DIR)-based automatic contour propagation method. Results: In most instances, statistically significant differences in the Dice similarity coefficient (DSC), 95% directed Hausdorff distance, and mean surface distance metrics were observed between the models, as the iCBCT-trained DLAS model outperformed the pCT-trained DLAS model and DIR-based method for all OARs and the intact prostate target volume. Mean DSC values for the proposed method were ≥0.90 for these volumes-of-interest. The iCBCT-trained DLAS model demonstrated a relatively suboptimal performance for the prostate bed segmentation, as the mean DSC value was <0.75 for this target contour. Overall, 90% of bladder, 93% of femoral head, 67% of rectum, and 92% of intact prostate contours generated by the proposed method were deemed clinically acceptable based on qualitative scoring, while approximately 63% of prostate bed contours required moderate or major manual editing to adhere to institutional contouring guidelines. Conclusion: The proposed method presents the potential for improved segmentation accuracy and efficiency when compared to the DIR-based automatic contour propagation method as commonly applied in CBCT-based dose evaluation and calculation studies.

The Association of Incidental Radiation Dose to the Heart Base with Overall Survival and Cardiac Events after Curative-intent Radiotherapy for Non-small Cell Lung Cancer: Results from the NI-HEART Study

AimsCardiac disease is a dose-limiting toxicity in non-small cell lung cancer radiotherapy. The dose to the heart base has been associated with poor survival in multiple institutional and clinical trial datasets using unsupervised, voxel-based analysis. Validation has not been undertaken in a cohort with individual patient delineations of the cardiac base or for the endpoint of cardiac events. The purpose of this study was to assess the association of heart base radiation dose with overall survival and the risk of cardiac events with individual heart base contours. Materials and methodsPatients treated between 2015 and 2020 were reviewed for baseline patient, tumour and cardiac details and both cancer and cardiac outcomes as part of the NI-HEART study. Three cardiologists verified cardiac events including atrial fibrillation, heart failure and acute coronary syndrome. Cardiac substructure delineations were completed using a validated deep learning-based autosegmentation tool and a composite cardiac base structure was generated. Cox and Fine–Gray regressions were undertaken for the risk of death and cardiac events. ResultsOf 478 eligible patients, most received 55 Gy/20 fractions (96%) without chemotherapy (58%), planned with intensity-modulated radiotherapy (71%). Pre-existing cardiovascular morbidity was common (78% two or more risk factors, 46% one or more established disease). The median follow-up was 21.1 months. Dichotomised at the median, a higher heart base Dmax was associated with poorer survival on Kaplan–Meier analysis (20.2 months versus 28.3 months; hazard ratio 1.40, 95% confidence interval 1.14–1.75, P = 0.0017) and statistical significance was retained in multivariate analyses. Furthermore, heart base Dmax was associated with pooled cardiac events in a multivariate analysis (hazard ratio 1.75, 95% confidence interval 1.03–2.97, P = 0.04). ConclusionsHeart base Dmax was associated with the rate of death and cardiac events after adjusting for patient, tumour and cardiovascular factors in the NI-HEART study. This validates the findings from previous unsupervised analytical approaches. The heart base could be considered as a potential sub-organ at risk towards reducing radiation cardiotoxicity.

Clinical Evaluation of Deep Learning-Based Auto-Segmentation toward MRI-Only Prostate Radiotherapy.

Due to its superior soft-tissue contrast, MRI is the imaging modality of choice for target and organs-at-risk delineation in prostate radiotherapy. A recent advanced MR scanner allows for an MRI-only workflow owing to its ability to generate synthetic CT (MRCAT) that enables accurate dose calculation and treatment planning. While auto-segmentation algorithms for CT images are increasingly being adopted in clinical practice, there are fewer such algorithms for MRI images. The purpose of this study was to evaluate a deep learning-based auto-segmentation tool (from MVision AI) for both MRI and MRCAT images. We hypothesize that such algorithms could produce accurate contours that require little manual editing, resulting in significant time saving and workflow efficiency in MRI-only treatment planning. T2 MRI and MRCAT image pairs (acquired on a Philips Ingenia 1.5T MRI Scanner) for 10 prostate patients treated in an MRI-only workflow at our institution were retrospectively collected with IRB approval. These image volumes were then segmented using deep learning-based auto-segmentation models trained for MRI and CT images, respectively. The structures were manually edited by an experienced physician in a commercial imaging informatics system (Velocity). Finally, the auto-segmented and the post-edit structures were compared using Dice Similarity Coefficient (DSC) and 95% Hausdorff Distance (HD95). Excellent agreement was found between the auto-segmented and post-edit structures. For MRCAT, the average DSC was 0.92±0.03, 0.99±0.00, 0.90±0.06, 0.89±0.07, 0.89±0.06, 0.95±0.04, and 0.96±0.03 for prostate, bladder, rectum, penile bulb, seminal vesicles, left femur, and right femur, respectively. The average HD95 for these structures were 0.65±0.51mm, 0.01±0.01mm, 2.07±1.68mm, 1.10±0.91mm, 1.67±1.81mm, 0.86±1.24mm, and 0.67±1.49mm, respectively. For MRI, the average DSC was 0.96±0.02, 0.99 ±0.00, 0.96±0.03, 0.87±0.03, 0.94±0.06, and 0.89±0.11 for prostate, bladder, rectum, penile bulb, seminal vesicles, and Barrigel spacer, respectively. The average HD95 was 0.25±0.21mm, 0.00±0.00mm, 1.60±1.87mm, 1.43±0.98mm, 0.50±0.86mm, and 1.56±2.26mm, respectively. We evaluated the clinical acceptability of deep learning-based auto-segmentation models for prostate radiotherapy, one for MRI and one for MRCAT delineation. Both models demonstrated superior accuracy, and a high degree of consistency, offering the potential for significant time saving in treatment planning and improved workflow efficiency.

Patient-Level and Endpoint-Specific Clinico-Dosimetric Analysis of the Cardiac Base as a Mediator of Radiation Cardiotoxicity in Non-Small Cell Lung Cancer

Cardiac disease is a dose-limiting toxicity in non-small cell lung cancer (NSCLC) radiation therapy. Radiation dose to the cardiac base is associated with poor overall survival in several clinical studies, but has not been validated in a non-dose escalated cohort, or with individual patient delineations. In this study we examined the impact of cardiac base dose on overall survival (OS) and cardiac events (CEs), and interrogated the relationships of the substructures comprising the heart base with OS and CEs. Patients with stage I-III NSCLC treated with curative-intent radiation therapy between 2015 and 2020 at a regional cancer center were identified. Clinical notes were examined for baseline patient, tumor and cardiac details, and both cancer and cardiac outcomes. Three cardiologists verified CEs. Cardiac delineations were completed using a validated deep learning-based autosegmentation tool. Cox and Fine and Gray regressions were undertaken for the risk of death and CEs respectively, accounting for pre-specified evidence-based dose metrics and clinically relevant cardiac covariates. Most patients received 55 Gy/20# (n = 461/478, 96%) without chemotherapy (58%), planned with VMAT (51%) or IMRT (20%). Pre-existing cardiovascular morbidity was common, with 78% having ≥2 risk factors, and 46% having >1 established cardiac disease. The median follow-up was 21.1 months. Dichotomized at the median, higher heart base Dmax was associated with poorer survival on Kaplan-Meier analysis (21.6 months (95% CI 19.3-24.9) versus 29.4 months (95% CI 21.6-36.6), p = 0.021), and remained significant when statistically compared in published multivariate models. In a multivariate analysis for pooled acute CEs, heart base Dmax was associated with CEs (HR 1.75, 95% CI 1.01-1.06, p = 0.04), but this was not the case for individual CEs. Using Fine and Gray models to account for the competing risk of death, left main coronary maximum dose was associated with atrial fibrillation (p = 0.024), proximal right coronary artery V15 (p = 0.023) and mean dose (p = 0.032), and the right atrium mean dose (p = 0.029) were associated with heart failure. No dose-volume metrics were significantly associated with acute coronary syndrome. None of the constituent base substructures dose were significantly associated with death. Dose to the heart base was associated with increased mortality and an increased pooled cardiac event rate. Accounting for endpoint-specific clinical covariates, only select constituent substructures of the heart base were associated with CEs and no substructures were independently associated with survival. Together, these findings are suggestive of possible interplay between the constituent base substructures in their mediation of radiation cardiotoxicity.

Patient-Adaptive Automated Segmentation in Daily kVCT Images for Radiotherapy of Head and Neck and Prostate Cancer

The purpose of this study was to examine the use of transfer learning in deep learning-based auto-segmentation of daily kilovoltage computed tomography (kVCT) images for patient-specific adaptive radiotherapy. Using data from the first cohort of patients treated with the innovative BgRT system, the objective of this study was to evaluate the potential benefits of this approach in facilitating efficient and effective adaptive radiotherapy. For the head and neck (HaN) site and pelvic site, we first trained a deep convolutional segmentation network using a population dataset, consisting of 67 and 56 patient cases, respectively. This population network was then fine-tuned for a specific patient using a transfer learning approach, adapting the network weights. The auto-segmentation network utilized in this study was a 23-layer U-Net with batch normalization, a dropout rate of 0.5, and four skip connections between the encoder and decoder at different levels. We used initial planning CT and 5-26 sets of daily kVCT scans with a total of 8,039 images for patient-specific learning in the 6 HaN cases and 4 pelvic cases, particularly analyzing the relationship between the number of sequential patient-specific training data and the performance of the auto-segmentation. We compared the performance of the patient-specific network with the population network and the clinical rigid registration method, using the Dice similarity coefficient (DSC) as the evaluation metric. Additionally, we investigated the corresponding dosimetric impacts of the different auto-segmentation and registration methods. The patient-specific network showed improved mean DSC scores of 0.88 and 0.90 for three HaN organs at risk (OARs) and eight pelvic targets and OARs, respectively, compared to the population network (0.70 and 0.63) and the registration method (0.72 and 0.72). The DSC of the patient-specific network steadily improved as the number of longitudinal training cases increased, reaching near saturation after 6 training cases. The use of the patient-specific auto-segmentation resulted in a reduction of the mean discrepancy in target and OAR doses between delivery and planning from 5.5% with the clinical rigid registration to 1.1%. The use of patient-specific transfer learning in auto-segmenting kVCT images showed higher accuracy compared to a conventional population network and clinical registration-based method. This approach holds promise for enhancing dose evaluation accuracy in adaptive radiotherapy.

Association between statin therapy dose intensity and radiation cardiotoxicity in non-small cell lung cancer: Results from the NI-HEART study

IntroductionRadiation cardiotoxicity is a dose-limiting toxicity and major survivorship issue for patients with non-small cell lung cancer (NSCLC) completing curative-intent radiotherapy, however patients’ cardiovascular baseline is not routinely optimised prior to treatment. In this study we examined the impact of statin therapy on overall survival and post-radiotherapy cardiac events. MethodsPatients treated between 2015–2020 at a regional center were identified. Clinical notes were interrogated for baseline patient, tumor and cardiac details, and both follow-up cancer control and cardiac events. Three cardiologists verified cardiac events. Radiotherapy planning scans were retrieved for application of validated deep learning-based autosegmentation. Pre-specified Cox regression analyses were generated with varying degrees of adjustment for overall survival. Fine and Gray regression for the risk of cardiac events, accounting for the competing risk of death and cardiac covariables was undertaken. ResultsStatin therapy was prescribed to 59% of the 478 included patients. The majority (88%) of patients not prescribed a statin had at least one indication for statin therapy according to cardiovascular guidelines. In total, 340 patients (71%) died and 79 patients (17%) experienced a cardiac event. High-intensity (HR 0.68, 95%CI 0.50–0.91, p = 0.012) and medium-intensity (HR 0.70, 95%CI 0.51–0.97, p = 0.033) statin therapy were associated with improved overall survival after adjustment for patient, cancer, treatment, response and cardiovascular clinical factors. There were no consistent differences in the rate or grade of cardiac events according to statin intensity. ConclusionsStatin therapy is associated with improved overall survival in patients receiving curative-intent radiotherapy for NSCLC, and there is evidence of a dose–response relationship. This study highlights the importance of a pre-treatment cardiovascular risk assessment in this cohort. Further studies are needed to examine if statin therapy is cardioprotective in patients undergoing treatment for NSCLC with considerable incidental cardiac radiation dose and a low baseline cardiac risk.

Comprehensive clinical evaluation of deep learning-based auto-segmentation for radiotherapy in patients with cervical cancer.

Deep learning-based models have been actively investigated for various aspects of radiotherapy. However, for cervical cancer, only a few studies dealing with the auto-segmentation of organs-at-risk (OARs) and clinical target volumes (CTVs) exist. This study aimed to train a deep learning-based auto-segmentation model for OAR/CTVs for patients with cervical cancer undergoing radiotherapy and to evaluate the model's feasibility and efficacy with not only geometric indices but also comprehensive clinical evaluation. A total of 180 abdominopelvic computed tomography images were included (training set, 165; validation set, 15). Geometric indices such as the Dice similarity coefficient (DSC) and the 95% Hausdorff distance (HD) were analyzed. A Turing test was performed and physicians from other institutions were asked to delineate contours with and without using auto-segmented contours to assess inter-physician heterogeneity and contouring time. The correlation between the manual and auto-segmented contours was acceptable for the anorectum, bladder, spinal cord, cauda equina, right and left femoral heads, bowel bag, uterocervix, liver, and left and right kidneys (DSC greater than 0.80). The stomach and duodenum showed DSCs of 0.67 and 0.73, respectively. CTVs showed DSCs between 0.75 and 0.80. Turing test results were favorable for most OARs and CTVs. No auto-segmented contours had large, obvious errors. The median overall satisfaction score of the participating physicians was 7 out of 10. Auto-segmentation reduced heterogeneity and shortened contouring time by 30 min among radiation oncologists from different institutions. Most participants favored the auto-contouring system. The proposed deep learning-based auto-segmentation model may be an efficient tool for patients with cervical cancer undergoing radiotherapy. Although the current model may not completely replace humans, it can serve as a useful and efficient tool in real-world clinics.

First Report On Physician Assessment and Clinical Acceptability of Custom-Retrained Artificial Intelligence Models for Clinical Target Volume and Organs-at-Risk Auto-Delineation for Postprostatectomy Patients

To assess the clinical acceptability of a commercial deep-learning-based auto-segmentation (DLAS) prostate model that was retrained using institutional data for delineation of the clinical target volume (CTV) and organs-at-risk (OARs) for postprostatectomy patients, accounting for clinical and imaging protocol variations. CTV and OARs of 109 prostate-bed patients were used to evaluate the performance of the vendor-trained model and custom retrained DLAS models using different training quantities. Two new models for OAR structures were retrained (n=30, 60 data sets), while separate models were trained for a new CTV structure (n=30, 60, 90 data sets), with the remaining data sets used for testing (n=49, 19). The dice similarity coefficient (DSC), Hausdorff distance, and mean surface distance were evaluated. Six radiation oncologists performed a qualitative evaluation scoring both preference and clinical utility for blinded structure sets. Physician consensus data sets identified from the qualitative evaluation were used toward a separate CTV model. Both the 30- and 60-case retrained OAR models had median DSC values between 0.91 to 0.97, improving significantly over the vendor-trained model for all OARs except the penile bulb. The brand new 60-case CTV model had a median DSC of 0.70 improving significantly over the 30-case model. DLAS (60-case model) and manual contours were blinded and evaluated by physicians with contours deemed acceptable or precise for 87% and 94% of cases for DLAS and manual delineations, respectively. DLAS-generated CTVs were scored precise or acceptable in 54% of cases, compared with the manual delineation value of 73%. The 30-case physician consensus CTV model did not show a significant difference compared with the randomly selected models. Custom retraining using institutional data leads to performance improvement in the clinical utility and accuracy of DLAS for postprostatectomy patients. A small number of data sets are sufficient for building an institutional site-specific DLAS OAR model, as well as for training new structures. Data indicates the workload for identifying training data sets could be shared among groups for the male pelvic region, making it accessible to clinics of all sizes.

Contouring quality assurance methodology based on multiple geometric features against deep learning auto-segmentation.

Contouring error is one of the top failure modes in radiation treatment. Multiple efforts have been made to develop tools to automatically detect segmentation errors. Deep learning-based auto-segmentation (DLAS) has been used as a baseline for flagging manual segmentation errors, but those efforts are limited to using only one or two contour comparison metrics. The purpose of this research is to develop an improved contouring quality assurance system to identify and flag manual contouring errors. DLAS contours were used as a reference to compare with manually segmented contours. A total of 27 geometric agreement metrics were determined from the comparisons between the two segmentation approaches. Feature selection was performed to optimize the training of a machine learning classification model to identify potential contouring errors. A public dataset with 339 cases was used to train and test the classifier. Four independent classifiers were trained using five-fold cross validation, and the predictions from each classifier were ensembled using soft voting. The trained model was validated on a held-out testing dataset. An additional independent clinical dataset with 60 cases was used to test the generalizability of the model. Model predictions were reviewed by an expert to confirm or reject the findings. The proposed machine learning multiple features (ML-MF) approach outperformed traditional nonmachine-learning-based approaches that are based on only one or two geometric agreement metrics. The machine learning model achieved recall (precision) values of 0.842 (0.899), 0.762 (0.762), 0.727 (0.842), and 0.773 (0.773) for Brainstem, Parotid_L, Parotid_R, and mandible contours, respectively compared to 0.526 (0.909), 0.619 (0.765), 0.682 (0.882), 0.773 (0.568) for an approach based solely on Dice similarity coefficient values. In the external validation dataset, 66.7, 93.3, 94.1, and 58.8% of flagged cases were confirmed to have contouring errors by an expert for Brainstem, Parotid_L, Parotid_R, and mandible contours, respectively. The proposed ML-MF approach, which includes multiple geometric agreement metrics to flag manual contouring errors, demonstrated superior performance in comparison to traditional methods. This method is easy to implement in clinical practice and can help to reduce the significant time and labor costs associated with manual segmentation and review.

Deep Learning-Based Auto-Segmentation of Spinal Cord Internal Structure of Diffusion Tensor Imaging in Cervical Spondylotic Myelopathy.

Cervical spondylotic myelopathy (CSM) is a chronic disorder of the spinal cord. ROI-based features on diffusion tensor imaging (DTI) provide additional information about spinal cord status, which would benefit the diagnosis and prognosis of CSM. However, the manual extraction of the DTI-related features on multiple ROIs is time-consuming and laborious. In total, 1159 slices at cervical levels from 89 CSM patients were analyzed, and corresponding fractional anisotropy (FA) maps were calculated. Eight ROIs were drawn, covering both sides of lateral, dorsal, ventral, and gray matter. The UNet model was trained with the proposed heatmap distance loss for auto-segmentation. Mean Dice coefficients on the test dataset for dorsal, lateral, and ventral column and gray matter were 0.69, 0.67, 0.57, 0.54 on the left side and 0.68, 0.67, 0.59, 0.55 on the right side. The ROI-based mean FA value based on segmentation model strongly correlated with the value based on manual drawing. The percentages of the mean absolute error between the two values of multiple ROIs were 0.07, 0.07, 0.11, and 0.08 on the left side and 0.07, 0.1, 0.1, 0.11, and 0.07 on the right side. The proposed segmentation model has the potential to offer a more detailed spinal cord segmentation and would be beneficial for quantifying a more detailed status of the cervical spinal cord.

62P Development of deep learning-based auto-segmentation algorithms to diagnose and indicate peritoneal carcinomatosis lesions of ovarian cancer

62P Development of deep learning-based auto-segmentation algorithms to diagnose and indicate peritoneal carcinomatosis lesions of ovarian cancer

Implementation of a Commercial Deep Learning-Based Auto Segmentation Software in Radiotherapy: Evaluation of Effectiveness and Impact on Workflow.

Proper delineation of both target volumes and organs at risk is a crucial step in the radiation therapy workflow. This process is normally carried out manually by medical doctors, hence demanding timewise. To improve efficiency, auto-contouring methods have been proposed. We assessed a specific commercial software to investigate its impact on the radiotherapy workflow on four specific disease sites: head and neck, prostate, breast, and rectum. For the present study, we used a commercial deep learning-based auto-segmentation software, namely Limbus Contour (LC), Version 1.5.0 (Limbus AI Inc., Regina, SK, Canada). The software uses deep convolutional neural network models based on a U-net architecture, specific for each structure. Manual and automatic segmentation were compared on disease-specific organs at risk. Contouring time, geometrical performance (volume variation, Dice Similarity Coefficient-DSC, and center of mass shift), and dosimetric impact (DVH differences) were evaluated. With respect to time savings, the maximum advantage was seen in the setting of head and neck cancer with a 65%-time reduction. The average DSC was 0.72. The best agreement was found for lungs. Good results were highlighted for bladder, heart, and femoral heads. The most relevant dosimetric difference was in the rectal cancer case, where the mean volume covered by the 45 Gy isodose was 10.4 cm3 for manual contouring and 289.4 cm3 for automatic segmentation. Automatic contouring was able to significantly reduce the time required in the procedure, simplifying the workflow, and reducing interobserver variability. Its implementation was able to improve the radiation therapy workflow in our department.

Evaluation of an Externally Trained Deep Learning-Based Auto-Segmentation Software in the Process of Artificial Intelligence-Assisted Radiation Treatment Planning for Thoracic Cancers

<h3>Purpose/Objective(s)</h3> Contouring Organs-at-risk (OAR) is a laborious process that often delays radiation treatment plan design. A few FDA approved auto-segmentation software (AS) have become available. Our goal is to validate such a commercial AS in thoracic cancer OAR contouring. <h3>Materials/Methods</h3> We installed an externally trained AS into our AI computer. Validation is judged by our current gold standard contouring (GSC) by two experienced planners and one radiation oncologist (RO). We used 30 lung or esophageal cancer planning datasets to generate GSC and AI contours (AIC). Objective analysis included Dice Similarity Coefficient (DSC) and 95% Hausdorff distance (95% HD). Subjective analysis was done by two ROs to score 1 to 3 on all OARs by GSC and AIC that were randomly blended and anonymized with consistent nomenclature (1: no modification required; 2: minor modification required but adequate for clinical use; 3: major modification required and not suitable for clinical use). <h3>Results</h3> Most retrospective peer-reviewed OAR contours neglected some less important structures on CT slices typically far away from the target of the 30 patients, median age 75 years (54-90), including 22 males and 8 females, with 28 average pixel density data-sets from 4D-CT for lung cancer and 2 fast helical scans for esophageal cancer. We had to re-contour most of the OARs to generate GSC. The median GSC and AIC contouring times were 60 vs 2.5 minutes for up to 12 OARs, some of which were only partially available in the datasets (e.g., stomach and liver). Due to the inconsistency of contouring organs far away from the planning target volume, we only chose six main OARs for initial validation and analysis. Comparing AICs to GSCs, the mean DSC and 95% HD were: esophagus 0.61 and 16 mm, heart 0.85 and 13.1 mm, left lung 0.97 and 5.9 mm, right lung 0.96 and 5.7 mm, spinal cord 0.82 and 10.7 mm, trachea and proximal bronchial tree (TPB) 0.67 and 19.1 mm, respectively. The two ROs agreed with 100% of four OARs on GSC, i.e., both RO scoring 1 or 2 meaning adequate for planning purpose, with the exception of esophagus having 96.7% vs 100% and right lung having 100% vs 96.7% agreement, respectively. They had less agreement on AIC, with esophagus 90% vs 60%, heart 83.3% vs 86.7%, left lung 100% vs 96.7%, right lung 100% vs 96.7%, spinal cord 100% vs 100%, TPB 96.7% vs 86.7% agreement, respectively. The inter-observer variabilities are significantly larger when ROs evaluated esophagus and heart AIC (p=0.046 and 0.05, respectively, Student's t-test). <h3>Conclusion</h3> The accuracy of an externally trained deep learning-based AS might not be acceptable without in-house training from local protocols. Retrospective peer-reviewed OAR contours might not be good enough in the training and evaluation of AS. Our future work involves training AS using our GSCs and re-evaluating its performance.

A Feasibility Study of Deep Learning-Based Auto-Segmentation Directly Used in VMAT Planning Design and Optimization for Cervical Cancer.

PurposeTo investigate the dosimetric impact on target volumes and organs at risk (OARs) when unmodified auto-segmented OAR contours are directly used in the design of treatment plans. Materials and MethodsA total of 127 patients with cervical cancer were collected for retrospective analysis, including 105 patients in the training set and 22 patients in the testing set. The 3D U-net architecture was used for model training and auto-segmentation of nine types of organs at risk. The auto-segmented and manually segmented organ contours were used for treatment plan optimization to obtain the AS-VMAT (automatic segmentations VMAT) plan and the MS-VMAT (manual segmentations VMAT) plan, respectively. Geometric accuracy between the manual and predicted contours were evaluated using the Dice similarity coefficient (DSC), mean distance-to-agreement (MDA), and Hausdorff distance (HD). The dose volume histogram (DVH) and the gamma passing rate were used to identify the dose differences between the AS-VMAT plan and the MS-VMAT plan.Results Average DSC, MDA and HD95 across all OARs were 0.82–0.96, 0.45–3.21 mm, and 2.30–17.31 mm on the testing set, respectively. The D99% in the rectum and the Dmean in the spinal cord were 6.04 Gy (P = 0.037) and 0.54 Gy (P = 0.026) higher, respectively, in the AS-VMAT plans than in the MS-VMAT plans. The V20, V30, and V40 in the rectum increased by 1.35% (P = 0.027), 1.73% (P = 0.021), and 1.96% (P = 0.008), respectively, whereas the V10 in the spinal cord increased by 1.93% (P = 0.011). The differences in other dosimetry parameters were not statistically significant. The gamma passing rates in the clinical target volume (CTV) were 92.72% and 98.77%, respectively, using the 2%/2 mm and 3%/3 mm criteria, which satisfied the clinical requirements.ConclusionsThe dose distributions of target volumes were unaffected when auto-segmented organ contours were used in the design of treatment plans, whereas the impact of automated segmentation on the doses to OARs was complicated. We suggest that the auto-segmented contours of tissues in close proximity to the target volume need to be carefully checked and corrected when necessary.

SP-0699 Uncertainties and out-of-distribution estimation in deep learning-based autosegmentation

SP-0699 Uncertainties and out-of-distribution estimation in deep learning-based autosegmentation

OC-0771 Uncertainty map for error prediction in deep learning-based head and neck tumor auto-segmentation

OC-0771 Uncertainty map for error prediction in deep learning-based head and neck tumor auto-segmentation