Auto Contouring Research Articles

Geometric and dosimetric evaluation for breast and regional nodal auto-segmentation structures.

The accuracy of artificial intelligence (AI) generated contours for intact-breast and post-mastectomy radiotherapy plans was evaluated. Geometric and dosimetric comparisons were performed between auto-contours (ACs) and manual-contours (MCs) produced by physicians for target structures. Breast and regional nodal structures were manually delineated on 66 breast cancer patients. ACs were retrospectively generated. The characteristics of the breast/post-mastectomy chestwall (CW) and regional nodal structures (axillary [AxN], supraclavicular [SC], internal mammary [IM]) were geometrically evaluated by Dice similarity coefficient (DSC), mean surface distance, and Hausdorff Distance. The structures were also evaluated dosimetrically by superimposing the MC clinically delivered plans onto the ACs to assess the impact of utilizing ACs with target dose (Vx%) evaluation. Positive geometric correlations between volume and DSC for intact-breast, AxN, and CW were observed. Little or anti correlations between volume and DSC for IM and SC were shown. For intact-breast plans, insignificant dosimetric differences between ACs and MCs were observed for AxNV95% (p=0.17) and SCV95% (p=0.16), while IMNV90% ACs and MCs were significantly different. The average V95% for intact-breast MCs (98.4%) and ACs (97.1%) were comparable but statistically different (p=0.02). For post-mastectomy plans, AxNV95% (p=0.35) and SCV95% (p=0.08) were consistent between ACs and MCs, while IMNV90% was significantly different. Additionally, 94.1% of AC-breasts met ΔV95% variation<5% when DSC>0.7. However, only 62.5% AC-CWs achieved the same metrics, despite AC-CWV95% (p=0.43) being statistically insignificant. The AC intact-breast structure was dosimetrically similar to MCs. The AC AxN and SC may require manual adjustments. Careful review should be performed for AC post-mastectomy CW and IMN before treatment planning. The findings of this study may guide the clinical decision-making process for the utilization of AI-driven ACs for intact-breast and post-mastectomy plans. Before clinical implementation of this auto-segmentation software, an in-depth assessment of agreement with each local facilities MCs is needed.

1652: Atlas-based segmentation to auto contour targets and organ at risks for craniospinal irradiation

1652: Atlas-based segmentation to auto contour targets and organ at risks for craniospinal irradiation

Artificial Intelligence Auto Contouring and its Impact on Resident Education

Artificial Intelligence Auto Contouring and its Impact on Resident Education

Clinical validation of commercial deep-learning based auto-segmentation models for organs at risk in the head and neck region: a single institution study.

To evaluate organ at risk (OAR) auto-segmentation in the head and neck region of computed tomography images using two different commercially available deep-learning-based auto-segmentation (DLAS) tools in a single institutional clinical applications. Twenty-two OARs were manually contoured by clinicians according to published guidelines on planning computed tomography (pCT) images for 40 clinical head and neck cancer (HNC) cases. Automatic contours were generated for each patient using two deep-learning-based auto-segmentation models-Manteia AccuContour and MIM ProtégéAI. The accuracy and integrity of autocontours (ACs) were then compared to expert contours (ECs) using the Sørensen-Dice similarity coefficient (DSC) and Mean Distance (MD) metrics. ACs were generated for 22 OARs using AccuContour and 17 OARs using ProtégéAI with average contour generation time of 1 min/patient and 5 min/patient respectively. EC and AC agreement was highest for the mandible (DSC 0.90 ± 0.16) and (DSC 0.91 ± 0.03), and lowest for the chiasm (DSC 0.28 ± 0.14) and (DSC 0.30 ± 0.14) for AccuContour and ProtégéAI respectively. Using AccuContour, the average MD was<1mm for 10 of the 22 OARs contoured, 1-2mm for 6 OARs, and 2-3mm for 6 OARs. For ProtégéAI, the average mean distance was<1mm for 8 out of 17 OARs, 1-2mm for 6 OARs, and 2-3mm for 3 OARs. Both DLAS programs were proven to be valuable tools to significantly reduce the time required to generate large amounts of OAR contours in the head and neck region, even though manual editing of ACs is likely needed prior to implementation into treatment planning. The DSCs and MDs achieved were similar to those reported in other studies that evaluated various other DLAS solutions. Still, small volume structures with nonideal contrast in CT images, such as nerves, are very challenging and will require additional solutions to achieve sufficient results.

A pair of deep learning auto-contouring models for prostate cancer patients injected with a radio-transparent versus radiopaque hydrogel spacer.

Absorbable hydrogel spacer injected between prostate and rectum is gaining popularity for rectal sparing. The spacer alters patient anatomy and thus requires new auto-contouring models. To report the development and comprehensive evaluation of two deep-learning models for patients injected with a radio-transparent (model I) versus radiopaque (model II) spacer. Model I was trained and cross-validated by 135 cases with transparent spacer and tested on 24 cases. Using refined training methods, model II was trained and cross-validated by the same dataset, but with the Hounsfield Unit distribution in the spacer overridden by that obtained from ten cases with opaque spacer. Model II was tested on 64 cases. The models auto-contour eight regions of interest (ROIs): spacer, prostate, proximal seminal vesicles (SVs), left and right femurs, bladder, rectum, and penile bulb. Qualitatively, each auto contour (AC), as well as the composite set, was assessed against manual contour (MC), by a radiation oncologist using a 1 (accepted directly or after minor editing), 2 (accepted after moderate editing), 3 (accepted after major editing), and 4 (rejected) scoring scale. The efficiency gain was characterized by the mean score as nearly complete [1-1.75], substantial (1.75-2.5], meaningful (2.5-3.25], and no (3.25-4.00]. Quantitatively, the geometric similarity between AC and MC was evaluated by dice similarity coefficient (DSC) and mean distance to agreement (MDA), using tolerance recommended by AAPM TG-132 Report. The results by the two models were compared to examine the outcome of the refined training methods. The large number of testing cases for model II allowed further investigation of inter-observer variability in clinical dataset. The correlation between score and DSC/MDA was studied on the ROIs with 10 or more counts of each acceptable score (1, 2, 3). For model I/model II: the mean score was 3.63/1.30 for transparent/opaque spacer, 2.71/2.16 for prostate, 3.25/2.44 for proximal SVs, 1.13/1.02 for both femurs, 2.25/1.25 for bladder, 3.00/2.06 for rectum, 3.38/2.42 for penile bulb, and 2.79/2.20 for the composite set; the mean DSC was 0.52/0.84 for spacer, 0.84/0.85 for prostate, 0.60/0.62 for proximal SVs, 0.94/0.96 for left femur, 0.95/0.96 for right femur, 0.91/0.95 for bladder, 0.81/0.84 for rectum, and 0.65/0.65 for penile bulb; and the mean MDA was 2.9/0.9mm for spacer, 1.9/1.7mm for prostate, 2.4/2.3mm for proximal SVs, 0.8/0.5mm for left femur, 0.7/0.5mm for right femur, 1.5/0.9mm for bladder, 2.3/1.9mm for rectum, and 2.2/2.2mm for penile bulb. Model II showed significantly improved scores for all ROIs, and metrics for spacer, femurs, bladder, and rectum. Significant inter-observer variability was only found for prostate. Highly linear correlation between the score and DSC was found for the two qualified ROIs (prostate and rectum). The overall efficiency gain was meaningful for model I and substantial for model II. The ROIs meeting the clinical deployment criteria (mean score below 3.25, DSC above 0.8, and MDA below 2.5mm) included prostate, both femurs, bladder and rectum for both models, and spacer for model II.

Target and organ-at-risk (OAR) delineation for VMAT craniospinal irradiation using Raystation's Automatic Atlas Based Segmentation

<h3>Objectives</h3> VMAT Craniospinal irradiation (CSI-VMAT) requires extensive contouring of multiple targets and OARs. Manual contouring of these structures is time consuming (3-5 hours). Using an automatic contouring module within Raystation treatment planning system there is potential to automate outlining and streamline planning. The aim of this project was to produce and evaluate an automatic contouring atlas to facilitate CSI-VMAT planning. <h3>Methods</h3> Two atlases were created using Raystation's atlas-based initialisation (ABI) module; Craniospinal_Ph1_Localise_Adult_Atlas (Atlas 1) and Craniospinal_Ph1_Localise_PaedYA_Atlas (Atlas 2). Automatic contouring was attempted for the following targets and OARs: whole brain, optic nerves, cervical spine, thoracic spine, and lumbar spine, adjacent vertebrae, optic chiasm, globes, lenses, parotids, cochlea's, kidneys, lungs, heart, liver, thyroid gland and oesophagus. 28 anonymised image sets were incorporated into Atlas 1. The paediatric and young adults (YA) atlas (Atlas 2) included an additional ‘Vertebral Body Adjacent' structure for 28 anonymised image sets. The quality of the auto contours was compared to SIOPE consensus atlas for target volume delineation for craniospinal irradiation [1] with institutional peer review and OAR contouring guidelines. <h3>Results</h3> The final Atlases included automation of all CTVs and the following OARs: Chiasm, Globes, Lenses, Cochlea's, Kidneys, Lungs, Heart, Liver and Thyroid. The quality of contouring produced was deemed clinically acceptable except for oesophagus and parotid contours, which were omitted due to inferior contouring. The atlases segmented all structures within 15 minutes. Following minor edits and manual delineation by experienced dosimetrists of omitted contours, delineation time is reduced to 1.5 – 2.5 hours. Further clinician review and approval is undertaken prior to planning. <h3>Conclusion</h3> Atlas's 1 and 2 are now implemented clinically for paediatric and adult CSI-VMAT, providing valuable time saving tools for the planning of these patients. There is the potential for this to be applied to other non-craniospinal paediatric cases, or different techniques including Proton Therapy.

Prospective Analysis of a Deep Learning Auto-Contouring Model for Definitive Radiation of Localized Prostate Cancer

<h3>Purpose/Objective(s)</h3> To analyze the prospective use of a deep learning auto contouring model for patients with localized prostate cancer with or without insertion of a radiopaque hydrogel spacer. <h3>Materials/Methods</h3> The model was initially trained with patients from an academic center. It was implemented for prospective clinical use at a satellite center by three experienced radiation oncologists. The model auto contours prostate, proximal seminal vesicles (SVs), bladder, rectum, femurs, penile bulb and spacer (when existing). The auto contours (ACs) were reviewed and edited by treating physician. The ACs were compared to the approved planning contours (PCs) for the initial 23 patients to evaluate the extent of physician editing required. Each AC was compared to the final PC by one physician using a qualitative 4-point system: 1 (minor discrepancy with significant efficiency gain), 2 (moderate discrepancy with substantial gain), 3 (significant discrepancy with meaningful gain), and 4 (rejected for gross error or no efficiency gain). A composite score was calculated by averaging the score of each structure. A Turing test was performed to see if the physician can differentiate AC vs. PC. Quantitatively, the geometric differences between each AC and PC were analyzed using Dice similarity coefficient (DSC) and mean distance to agreement (MDA). <h3>Results</h3> As shown in Table 1, the mean score was ≤1.4 for all structures except for SVs, and 1.26 for the composite, indicating overall significant efficiency gain. In 97.8% of cases, the ACs led to an efficiency gain. While the passing rate of the Turing test was not high, the failure was frequently due to very minor differences. The DSC was close to 1 for femurs, bladder and spacer, and >0.9 for prostate and rectum. The MDA was <2 mm for all structures except SVs. All quantitative measures except for the SVs were within the tolerance recommended by AAPM TG 132 (DSC ≥ 0.8, MDA ≤ 3 mm). On average, treating physicians estimated 17 minutes were saved for each patient. <h3>Conclusion</h3> Overall, the model can accurately auto contour targets, organs, and spacer with significant efficiency gain. A majority of ACs qualitatively evaluated by a physician had only mild discrepancies. Quantitative analysis demonstrated ACs for most organs had only minor geometric differences, unlikely to result in any significant dose-volume impact.

Dosimetric Analysis of Radiation Treatment Plans Based on a Deep Learning Auto Contouring Model for Patients with Localized Prostate Cancer

<h3>Purpose/Objective(s)</h3> To evaluate the feasibility and safety of creating radiation treatment plans based on a convolutional neural network auto-contouring (AC) model for patients receiving radiation for localized prostate cancer with a radio-opaque rectal spacer hydrogel. <h3>Materials/Methods</h3> The deep learning model produces ACs for target volumes (prostate, proximal seminal vesicle), OARs (rectum, bladder, penile bulb, femoral heads), and rectal hydrogel spacers. ACs were compared to manual contours (MC) by MDs for clinical use. Individual volumes and overall performance (composite score) were previously evaluated and reported, qualitatively using a 1 (minor discrepancy), 2 (moderate discrepancy), 3 (significant discrepancy), and 4 (rejected, gross error) scale and quantitatively using Dice Similarity Coefficient (DSC) and mean distance to agreement. VMAT treatment plans were created on unedited AC volumes from five composite 1 (C1) cases and five composite 2 (C2) cases and dose volume metrics were computed for the final MCs. Dosimetry comparison was performed to assess the safety of delivering a treatment plan created from unedited ACs. Cases were planned to 50.4 Gy in 28 fractions to the SVs, with a SIB of 70 Gy the prostate. PTV expansions were CTV+5 mm (CTV+4 mm posteriorly). <h3>Results</h3> Of 62 initial cases evaluated, 11 were scored as a C1, 28 were scored as a C2, and 23 received a C3. 10 cases were randomly selected for treatment planning. For prostate (CTV P) in selected C1 and C2 cases, mean score was 1 and 1.8 and mean DSC was 0.919 and 0.876, respectively. Clinical goals for planning and dosimetric data from treatment plans are shown in Table 1. The mean V70 Gy for prostate CTV/PTV was 99.83%/90.05% for C1 cases and 99.46%/89.15% for C2 cases. All clinical goals for OARs were met in both cohorts. <h3>Conclusion</h3> 63% of all cases evaluated received a C1 or C2 score. Our results indicate the treatment plans generated from unedited ACs on these cases respected all clinical goals for OARs. Although C1 plans produced better PTV coverage than C2, almost all plans had sub-optimal PTV coverage, indicating the need for MDs to edit target volumes before treatment planning.

Clinical Feasibility of Commercial Artificial Intelligence-Based Auto Contouring of Target Volumes and Organs-at-Risk in Breast Cancer Patients

<h3>Purpose/Objective(s)</h3> Manual clinical target volume (CTV) and organ at-risk (OAR) segmentation for planning remains a significant component of physician workload. In addition, inter-physician segmentation variability can introduce uncertainty in planning process. While current atlas-based auto-segmentation has proven to be useful for contouring, it typically requires significant upfront time investment and is physician dependent. Recently, artificial-intelligence-based contouring have been made commercially available. These systems are physician-independent, and rely on machine learning rather than a user-selected atlas of patient datasets. In this study, we evaluated the potential benefits of a commercially available artificial intelligence-based auto contouring (AI-AC) system by comparing AI-AC generated and manually delineated contours for breast patients. <h3>Materials/Methods</h3> CTVs (ipsilateral breast) and OARs (contralateral breast, heart, lungs, esophagus, and spinal cord) were manually contouring (MC) on planning computed tomography scans of 10 breast cancer patients (5 patients received breast-conserving surgery). Commercial AI auto-contouring software (AI-Rad Companion Organs RT) was then used to generate the same target and risk volumes. Volumes were compared using quantitative metrics, including the Dice Similarity Coefficient (DSC), and mean distance agreement (MDA). <h3>Results</h3> Correlation between the AI-AC and MC generated volumes was strong for the evaluated OARs, with a mean DSC higher than 0.90 for all OARs except the esophagus (mean DSC 0.83 ± 0.09)). In addition, Comparison of CTV volumes also showed excellent results, with a mean DSC higher than 0.93. For the OARs, mean DSC values ranged from 0.98 ± 0.01 to 0.83 ± 0.09, while the mean MDA ranged from 0.68 ± 0.3 to 0.84 +0.54 mm to mm. [Similar results were achieved for the CTV, with the mean DSC ranging from 0.93 ± 0.03 to 0.94 ± 0.06 to 0.93, and mean MDA from 1.71 ± 0.91 mm to 1.76 ± 0.68 mm. AI-AC generated volumes were produced in approximately 2 min, while MC of the same volumes takes the physician an estimated 30 min. <h3>Conclusion</h3> The feasibility of commercial AI-AC in breast RT planning was demonstrated. Immediately on activation, the AI-AC system was able to produce volumes showing significant agreement with physician-generated volumes. While AI-AC-generated volumes may require minor edits by the physician, clinical efficiency is substantially improved. Furthermore, initial volumes created by AI-AC are physician-independent, and may help to improve contour consistency in clinic compared to strictly manual approaches.

Clinical Validation of a Deep-Learning Segmentation Software in Head and Neck: An Early Analysis in a Developing Radiation Oncology Center.

Background: Organs at risk (OARs) delineation is a crucial step of radiotherapy (RT) treatment planning workflow. Time-consuming and inter-observer variability are main issues in manual OAR delineation, mainly in the head and neck (H & N) district. Deep-learning based auto-segmentation is a promising strategy to improve OARs contouring in radiotherapy departments. A comparison of deep-learning-generated auto-contours (AC) with manual contours (MC) was performed by three expert radiation oncologists from a single center. Methods: Planning computed tomography (CT) scans of patients undergoing RT treatments for H&N cancers were considered. CT scans were processed by Limbus Contour auto-segmentation software, a commercial deep-learning auto-segmentation based software to generate AC. H&N protocol was used to perform AC, with the structure set consisting of bilateral brachial plexus, brain, brainstem, bilateral cochlea, pharyngeal constrictors, eye globes, bilateral lens, mandible, optic chiasm, bilateral optic nerves, oral cavity, bilateral parotids, spinal cord, bilateral submandibular glands, lips and thyroid. Manual revision of OARs was performed according to international consensus guidelines. The AC and MC were compared using the Dice similarity coefficient (DSC) and 95% Hausdorff distance transform (DT). Results: A total of 274 contours obtained by processing CT scans were included in the analysis. The highest values of DSC were obtained for the brain (DSC 1.00), left and right eye globes and the mandible (DSC 0.98). The structures with greater MC editing were optic chiasm, optic nerves and cochleae. Conclusions: In this preliminary analysis, deep-learning auto-segmentation seems to provide acceptable H&N OAR delineations. For less accurate organs, AC could be considered a starting point for review and manual adjustment. Our results suggest that AC could become a useful time-saving tool to optimize workload and resources in RT departments.

Assessment of CT to CBCT contour mapping for radiomic feature analysis in prostate cancer

This study provides a quantitative assessment of the accuracy of a commercially available deformable image registration (DIR) algorithm to automatically generate prostate contours and additionally investigates the robustness of radiomic features to differing contours. Twenty-eight prostate cancer patients enrolled on an institutional review board (IRB) approved protocol were selected. Planning CTs (pCTs) were deformably registered to daily cone-beam CTs (CBCTs) to generate prostate contours (auto contours). The prostate contours were also manually drawn by a physician. Quantitative assessment of deformed versus manually drawn prostate contours on daily CBCT images was performed using Dice similarity coefficient (DSC), mean distance-to-agreement (MDA), difference in center-of-mass position (ΔCM) and difference in volume (ΔVol). Radiomic features from 6 classes were extracted from each contour. Lin’s concordance correlation coefficient (CCC) and mean absolute percent difference in radiomic feature-derived data (mean |%Δ|RF) between auto and manual contours were calculated. The mean (± SD) DSC, MDA, ΔCM and ΔVol between the auto and manual prostate contours were 0.90 ± 0.04, 1.81 ± 0.47 mm, 2.17 ± 1.26 mm and 5.1 ± 4.1% respectively. Of the 1,010 fractions under consideration, 94.8% of DIRs were within TG-132 recommended tolerance. 30 radiomic features had a CCC > 0.90 and 21 had a mean |%∆|RF < 5%. Auto-propagation of prostate contours resulted in nearly 95% of DIRs within tolerance recommendations of TG-132, leading to the majority of features being regarded as acceptably robust. The use of auto contours for radiomic feature analysis is promising but must be done with caution.

Development and Evaluation of the First Pediatric Deep-Learning Auto-Contouring Models for Cranio-Spinal Irradiation (CSI)

Cranio-spinal irradiation (CSI) using advanced modality (IMRT, VMAT, protons) requires contouring of the thecal sac, a tedious and time-consuming task. Additionally, sparing of the vertebrae to improve growth and bone marrow reserve necessitates vertebrae contours. The COG and other investigators have great interest in studying growth effects from vertebrae-sparing technique; this effort requires reporting of dose to the thecal sac and the vertebrae. The presence of bone-soft tissue interface and time required for manual contours make this an ideal use of deep-learning auto-contouring.Two convolutional neural network (CNN) models were trained and validated, one with 65 patients to auto contour thecal sac, kidneys and lungs, and the other with 55 to draw vertebrae. The training set consisted of mostly pediatric and young adult patients. The models were tested on ten new patients at the age of 5, 6, 8, 9, 11, 13, 16, 20, 31 and 47. Quantitatively, auto contours were evaluated against clinical manual contours using dice similarity coefficient (DSC) and mean distance to agreement (MDA). A qualitative scoring system was used by a dosimetrist experienced in CSI contouring: 0 (accepted upright), 1 (accepted, minor editing in 1 min), 2 (accepted, moderate editing in 3 min), 3 (accepted, significant editing with efficiency gain) and 4 (rejected, gross error or no efficiency gain). The time for editing each auto contour for each patient was recorded and compared to the time required for manual contouring measured using time/slice and the number of slices for thecal sac, vertebrae and each kidney, and as whole organ for each lung.As shown in Table 1, on average, the DSC was above 0.90 and MDA below 1.2 mm, both well within the tolerance recommended by AAPM TG 132. The mean score was below 1.0 for all volumes. Editing thecal sac and vertebrae auto contours only required 1 min or less for all patients. The models reduced contouring time by 84 and 40 min for thecal sac and vertebrae, respectively.The deep-learning models provided high-quality auto contours, directly acceptable for lungs and requiring only minor editing in less than one min for thecal sac, vertebrae and kidneys. Given the labor-intensive nature of manually contouring thecal sac and vertebrae, the high accuracy of the auto contours resulted in substantial reduction of contouring time and thus faster turnaround for CSI treatment. The models can provide a uniform starting point for CSI contouring in our field, which is particularly beneficial to clinics with less experiences.

Qualitative and Quantitative Analysis of a Deep Learning Auto Contouring Model for Radiotherapy in Localized Prostate Cancer

Purpose/Objective(s)To evaluate a convolutional neural network auto-contouring (AC) model created for treatment planning in patients receiving radiotherapy for localized prostate cancer after insertion of a radiopaque rectal spacer hydrogel.Materials/MethodsThe deep learning model, trained by 125 patients, auto contours target volumes (prostate and proximal seminal vesicles), OARs (bladder, rectum, femoral heads and penile bulb) and radiopaque rectal spacers. ACs were evaluated against MD manual contours (MCs) submitted for treatment planning. ACs were not available for MD review while creating MCs. Individual volumes as well as composite volumes (overall performance) were qualitatively evaluated by a radiation oncologist using a 1 (minor discrepancy, little to no dose-volume impact), 2 (moderate discrepancy or editable with substantial efficiency gain), 3 (significant discrepancy or editable with meaningful efficiency gain), and 4 (rejected due to gross error or editable without efficiency gain) scoring scale. Quantitative evaluation of geometric differences was performed with mean distance to agreement (MDA) and Dice Similarity Coefficient (DSC) for each volume.ResultsA total of 68 cases were evaluated by quantitative and qualitative metrics (Table 1). MCs were performed by 4 experienced MDs. Average composite score of 2.22 (SD 0.73) indicates substantial to meaningful efficiency gain in contouring process. Composite scores for 100% of cases evaluated indicate a meaningful efficiency gain in the contouring process, with 60% of composite scores indicating only minor to moderate discrepancy between contours. Prostate and SV AC average scores were above 2. Mean prostate MDA and DSC were 1.71 and 0.85, respectively, both within tolerance range recommended by AAPM TG 132.ConclusionThe model can accurately auto contour target volumes, OARs and spacer with meaningful to substantial efficiency gain, with a majority of target volumes qualitatively evaluated by a physician as having only mild to moderate discrepancies. Quantitative analysis shows contours for most OARs have only minor geometric differences unlikely to have significant dose-volume impacts. Further analysis with comparison of treatment plans generated from ACs and MCs is warranted. To evaluate a convolutional neural network auto-contouring (AC) model created for treatment planning in patients receiving radiotherapy for localized prostate cancer after insertion of a radiopaque rectal spacer hydrogel. The deep learning model, trained by 125 patients, auto contours target volumes (prostate and proximal seminal vesicles), OARs (bladder, rectum, femoral heads and penile bulb) and radiopaque rectal spacers. ACs were evaluated against MD manual contours (MCs) submitted for treatment planning. ACs were not available for MD review while creating MCs. Individual volumes as well as composite volumes (overall performance) were qualitatively evaluated by a radiation oncologist using a 1 (minor discrepancy, little to no dose-volume impact), 2 (moderate discrepancy or editable with substantial efficiency gain), 3 (significant discrepancy or editable with meaningful efficiency gain), and 4 (rejected due to gross error or editable without efficiency gain) scoring scale. Quantitative evaluation of geometric differences was performed with mean distance to agreement (MDA) and Dice Similarity Coefficient (DSC) for each volume. A total of 68 cases were evaluated by quantitative and qualitative metrics (Table 1). MCs were performed by 4 experienced MDs. Average composite score of 2.22 (SD 0.73) indicates substantial to meaningful efficiency gain in contouring process. Composite scores for 100% of cases evaluated indicate a meaningful efficiency gain in the contouring process, with 60% of composite scores indicating only minor to moderate discrepancy between contours. Prostate and SV AC average scores were above 2. Mean prostate MDA and DSC were 1.71 and 0.85, respectively, both within tolerance range recommended by AAPM TG 132. The model can accurately auto contour target volumes, OARs and spacer with meaningful to substantial efficiency gain, with a majority of target volumes qualitatively evaluated by a physician as having only mild to moderate discrepancies. Quantitative analysis shows contours for most OARs have only minor geometric differences unlikely to have significant dose-volume impacts. Further analysis with comparison of treatment plans generated from ACs and MCs is warranted.

Comparing Deep Learning Based Prostate Delineation With Manual Contouring From Multiple Experts

Accurate contouring of the target is very important in radiation therapy treatment planning. Unlike most other cancers where the radiation target occupies a small area of an organ, prostate cancer treatment involves the irradiation of the entire prostate. This study evaluates the quality of prostate contours generated from a deep learning segmentation (DLSEG) model.A total of 23 prostate cancer patients who did not underwent prostatectomy was randomly selected. The original contours used clinically was carefully quality checked and revised when necessary by one expert radiation oncologist (RO) and regarded as the ground truth (GT). Two additional ROs were invited to perform manual contouring of the prostate from scratch. Auto contours (labelled as AI) generated by a research version of DLSEG was then compared with those from manual delineations. A paired t-test was employed for statistical comparison. A comparative geometric analysis of the manual and automatic delineations was performed to quantify the multi-observer delineation variability, including dice similarity coefficient (DSC), 95% Hausdorff distance (HD95), and mean surface distance (MSD).Using GT contour as reference, the contours from three additional RO achieved an average of 0.81 ± 0.04 for DSC, 0.60 ± 0.16cm and 0.21 ± 0.05cm for HD95 and MSD, respectively. In contrast, AI contours achieved an average of 0.83 ± 0.05 for DSC, 0.60 ± 0.18cm and 0.21 ± 0.07 cm for HD95 and MSD, respectively. No statistical difference was found for the delineation accuracy between the three observers (P > 0.05). AI contours were considered to have similar performance to the three human experts with all geometric metrics (P > 0.05).The DLSEG model for prostate contouring tested demonstrates similar performance in terms of accuracy compared with expert ROs.

An anomaly detector as a clinical decision support system for parotid gland delineations

Purpose. Auto-contouring (AC) is rapidly becoming standard practice for OAR contouring. However, in clinical practice, clinicians still need to manually check and correct contours. Anomaly detection systems (ADS) can aid the clinical decision process by suggesting which structures require corrections or not, greatly enhancing the value of AC. The purpose of this work is to develop and evaluate a decision support system for detecting anomalies in the case of parotid gland delineations. Methods. Head and neck parotid gland delineations (1037 right, 1038 left), were retrieved from the Netherlands Cancer Institute (NKI) database. Morphological and image-based features were extracted from each patient’s CT and structure set. An isolation forest model was initially trained on 70% of the data, of which 10% had synthetically generated anomalies and validated on the remaining 30% of clinical data. The ADS was tested on an independent set of 250 patients (Normal: 174, Anomalies: 76) and on a clinical autocontouring software. Results. Applied to the validation set, the ADS system resulted in area under the curve (AUC) values of 0.93 and 0.94 for the parotid left and right respectively. Image features appeared more important than morphological, but using all features resulted marginally in the best model. Applied to the test set the ADS system reached an accuracy level of 0.83 and 0.81 for the parotid left and right respectively. The ADS was particularly sensitive to uniform expansions/contractions, misplacements, extra/missing slices and anisotropic over-contouring. Conclusion. Anomaly detection can serve as a powerful contour quality assurance tool, especially for cases of organ misplacement and over-contouring.

An evaluation of MR based deep learning auto-contouring for planning head and neck radiotherapy

IntroductionAuto contouring models help consistently define volumes and reduce clinical workload. This study aimed to evaluate the cross acquisition of a Magnetic Resonance (MR) deep learning auto contouring model for organ at risk (OAR) delineation in head and neck radiotherapy. MethodsTwo auto contouring models were evaluated using deep learning contouring expert (DLCExpert) for OAR delineation: a CT model (modelCT) and an MR model (modelMRI). Models were trained to generate auto contours for the bilateral parotid glands and submandibular glands. Auto-contours for modelMRI were trained on diagnostic images and tested on 10 diagnostic, 10 MR radiotherapy planning (RTP), eight MR-Linac (MRL) scans and, by modelCT, on 10 CT planning scans. Goodness of fit scores, dice similarity coefficient (DSC) and distance to agreement (DTA) were calculated for comparison. ResultsModelMRI contours improved the mean DSC and DTA compared with manual contours for the bilateral parotid glands and submandibular glands on the diagnostic and RTP MRs compared with the MRL sequence. There were statistically significant differences seen for modelMRI compared to modelCT for the left parotid (mean DTA 2.3 v 2.8 mm), right parotid (mean DTA 1.9 v 2.7 mm), left submandibular gland (mean DTA 2.2 v 2.4 mm) and right submandibular gland (mean DTA 1.6 v 3.2 mm). ConclusionA deep learning MR auto-contouring model shows promise for OAR auto-contouring with statistically improved performance vs a CT based model. Performance is affected by the method of MR acquisition and further work is needed to improve its use with MRL images.

Optimal Standardized Uptake Value Threshold for Auto contouring of Gross Tumor Volume using Positron Emission Tomography/Computed Tomography in Patients with Operable Nonsmall-Cell Lung Cancer: Comparison with Pathological Tumor Size.

Purpose:Incorporating 18F-fluorodeoxyglucose positron emission tomography-computed tomography (18F-FDG-PET/CT) for gross tumor volume (GTV) delineation is challenging due to varying tumor edge based on the set threshold of the standardized uptake value (SUV). This study aims to determine an optimal SUV threshold that correlates best with the pathological tumor size.Materials and Methods:From January 2013 to July 2014, 25 consecutive patients of operable nonsmall-cell lung cancer (NSCLC) who underwent staging18F-FDG-PET/CT before surgical resection were included in the test cohort and 12 patients in the validation cohort. GTVs were delineated on the staging PET/CT by automatic delineation using various percentage threshold of maximum SUV (SUVmax) and absolute SUV. The maximum pathological tumor diameter was then matched with the maximum auto-delineated tumor diameter with varying SUV thresholds. First-order linear regression and Bland–Altman plots were used to obtain an optimal SUV threshold for each patient. Three radiation oncologists with varying degrees of experiences also delineated GTVs with the visual aid of PET/CT to assess interobserver variation in delineation.Results:In the test set, the mean optimal percentage threshold for GTV was SUVmax of 35.6%±18.6% and absolute SUV of 4.35 ± 1.7. In the validation set, the mean optimal percentage threshold SUV and absolute SUV were 36.9 ± 16.9 and 4.1 ± 1.6, respectively. After a combined analysis of all 37 patients, the mean optimal threshold was 36% ± 17.9% and 4.27 ± 1.7, respectively. Using Bland–Altman plots, auto-contouring with 40% SUVmax and SUV 4 was in greater agreement with the pathological tumor diameter.Conclusion:Automatic GTV delineation on PETCT in NSCLC with percentage threshold SUV of 40% and absolute SUV of 4 correlated best with pathological tumor size. Auto-contouring using these thresholds will increase the precision of radiotherapy contouring of GTV and will save time.

OC-0681: Deep learning auto contouring of OAR for HN radiotherapy: a blinded evaluation by clinical experts

OC-0681: Deep learning auto contouring of OAR for HN radiotherapy: a blinded evaluation by clinical experts

Quantitative and Qualitative Evaluation of a Deep Learning Auto Contouring Model for Prostate Cancer Patients With Hydrogel Spacer

To quantitatively and qualitatively evaluate a deep learning auto contouring model for prostate radiotherapy patients with pretreatment insertion of a hydrogel spacer (about water equivalent with no contrast) between prostate and rectum. The model employs convolutional neural networks (CNN) to learn features from input images that can be used to generate semantic segmentation. The study used 163 patients from three specialized GU radiation oncologists (referred to as A/B/C). The first 135 patients (A/B/C = 82/39/14) were used for training (125) and validation (10). The validation patients were randomly selected. The validated model was tested on 28 patients (A/B/C = 18/6/4) accrued during model development. There was no change of practice during the whole period. A simulation CT and MR were taken on the same day for each patient. In manual contouring, with MR fused to CT, spacer was contoured on T2 MR, prostate on CT with MR guidance, and other structures on CT only. The model was trained to auto contour prostate, proximal seminal vesicles (SV), bladder, rectum, penile bulb, femurs and spacer on CT without MR. Quantitatively, auto contours were evaluated against manual contours using the following metrics: sensitivity (% of voxels correctly drawn), false positive rate (FPR, % of voxels overdrawn), dice similarity coefficient (DSC), 95-percentile of Hausdorff distance (HD) and mean distances (dmean) between the two contours over all slices. The structures with high DSC were qualitatively evaluated by the original attending using a 1 (acceptable with minor editing), 2 (editable with efficiency gain over manual contouring) and 3 (rejected for no efficiency gain or gross error) scoring system. A gross error on rectum occurred for two patients (A/B = 1/1). These two points were excluded from quantitative analysis but counted as rejected in qualitative evaluation. On average, DSC was high for femurs (>0.95) and bladder (0.91), moderate for prostate (0.85) and rectum (0.81), but low for bulb (0.67), proximal SV (0.62) and spacer (0.52). For right femur/left femur/bladder/prostate/rectum, sensitivity = 0.93/0.92/0.88/0.86/0.81, FPR = 1.8%/1.5%/4.5%/15%/17%, 95% 95%-HD = 2.8/2.6/12.1/7.4/9.5 mm, and dmean = 0.9/1.0/2.6/2.5/2.4 mm. Qualitatively, femurs scored 1 in all cases. The average scores for bladder/prostate/rectum = 1.28/1.44/1.50, 1.83/2.17/1.67, 1.25/1.50/1.25 for physicians A, B, C, respectively, and 1.39/1.61/1.50 overall. Prostate and rectum both scored well below 2, despite their lower quantitative performance, as some errors caused by the inaccurate prediction of spacer without MR were deemed easily correctable by the physicians. The model produced clinically satisfactory results, both quantitatively and qualitatively, for femurs, bladder, prostate and rectum. The results for proximal SV and bulb were less ideal. The model drew the spacer in the correct location, but could not draw it accurately due to lack of contrast on CT.

A Blinded Prospective Evaluation Of Clinical Applicability Of Deep Learning-Based Auto Contouring Of OAR For Head and Neck Radiotherapy

A Blinded Prospective Evaluation Of Clinical Applicability Of Deep Learning-Based Auto Contouring Of OAR For Head and Neck Radiotherapy