Evaluation of MIM ProtegeAI Prostate: Time Gain, Accuracy and Dosimetric Consequences

Abstract

<h3>Purpose/Objective(s)</h3> To compare and measure the potential benefit of time-saving, accuracy and dosimetric impact, the MIM ProtegeAI Prostate 2.0 auto-segmentation™ was compared to the manual delineation of the radiation oncologist. Furthermore, the intra-observer variability and familiarization bias were also assessed. <h3>Materials/Methods</h3> Forty-eight consecutive prostate cancer cases have been retrospectively investigated. The planning CT scan (3mm slice thickness, from vertebrae L1/2 to 3cm below the ischial tuberosity, without contrast) was used for auto-contouring (<i>AI</i>) and manual delineation by a single observer (<i>Manual</i>). Femur_L/_R, PenileBulb, Rectum, SeminalVes and Bladder were evaluated; while another five AI-generated OARs were discarded. Time (minute: second; [Q1-Q3]) of <i>AI</i> delineation scoring (<i>AIscor</i>: major/minor/no correction needed), AI correction (<i>AIcor</i>), total <i>AI</i> (=AIscor+AIcor) and manual delineation were measured. The potential gain was assessed per individual OARs. To access familiarization half of the cohort started with <i>AIscor</i> and <i>AIcor</i> followed by <i>Manual</i>, while the other half started with <i>Manual</i>, followed by <i>AIscor</i> and <i>AIcor</i>. T-test at p<0.05 significance level was used for comparisons. Dice Similarity Coefficient (DSC), 95% Hausdorff and median surface distance (HD95, MSD) were also determined. Given that both contours were found to be clinically acceptable by the same radiation oncologist, the <i>AIcor</i>/<i>Manual</i> was used to define intra-observer variability. Original dose matrices were used to access median dose differences for each evaluated organ. <h3>Results</h3> A total of 473 contours were generated by <i>AI</i><b>,</b> with a fixed of 5 minutes per patient. For 36 patients (75%), AI failed to generate Kidney_L/_R. Major, minor, or no correction was considered in 14%, 72% and 14% of delineations, respectively. <i>Manual</i> delineation took on average 12:52 [7:51-24:19], <i>AIscor</i> and <i>AIcor</i> 2:45 [1:52-5:00] and 6:18 [2:49-14:14], respectively. AI gave up to 16:16 time gain with an average of 5:37 (p<0.001). Per OARs, the average time gain was 0:47 [0:11-1:44]. The familiarization bias, observed for <i>Manual</i> (p=0.09), was on average 1:27 faster when AI workflow started first, while for <i>AIcor</i> no significant bias was observed (p=0.168). Good DSC (>0.8) was observed for <i>AI/AIcor</i>, while HD95 and MSD showed a larger discrepancy. Intra-observer (<i>AIcor</i>/<i>Manual</i>) variability was worse for DSC and better for HD95 and MSD compared to <i>AI</i> vs. <i>AIcor</i>. Relevant (>5%) median dose (Gray) differences were observed for only two OARs: rectum and penile bulb. <h3>Conclusion</h3> ProtegeAI Prostate 2.0 auto-segmentation™ provides on average more than 4 minutes of gain per patient and requires only minor corrections. The realistic time saving is likely higher, as <i>AIscor+AIcor</i> prior manual delineation significantly reduced manual delineation time. Intra-observer variability remains a substantial source of differences, especially when relying on the DSC. Dose differences may be as high as 5% when using the corrected AI.