Automated OAR Anomaly and Error Detection Tool in Radiation Therapy

Abstract

Organ at risk (OAR) segmentation anomalies can negatively impact treatment planning quality, and potentially increase treatment morbidity. Therefore, we developed and tested the efficacy of a quality assurance (QA) tool to assist in identifying potential OAR segmentation errors. The QA tool computes OAR volumetric features and compares them with respect to a statistical reference developed from historical priors. In contrast to previous approaches, all features were computed in 3-dimensional space. Furthermore, feature comparison was used in a multi-criteria triggering system that warns user of specific errors based on combinations of feature anomalies. The statistical reference contained features derived from 73 prior lung cancer patients. The tool was assessed using 15 independent test sets: 9 clinical manual segmentation (MS), 3 automatic segmentation (AS) and 3 propagated segmentation (PS) from deformable mapping. The test sets contained 107 OAR structures. The test MS sets included 34 OAR manipulations (ditzel, missing slice, expansion/contraction, extra segment and mislabeling) to mimic common segmentation errors. Three experienced radiation oncology residents reviewed the test sets to identify segmentation errors, first without; and then one week after with the help of the QA tool. Each reviewer classified findings as minor or major errors. For the purpose of analysis, a minor or major error was defined if 2 or more reviewers identified the exact same error during either trial. Of the 34 manipulations, reviewers identified 8 as minor and 23 as major errors. Three manipulations were not classified as errors by the reviewers. Reviewers identified an additional 30 minor and 11 major errors in the OAR structures. Overall, the QA tool detected 37% and 85% of the minor and major errors respectively. The tool sensitivity for specific error types was 47% for boundary errors in the axial plane, 25% for slice extent errors, and 100% for ditzel, missing slice, extra segment and mislabeling. The QA tool improved the user error detection sensitivity from 61% to 68% (p = 0.17) for minor errors, and from 78% to 87% (p = 0.02) for major errors. Table 1 summarizes the error detection sensitivity by the QA tool and the reviewers. The proposed QA tool can assist users in detection of segmentation errors. Our experiment demonstrated that it can significantly improve detection of major errors. The QA tool’s ability to reduce the frequency of OAR segmentation error can potentially improve safety and quality of radiation therapy.Abstract 3317; Table 1Error detection sensitivity by the QA tool and reviewersQA tool (%)Reviewers - w/o QA tool (%)Reviewers - w/ QA tool (%)Minor ErrorAutomatic segmentation (AS)676789Propagated segmentation (PS)08164Manual segmentation w/ manipulation (MS)564465Total376168Major ErrorAS809393PS2510092MS967488Total858089 Open table in a new tab