Six Sigma-driven Automated Plan Check (APC) Tool Enhances Safety and Efficiency in External Beam Radiation Therapy

Abstract

To evaluate the impact on safety and efficiency of an in-house developed Automated Plan Check (APC) tool. The common treatment planning error pathways reported to our departmental incidence learning system (ILS) were combined with our departmental physics plan check list, as well as AAPM TG-275 identified plan check items to serve as the foundation for the APC check items. This list was further evaluated using Failure Mode and Effects Analysis (FMEA) by a multidisciplinary team of physicians, physicists, dosimetrists and therapists to identify high-severity and high-risk Priority Numbers (RPN) check items and prioritize them for implementation into the APC tool. The tool was built using a treatment planning system, implemented in the clinic in January 2018 and since then is continually updated within a Six Sigma Define-Measure-Analyze-Implement-Control (DMAIC) framework. At 2 years post-APC implementation, the tool encompassed 154 plan check items verifying 3D-Conformal, clinical electron, VMAT, SBRT, Total Skin Electron Irradiation (TSEI) and Total Body Irradiation (TBI) plans. Full plan check automation was achieved for the templated TSEI treatment plans. Population statistics was used to identify outliers for TBI treatment plans. Evaluation of dosimetric indices based on site-specific and physician-specific dosimetric constraints and RTOG/NRG/QUANTEC/TG-101 dosimetric constraints was added to the APC to improve evaluation of plan quality and clinical trial compliance. The APC effectiveness was assessed by comparing FMEA RPN values and rates of reported errors. To test the efficiency gains, physics plan check time and reported error rate were prospectively compared for 20 treatment plans. The APC tool was successfully implemented for external beam plan checking. Since its implementation, the APC tool was used to verify 5946 treatment plans. FMEA RPN ranking re-evaluation at 2 years post-APC showed a statistically significant decrease in average RPN values from 129.2±51.1 to 66.1±57.5 (p < 0.001). After the introduction of APC, the average frequency of reported treatment-planning errors was reduced from 16.1% to 2.3%. For high-severity errors, the reduction was 82.7% for prescription/plan mismatches and 84.4% for incorrect shift note. There were no reported high-severity treatment planning errors that propagated to treatment since the APC introduction. The process shifted from 4σ to 5σ quality for isocenter-shift errors frequent prior to APC implementation. The efficiency study showed a statistically significant gain in plan check time (10.1±7.3 minutes, p = 0.005) and decrease in errors propagating to physics plan check (80%). Incorporation of APC tool has significantly reduced the error rate. The Six Sigma-driven framework can provide an iterative and robust workflow to improve the efficiency and quality of treatment planning procedure enabling a safer radiotherapy process.