WE-G-108-06: Evaluating Pre-Treatment IMRT & VMAT QA Techniques Quantitatively Using Receiver Operating Characteristic (ROC) Analysis

Abstract

Purpose: Pre‐treatment IMRT and VMAT QA techniques are often commissioned without knowledge of their sensitivity to clinically relevant delivery errors. The purpose of this work is to develop a method to quantify the sensitivity and specificity of pre‐treatment IMRT and VMAT QA techniques to treatment delivery errors. Methods: To evaluate a QA technique, a population of treatment plans and a population of clinically relevant delivery errors are defined. For each delivery error, a threshold magnitude is determined that induces a substantial change in clinically relevant dosimetric indices. Errors at the threshold magnitude are introduced into the plans and QA is performed with and without intentionally introduced errors. The QA technique is treated as a binary classifier to predict error plans using Receiver Operator Characteristic (ROC) analysis. We applied this technique to evaluate portal imager and 2D ion chamber array based QA for VMAT treatment of brain lesions. Delivery errors included discrepancies in MLC positioning (single leaf and leaf bank); lag of MLC trajectory; and discrepancy in dose rate per control point or gantry angle. The threshold magnitude was determined by achieving a 5% change in target conformity index. Results: The area under the curve (AUC) for the ROC analysis was 0.592 and 0.509 for the ion chamber array and portal imager, respectively, using a gamma index of 3%, 3mm. The AUC increased to 0.632 and 0.777 when 2%, 2mm was used for the ion chamber array and portal imager, respectively. Comparison based on 3% dose agreement resulted in an AUC of 0.557 and 0.693, respectively. Conclusion: For both portal imager and ion chamber array based QA, stricter tolerance than 3%, 3mm is needed to detect clinically relevant delivery errors. This method can be used to quantitatively compare the sensitivity of various QA techniques to clinically relevant dosimetric errors.