The Objectives-Met Histogram: A Novel Way to Measure Plan Quality

Abstract

As radiation therapy technologies grow ever more complex and precise, the assessment of plan quality remains largely subjective. The industry would benefit from objective, comprehensive, and clinically relevant methods to quantify plan quality. In this work, we introduce a novel construct called the “Objectives-Met Histogram” (OMH). It is hypothesized that use of the OMH would (1) help trained clinicians make efficient, data-driven decisions in reviewing patient plans prior to treatment, and (2) form the basis for scientifically valid, robust comparisons of competing treatment modalities and methods. The OMH quantifies and graphs plan quality based on all relevant treatment plan metrics, each weighted by its clinical importance. An OMH algorithm is first built per treatment protocol, and then OMH results are calculated and graphed for any patient plan in that class. Building the OMH algorithm follows these steps: (1) Clinical experts identify all metrics (dose-volume histograms [DVH], conformality, efficiency, etc.) to extract and quantify per plan; (2) each metric type is assigned five “performance bins” (unacceptable, marginal, acceptable, good, and ideal) to quantify performance of the metric’s result; and (3) each metric is weighted according to its relative importance. For any given treatment plan dataset (which includes all of the plan parameters, resulting 3D dose, and input target and critical structures), an OMH is computed as follows: (1) each metric result is extracted and the corresponding performance bin derived; (2) the bin result is multiplied by its relative weight; (3) results are tallied over all metrics; and (4) the plan quality is presented as a line graph with the weighted count of “objectives met” on the y-axis and the increasing objectives (i.e., performance bins) on the x-axis. Effectiveness of OMH to assess plan quality was studied for multiple plans produced for 3 breast patient datasets (intact, postmastectomy, and with expander). Competing plans were compared across multiple planners, planning systems, and modalities (i.e., VMAT vs. proton). OMH curves allow rapid interpretation of otherwise complex data. Analysis is similar to how cumulative DVHs are analyzed for target coverage. For the collection of breast plans analyzed, OMH analyses facilitate (1) identification of any “unacceptable” results, as indicated by a gap at the top of the graph; (2) knowing how many metrics (and cumulative clinical weights) achieved each performance level or better; and (3) objectively comparing multiple plans, free of any potential subjectivity or bias. OMH analysis of plan quality is effective, efficient, and objective. Adopting this method could be valuable in day-to-day plan review. OMH could also be used to fairly compare competing modalities and to help justify payment based not on specific technology used, but instead on actual quality achieved.