Machine Learning to Improve the Prioritization and Effectiveness of Pre-Treatment Physics Chart Checks

Abstract

Quality assurance (QA) is an integral part of planning for every patient prior to administering radiation therapy. The nationally regulated approach to QA involves performing multiple chart checks and often requires a high cognitive burden and significant amount of time. The purpose of this study is to explore the use of machine learning to classify the potential difficulty of physics pre-treatment chart check QA for a given radiation plan. Herein, we demonstrate a model that classifies radiation treatment plans as easy, medium, or hard with reference to QA time involved and cognitive burden. Pre-treatment data was collected for 1343 cases encompassing all cancer sites that were planned between August 2018 and October 2019. The outcome variable, a degree of difficulty on a scale of 1-10 was collected as a subjective rating by physicists who performed the pre-treatment chart checks. Scores of 1-3 were labeled as easy, 4-6 as intermediate, and 7-10 as difficult. A clinician and physicist then identified potential features, based on clinical relevance, contribution to plan complexity, and QA metrics. 1074 patient plans were used as a training set and the remaining 269 were set aside as a test set, an 80/20 split. The training data was used to train a k-nearest neighbor (KNN) classifier (k = 2) and a neural network (hidden layers = 3). Forward feature elimination was then utilized to incorporate only variables of importance prior to training decision tree (max depth = 10), random forest, and Adaboost (max depth = 0) classifiers. These five algorithms were incorporated into a majority voting classifier. Both training and test sets were equivalent in their proportion of difficult (11%), intermediate (41%), and easy (47%) cases. Accuracy on the test set was calculated for each classifier and was found to be 96% for the Adaboost and random forest classifiers, 94% for the neural network, 90% for the decision tree, and 84% for the KNN. The majority voting classifier achieved an accuracy of 95% overall, with 94% accuracy on difficult cases, 95% on medium cases, and 95% on easy cases. Precision, recall, and F-score were also calculated for each classifier and for the majority vote (see Table). We present a majority voting classifier that uses clinical, plan complexity, and QA features in order to predict the difficulty of radiation treatment plans. This will be used in our department to guide the physicist in prioritizing their activities in an effort to reduce cognitive workload and improve the effectiveness of pre-treatment chart checks. Future directions include validating this model at our community practices and addressing limitations such as class imbalance and missing data.Abstract 110; TableAdaboostDecision TreeKNNNeural NetworkRandom ForestMajority (hard vote)Accuracy96%90%84%94%96%95%Precision (PPV)96%93%86%95%97%96%Recall (Sensitivity)96%88%82%93%96%95%F-score96%90%84%94%96%95% Open table in a new tab