Plan Checking Research Articles

Analysis of human errors in the operation of various treatment planning systems over a 10-year period.

The present study aimed to summarize and report data on errors related to treatment planning, which were collected by medical physicists. The following analyses were performed based on the 10-year error report data: (1) listing of high-risk errors that occurred and (2) the relationship between the number of treatments and error rates, (3) usefulness of the Automated Plan Checking System (APCS) with the Eclipse Scripting Application Programming Interface and (4) the relationship between human factors and error rates. Differences in error rates were observed before and after the use of APCS. APCS reduced the error rate by ~1% for high-risk errors and 3% for low-risk errors. The number of treatments was negatively correlated with error rates. Therefore, we examined the relationship between the workload of medical physicists and error occurrence and revealed that a very large workload may contribute to overlooking errors. Meanwhile, an increase in the number of medical physicists may lead to the detection of more errors. The number of errors was correlated with the number of physicians with less clinical experience; the error rates were higher when there were more physicians with less experience. This is likely due to the lack of training among clinically inexperienced physicians. An environment to provide adequate training is important, as inexperience in clinical practice can easily and directly lead to the occurrence of errors. In any environment, the need for additional plan checkers is an essential factor for eliminating errors.

Standardization of Stereotactic MR-Guided Adaptive Radiation Therapy Planning for Prostate Cancer

To assess the impact of a standardized planning class solution (PCS) in stereotactic MR-guided adaptive radiation therapy (SMART) for prostate cancer on efficiency and robustness for daily adaptive radiation therapy treatments. Data collection of 20 previous SMART prostate plans: number of objective structures, number of beam segments, treatment delivery time (TDT) and the plan adaption robustness (PAR). The PAR was scored from 1 to 3 for the robustness of the plan to re-adapt and achieve similar dose metric scores to the baseline plan on all fractions. These scores were, 1: a single re-optimization required, 2: cost-function adjustments required and 3: medical physics advice required. Five medical dosimetrists provided preferences for objective structures and the optimization thresholds used. A PCS was created based on this data and implemented into SMART prostate plans between October 2022 and December 2022. A consecutive sample of 20 patients (PCSpost) were then retrospectively compared with the previous 20 patients prior to the PCS (PCSpre). The PCS showed minimal difference in the TDT and segment number mean ± (standard deviation), 9.5 ± (0.7) mins and 77.8 ± (6.9) segments PCSpost respectively compared to 9.5 ± (0.8) mins and 80.8 ± (5.9) segments PCSpre. However, the PCS resulted in a considerable reduction in PAR, with 11 patients requiring only a single re-optimization for each fraction to achieve dose metric targets compared to 7 PCSpre patients. Also, the median PAR score was 1 in PCSpost compared to a score of 2 with PCSpre. The PCS had good compliance with the criteria met in 18/20 patient baseline plans. Feedback surveys showed positive results for efficiency saving from multiple disciplines; in therapeutic radiographers from reduced PAR, medical physicists increased automation in plan checking and consistent planning guidelines for medical dosimetrists. The PCS is in clinical use, results have shown improved treatment adaption efficiency and robustness. Fewer re-optimizations per treatment were required onset to achieve similar dose metrics than before its implementation.

Augmented Reality for Building Authorities: A Use Case Study in Austria

The widespread usage of BIM and technological advances in augmented reality (AR) paves the way for Construction 4.0. This digital transformation has reached the construction industry and requires consideration of all life cycle phases. The approval phase has been little researched to date. In Vienna, the analogous process takes up to 18 months. This long duration inhibits progress in the building industry. One reason for this is objections from parties who are nonexperts and have difficulty understanding the project solely from its 2D plans. Therefore, the City of Vienna wants to redesign the building authority processes through the use of BIM and AR in the BRISE-Vienna research project. This article examines possible applications for public authorities. AR can be utilised at the interface between authorities and stakeholders and is intended to accelerate processes through better communication. For the City of Vienna, 12 AR use cases were developed from process analyses and expert interviews. In addition to the conception of the AR use cases for public authorities, a study was carried out regarding their benefits and feasibility. The use cases have the potential to increase the understanding of the project for experts and nonexperts. The best-rated AR use cases (plan checking and hearing during the permission process) will be further developed in the research project. The AR use cases should help to significantly accelerate processes in Vienna and, thus, serve as a basis for other cities and countries.

Anomaly detection in radiotherapy plans using deep autoencoder networks.

Treatment plans are used for patients under radiotherapy in clinics. Before execution, these plans are checked for safety and quality by human experts. A few of them were identified with flaws and needed further improvement. To automate this checking process, an unsupervised learning method based on an autoencoder was proposed. First, features were extracted from the treatment plan by human experts. Then, these features were assembled and used for model learning. After network optimization, a reconstruction error between the predicted and target signals was obtained. Finally, the questionable plans were identified based on the value of the reconstruction error. A large value of the reconstruction error indicates a longer distance from the standard distribution of normal plans. A total of 576 treatment plans for breast cancer patients were used for the test. Among them, 19 were questionable plans identified by human experts. To evaluate the performance of the autoencoder, it was compared with four baseline detection algorithms, namely, local outlier factor (LOF), hierarchical density-based spatial clustering of applications with noise (HDBSCAN), one-class support vector machine (OC-SVM), and principal component analysis (PCA). The results showed that the autoencoder achieved the best performance than the other four baseline algorithms. The AUC value of the autoencoder was 0.9985, while the second one was 0.9535 (LOF). While maintaining 100% recall, the average accuracy and precision of the results by the autoencoder were 0.9658 and 0.5143, respectively. While maintaining 100% recall, the average accuracy and precision of the results by LOF were 0.8090 and 0.1472, respectively. The autoencoder can effectively identify questionable plans from a large group of normal plans. There is no need to label the data and prepare the training data for model learning. The autoencoder provides an effective way to carry out an automatic plan checking in radiotherapy.

Assessing initial plan check efficacy using TG 275 failure modes and incident reporting.

Plan checks are important components of a robust quality assurance (QA) program. Recently, the American Association of Physicists in Medicine (AAPM) published two reports concerning plan and chart checking, Task Group (TG) 275 and Medical Physics Practice Guideline (MPPG) 11.A. The purpose of the current study was to crosswalk initial plan check failure modes revealed in TG 275 against our institutional QA program and local incident reporting data. Ten physicists reviewed 46 high‐risk failure modes reported in Table S1.A.i of the TG 275 report. The committee identified steps in our planning process which sufficiently checked each failure mode. Failure modes that were not covered were noted for follow‐up. A multidisciplinary committee reviewed the narratives of 1599 locally‐reported incidents in our Radiation Oncology Incident Learning System (ROILS) database and categorized each into the high‐risk TG 275 failure modes. We found that over half of the 46 high‐risk failure modes, six of which were top‐ten failure modes, were covered in part by daily contouring peer‐review rounds, upstream of the traditional initial plan check. Five failure modes were not adequately covered, three of which concerned pregnancy, pacemakers, and prior dose. Of the 1599 incidents analyzed, 710 were germane to the initial plan check, 23.4% of which concerned missing pregnancy attestations. Most, however, were caught prior to CT simulation (98.8%). Physics review and initial plan check were the least efficacious checks, with error detection rates of 31.8% and 31.3%, respectively, for some failure modes. Our QA process that includes daily contouring rounds resulted in increased upstream error detection. This work has led to several initiatives in the department, including increased automation and enhancement of several policies and procedures. With TG 275 and MPPG 11.A as a guide, we strongly recommend that departments consider an internal chart checking policy and procedure review.

Automated Plan Checking Software Demonstrates Continuous and Sustained Improvements in Safety and Quality: A 3-year Longitudinal Analysis

PurposeThis study aimed to perform a longitudinal analysis of the performance of our automated plan checking software by retrospectively evaluating the number of errors identified in plans delivered to patients in 3, month-long, data collection periods between 2017 and 2020. Methods and MaterialsEleven automated checks were retrospectively run on 1169 external beam radiation therapy treatment plans identified as meeting the following criteria: planning target volume–based multifield photon plans receiving a status of treatment approved in March 2017, March 2018, or March 2020. The number of passes (true positives) and flags were recorded. Flags were subcategorized into false negatives, false negatives due to naming conventions, and true negatives. In addition, 2 × 2 contingency tables using a 2-tailed Fisher's exact test were used to determine whether there were nonrandom associations between the output of the automated plan checking software and whether the check was manual or automated at the original time of treatment approval. ResultsA statistically significant decrease in flags between the pre- and postautomation data sets was observed for 4 contour-based checks, namely adjacent structures overlap, empty structures and missing slices, overlap between body and couch, and laterality, as well as a check that determined whether the plan's global maximum dose was within the planning target volume. A review of the origins of false negatives was fed back into the design of the checks to improve the reliability of the system and help avoid warning fatigue. ConclusionsPeriodic and longitudinal review of the performance of automated software was essential for monitoring and understanding its impact on error rates, as well as for optimization of the tool to adapt to regular changes of clinical practice. The automated plan checking software has demonstrated continuous contributions to the safe and effective delivery of external beam radiation therapy to our patient population, an impact that extends beyond its initial implementation and deployment.

Potentials of Augmented Reality in a BIM based building submission process

The digital transformation in the construction industry, which is also referred to as Construction 4.0, is gaining impact in recent years. BIM as the central point for model-based planning and communication is at the centre of this development. This paves the way for the implementation of further technologies, such as augmented reality (AR) or artificial intelligence (AI) in the construction process. However, the submission process currently receives little attention in the BIM project cycle, even though it has a significant impact on a building project. There is a huge potential for the application of BIM and AR in the building submission process. BIM allows partly automatic compliance checks to be performed. The BIM model and the results of these checks can be used as basis for the AR model. This paper investigates an application for AR in a BIM submission process. Here we show that especially the plan check process and hearings in the approval process can be improved and sped up by combining BIM and the visualization technology of AR. In this process, AR can support the building authority in checking the compliance with building regulations. Additionally, non-experts involved often have problems grasping the planned building project just based on the technical 2D plans. In this case, AR helps them to visualise the impact of the planned project and allows for better judgement. Use-cases are presented, showing strategies to improve the plan check process and assist the building authority as well as the persons involved, for both experts as well as non-experts. Finally, the requirements, technical implementations, and effects of an AR application for the plan check process are described. An AR-supported plan checking process can vastly improve the presentation of technical content. This leads to better understanding and more transparency for non-experts. Because AR is relatively easy to use, it might become ubiquitous not only to experts, but also to the general public.

Six Sigma-driven Automated Plan Check (APC) Tool Enhances Safety and Efficiency in External Beam Radiation Therapy

To evaluate the impact on safety and efficiency of an in-house developed Automated Plan Check (APC) tool. The common treatment planning error pathways reported to our departmental incidence learning system (ILS) were combined with our departmental physics plan check list, as well as AAPM TG-275 identified plan check items to serve as the foundation for the APC check items. This list was further evaluated using Failure Mode and Effects Analysis (FMEA) by a multidisciplinary team of physicians, physicists, dosimetrists and therapists to identify high-severity and high-risk Priority Numbers (RPN) check items and prioritize them for implementation into the APC tool. The tool was built using a treatment planning system, implemented in the clinic in January 2018 and since then is continually updated within a Six Sigma Define-Measure-Analyze-Implement-Control (DMAIC) framework. At 2 years post-APC implementation, the tool encompassed 154 plan check items verifying 3D-Conformal, clinical electron, VMAT, SBRT, Total Skin Electron Irradiation (TSEI) and Total Body Irradiation (TBI) plans. Full plan check automation was achieved for the templated TSEI treatment plans. Population statistics was used to identify outliers for TBI treatment plans. Evaluation of dosimetric indices based on site-specific and physician-specific dosimetric constraints and RTOG/NRG/QUANTEC/TG-101 dosimetric constraints was added to the APC to improve evaluation of plan quality and clinical trial compliance. The APC effectiveness was assessed by comparing FMEA RPN values and rates of reported errors. To test the efficiency gains, physics plan check time and reported error rate were prospectively compared for 20 treatment plans. The APC tool was successfully implemented for external beam plan checking. Since its implementation, the APC tool was used to verify 5946 treatment plans. FMEA RPN ranking re-evaluation at 2 years post-APC showed a statistically significant decrease in average RPN values from 129.2±51.1 to 66.1±57.5 (p < 0.001). After the introduction of APC, the average frequency of reported treatment-planning errors was reduced from 16.1% to 2.3%. For high-severity errors, the reduction was 82.7% for prescription/plan mismatches and 84.4% for incorrect shift note. There were no reported high-severity treatment planning errors that propagated to treatment since the APC introduction. The process shifted from 4σ to 5σ quality for isocenter-shift errors frequent prior to APC implementation. The efficiency study showed a statistically significant gain in plan check time (10.1±7.3 minutes, p = 0.005) and decrease in errors propagating to physics plan check (80%). Incorporation of APC tool has significantly reduced the error rate. The Six Sigma-driven framework can provide an iterative and robust workflow to improve the efficiency and quality of treatment planning procedure enabling a safer radiotherapy process.

Efficiency and safety increases after the implementation of a multi-institutional automated plan check tool at our institution.

PurposeThe plan check tool (PCT) is the result of a multi‐institutional collaboration to jointly develop a flexible automated plan checking framework designed with the versatility to be shared across collaborating facilities while supporting the individual differences between practices. We analyze the effect that PCT has had on the efficiency and effectiveness of initial chart checks at our institution.Methods and MaterialsData on errors identified during initial chart checks were acquired during two time periods: before the introduction of PCT in the clinic (6/24/2015 to 7/31/2015, 187 checks) and post‐clinical release (4/14/2016 to 5/2/2016, 186 checks). During each time period, human plan checkers were asked to record all issues that they either manually detected or that were detected by PCT as well as the amount of time, less breaks, or interruptions, it took to check each plan.ResultsAfter the clinical release of PCT, there was a statistically significant decrease in the number of issues recorded by the human plan checkers both related to checks explicitly performed by PCT (13 vs 50, P < 0.001) and in issues identified overall (127 vs 200, P < 0.001). The mean and medium time for a plan check decreased by 20%.ConclusionsThe use of a multi‐institutional, configurable, automated plan checking tool has resulted in both substantial gains in efficiency and moving error detection to earlier points in the planning process, decreasing their likelihood that they reach the patient. The sizeable startup effort needed to create this tool from scratch was mitigated by the sharing, and subsequent co‐development, of software code from a peer institution.

Optimizing Efficiency and Safety in External Beam Radiotherapy Using Automated Plan Check (APC) Tool and Six Sigma Methodology

PURPOSE To develop and implement an automated plan check (APC) tool using a Six Sigma methodology with the aim of improving safety and efficiency in external beam radiotherapy. METHODS The Six Sigma define-measure-analyze-improve-control (DMAIC) framework was used by measuring defects stemming from treatment planning that were reported to the departmental incidence learning system (ILS). The common error pathways observed in the reported data were combined with our departmental physics plan check list, and AAPM TG-275 identified items. Prioritized by risk priority number (RPN) and severity values, the check items were added to the APC tool developed using Varian Eclipse Scripting Application Programming Interface (ESAPI). At 9 months post-APC implementation, the tool encompassed 89 check items, and its effectiveness was evaluated by comparing RPN values and rates of reported errors. To test the efficiency gains, physics plan check time and reported error rate were prospectively compared for 20 treatment plans. RESULTS The APC tool was successfully implemented for external beam plan checking. FMEA RPN ranking re-evaluation at 9 months post-APC demonstrated a statistically significant average decrease in RPN values from 129.2 to 83.7 (P < .05). After the introduction of APC, the average frequency of reported treatment-planning errors was reduced from 16.1% to 4.1%. For high-severity errors, the reduction was 82.7% for prescription/plan mismatches and 84.4% for incorrect shift note. The process shifted from 4σ to 5σ quality for isocenter-shift errors. The efficiency study showed a statistically significant decrease in plan check time (10.1 ± 7.3 min, P = .005) and decrease in errors propagating to physics plan check (80%). CONCLUSIONS Incorporation of APC tool has significantly reduced the error rate. The DMAIC framework can provide an iterative and robust workflow to improve the efficiency and quality of treatment planning procedure enabling a safer radiotherapy process.

Optimizing efficiency and safety in external beam radiotherapy using automated plan check (APC) tool and six sigma methodology.

PurposeTo develop and implement an automated plan check (APC) tool using a Six Sigma methodology with the aim of improving safety and efficiency in external beam radiotherapy.MethodsThe Six Sigma define‐measure‐analyze‐improve‐control (DMAIC) framework was used by measuring defects stemming from treatment planning that were reported to the departmental incidence learning system (ILS). The common error pathways observed in the reported data were combined with our departmental physics plan check list, and AAPM TG‐275 identified items. Prioritized by risk priority number (RPN) and severity values, the check items were added to the APC tool developed using Varian Eclipse Scripting Application Programming Interface (ESAPI). At 9 months post‐APC implementation, the tool encompassed 89 check items, and its effectiveness was evaluated by comparing RPN values and rates of reported errors. To test the efficiency gains, physics plan check time and reported error rate were prospectively compared for 20 treatment plans.ResultsThe APC tool was successfully implemented for external beam plan checking. FMEA RPN ranking re‐evaluation at 9 months post‐APC demonstrated a statistically significant average decrease in RPN values from 129.2 to 83.7 (P < .05). After the introduction of APC, the average frequency of reported treatment‐planning errors was reduced from 16.1% to 4.1%. For high‐severity errors, the reduction was 82.7% for prescription/plan mismatches and 84.4% for incorrect shift note. The process shifted from 4σ to 5σ quality for isocenter‐shift errors. The efficiency study showed a statistically significant decrease in plan check time (10.1 ± 7.3 min, P = .005) and decrease in errors propagating to physics plan check (80%).ConclusionsIncorporation of APC tool has significantly reduced the error rate. The DMAIC framework can provide an iterative and robust workflow to improve the efficiency and quality of treatment planning procedure enabling a safer radiotherapy process.

On the use of a convolution-superposition algorithm for plan checking in lung stereotactic body radiation therapy.

Stereotactic body radiation therapy (SBRT) aims to deliver a highly conformal ablative dose to a small target. Dosimetric verification of SBRT for lung tumors presents a challenge due to heterogeneities, moving targets, and small fields. Recent software (M3D) designed for dosimetric verification of lung SBRT treatment plans using an advanced convolution–superposition algorithm was evaluated. Ten lung SBRT patients covering a range of tumor volumes were selected. 3D CRT plans were created using the XiO treatment planning system (TPS) with the superposition algorithm. Dose was recalculated in the Eclipse TPS using the AAA algorithm, M3D verification software using the collapsed‐cone‐convolution algorithm, and in‐house Monte Carlo (MC). Target point doses were calculated with RadCalc software. Near‐maximum, median, and near‐minimum target doses, conformity indices, and lung doses were compared with MC as the reference calculation. M3D 3D gamma passing rates were compared with the XiO and Eclipse. Wilcoxon signed‐rank test was used to compare each calculation method with XiO with a threshold of significance of p<0.05. M3D and RadCalc point dose calculations were greater than MC by up to 7.7% and 13.1%, respectively, with M3D being statistically significant (s.s.). AAA and XiO calculated point doses were less than MC by 11.3% and 5.2%, respectively (AAA s.s.). Median and near‐minimum and near‐maximum target doses were less than MC when calculated with AAA and XiO (all s.s.). Near‐maximum and median target doses were higher with M3D compared with MC (s.s.), but there was no difference in near‐minimum M3D doses compared with MC. M3D‐calculated ipsilateral lung V20 Gy and V5 Gy were greater than that calculated with MC (s.s.); AAA‐ and XiO‐calculated V20 Gy was lower than that calculated with MC, but not statistically different to MC for V5 Gy. Nine of the 10 plans achieved M3D gamma passing rates greater than 95% and 80%for 5%/1 mm and 3%/1 mm criteria, respectively. M3D typically calculated a higher target and lung dose than MC for lung SBRT plans. The results show a range of calculated doses with different algorithms and suggest that M3D is in closer agreement with Monte Carlo, thus discrepancies between the TPS and M3D software will be observed for lung SBRT plans. M3D provides a useful supplement to verification of lung SBRT plans by direct measurement, which typically excludes patient specific heterogeneities.PACS number(s): 87.55.D‐, 87.55.Qr, 87.55.K‐

The effectiveness of pretreatment physics plan review for detecting errors in radiation therapy.

The pretreatment physics plan review is a standard tool for ensuring treatment quality. Studies have shown that the majority of errors in radiation oncology originate in treatment planning, which underscores the importance of the pretreatment physics plan review. This quality assurance measure is fundamentally important and central to the safety of patients and the quality of care that they receive. However, little is known about its effectiveness. The purpose of this study was to analyze reported incidents to quantify the effectiveness of the pretreatment physics plan review with the goal of improving it. This study analyzed 522 potentially severe or critical near-miss events within an institutional incident learning system collected over a three-year period. Of these 522 events, 356 originated at a workflow point that was prior to the pretreatment physics plan review. The remaining 166 events originated after the pretreatment physics plan review and were not considered in the study. The applicable 356 events were classified into one of the three categories: (1) events detected by the pretreatment physics plan review, (2) events not detected but "potentially detectable" by the physics review, and (3) events "not detectable" by the physics review. Potentially detectable events were further classified by which specific checks performed during the pretreatment physics plan review detected or could have detected the event. For these events, the associated specific check was also evaluated as to the possibility of automating that check given current data structures. For comparison, a similar analysis was carried out on 81 events from the international SAFRON radiation oncology incident learning system. Of the 356 applicable events from the institutional database, 180/356 (51%) were detected or could have been detected by the pretreatment physics plan review. Of these events, 125 actually passed through the physics review; however, only 38% (47/125) were actually detected at the review. Of the 81 events from the SAFRON database, 66/81 (81%) were potentially detectable by the pretreatment physics plan review. From the institutional database, three specific physics checks were particularly effective at detecting events (combined effectiveness of 38%): verifying the isocenter (39/180), verifying DRRs (17/180), and verifying that the plan matched the prescription (12/180). The most effective checks from the SAFRON database were verifying that the plan matched the prescription (13/66) and verifying the field parameters in the record and verify system against those in the plan (23/66). Software-based plan checking systems, if available, would have potential effectiveness of 29% and 64% at detecting events from the institutional and SAFRON databases, respectively. Pretreatment physics plan review is a key safety measure and can detect a high percentage of errors. However, the majority of errors that potentially could have been detected were not detected in this study, indicating the need to improve the pretreatment physics plan review performance. Suggestions for improvement include the automation of specific physics checks performed during the pretreatment physics plan review and the standardization of the review process.

WE‐DE‐201‐02: A Statistical Analysis Tool for Plan Quality Verification in HDR Brachytherapy Forward Planning for Cervix Cancer

Purpose:High Dose Rate (HDR) brachytherapy forward planning is principally an iterative process; hence, plan quality is affected by planners’ experiences and limited planning time. Thus, this may lead to sporadic errors and inconsistencies in planning. A statistical tool based on previous approved clinical treatment plans would help to maintain the consistency of planning quality and improve the efficiency of second checking.Methods:An independent dose calculation tool was developed from commercial software. Thirty‐three previously approved cervical HDR plans with the same prescription dose (550cGy), applicator type, and treatment protocol were examined, and ICRU defined reference point doses (bladder, vaginal mucosa, rectum, and points A/B) along with dwell times were collected. Dose calculation tool then calculated appropriate range with a 95% confidence interval for each parameter obtained, which would be used as the benchmark for evaluation of those parameters in future HDR treatment plans. Model quality was verified using five randomly selected approved plans from the same dataset.Results:Dose variations appears to be larger at the reference point of bladder and mucosa as compared with rectum. Most reference point doses from verification plans fell between the predicted range, except the doses of two points of rectum and two points of reference position A (owing to rectal anatomical variations &amp; clinical adjustment in prescription points, respectively). Similar results were obtained for tandem and ring dwell times despite relatively larger uncertainties.Conclusion:This statistical tool provides an insight into clinically acceptable range of cervical HDR plans, which could be useful in plan checking and identifying potential planning errors, thus improving the consistency of plan quality.

A quantification of the effectiveness of EPID dosimetry and software-based plan verification systems in detecting incidents in radiotherapy.

Complex treatments in radiation therapy require robust verification in order to prevent errors that can adversely affect the patient. For this purpose, the authors estimate the effectiveness of detecting errors with a "defense in depth" system composed of electronic portal imaging device (EPID) based dosimetry and a software-based system composed of rules-based and Bayesian network verifications. The authors analyzed incidents with a high potential severity score, scored as a 3 or 4 on a 4 point scale, recorded in an in-house voluntary incident reporting system, collected from February 2012 to August 2014. The incidents were categorized into different failure modes. The detectability, defined as the number of incidents that are detectable divided total number of incidents, was calculated for each failure mode. In total, 343 incidents were used in this study. Of the incidents 67% were related to photon external beam therapy (EBRT). The majority of the EBRT incidents were related to patient positioning and only a small number of these could be detected by EPID dosimetry when performed prior to treatment (6%). A large fraction could be detected by in vivo dosimetry performed during the first fraction (74%). Rules-based and Bayesian network verifications were found to be complimentary to EPID dosimetry, able to detect errors related to patient prescriptions and documentation, and errors unrelated to photon EBRT. Combining all of the verification steps together, 91% of all EBRT incidents could be detected. This study shows that the defense in depth system is potentially able to detect a large majority of incidents. The most effective EPID-based dosimetry verification is in vivo measurements during the first fraction and is complemented by rules-based and Bayesian network plan checking.

SU-D-BRD-04: The Impact of Automatic Radiation Therapy Plan Checks in Treatment Planning

Purpose: The physics plan check verifies various aspects of a treatment plan after dosimetrists have finished creating the plan. Some errors in the plan which are caught by the physics check could be caught earlier in the departmental workflow. The purpose of this project was to evaluate a plan checking script that can be run within the treatment planning system (TPS) by the dosimetrists prior to plan approval and export to the record and verify system. Methods: A script was created in the Pinnacle TPS to automatically check 15 aspects of a plan for clinical practice conformity. The script outputs a list of checks which the plan has passed and a list of checks which the plan has failed so that appropriate adjustments can be made. For this study, the script was run on a total of 108 plans: IMRT (46/108), VMAT (35/108) and SBRT (27/108). Results: Of the plans checked by the script, 77/108 (71%) failed at least one of the fifteen checks. IMRT plans resulted in more failed checks (91%) than VMAT (51%) or SBRT (63%), due to the high failure rate of an IMRT-specific check, which checks that no IMRT segment < 5 MU. The dose gridmore » size and couch removal checks caught errors in 10% and 14% of all plans – errors that ultimately may have resulted in harm to the patient. Conclusion: Approximately three-fourths of the plans being examined contain errors that could be caught by dosimetrists running an automated script embedded in the TPS. The results of this study will improve the departmental workflow by cutting down on the number of plans that, due to these types of errors, necessitate re-planning and re-approval of plans, increase dosimetrist and physician workload and, in urgent cases, inconvenience patients by causing treatment delays.« less

TU‐G‐BRD‐01: Quantifying the Effectiveness of the Physics Pre‐Treatment Plan Review for Detecting Errors in Radiation Therapy

Purpose:Physics pre‐treatment plan review is crucial to safe radiation oncology treatments. Studies show that most errors originate in treatment planning, which underscores the importance of physics plan review. As a QA measure the physics review is of fundamental importance and is central to the profession of medical physics. However, little is known about its effectiveness. More hard data are needed. The purpose of this study was to quantify the effectiveness of physics review with the goal of improving it.Methods:This study analyzed 315 “potentially serious” near‐miss incidents within an institutional incident learning system collected over a two‐year period. 139 of these originated prior to physics review and were found at the review or after. Incidents were classified as events that: 1)were detected by physics review, 2)could have been detected (but were not), and 3)could not have been detected. Category 1 and 2 events were classified by which specific check (within physics review) detected or could have detected the event.Results:Of the 139 analyzed events, 73/139 (53%) were detected or could have been detected by the physics review; although, 42/73 (58%) were not actually detected. 45/73 (62%) errors originated in treatment planning, making physics review the first step in the workflow that could detect the error. Two specific physics checks were particularly effective (combined effectiveness of &gt;20%): verifying DRRs (8/73) and verifying isocenter (7/73). Software‐based plan checking systems were evaluated and found to have potential effectiveness of 40%. Given current data structures, software implementations of some tests such as isocenter verification check would be challenging.Conclusion:Physics plan review is a key safety measure and can detect majority of reported events. However, a majority of events that potentially could have been detected were NOT detected in this study, indicating the need to improve the performance of physics review.

TH‐EF‐BRD‐08: A Retrospective Statistical Analysis of Consistencies in Treatment Plan Parameters by the Treatment Site and Modality

Purpose:Simple rules and common senses based on prior knowledge could be powerful tools for medical physicists to identify and prevent severe errors in radiation therapy (RT) treatment plans. However, rules, e.g. the acceptable MU/cGy ratio for different treatment sites, are not yet available quantitatively and therefore cannot be utilized effectively. In this study, we have retrospectively analyzed all the EBRT treatment plans of the past 8 years at authors’ institution. The goal is to better understand the most common treatment plan data statistics, involving MU/cGy ratio, numbers of beams and segments, and ranges of gantry angles for lateral targets, in order to quantitatively define the common sense‐based rules that could be used to guide manual or automatic physics checks.Methods:15844 previous treatment plans were obtained from Mosaiq. Plan data were extensively processed including 1) reducing data size, 2) removing empty and unapproved site and fields, 3) deriving common site names from names of prescriptions and beams that were spelled and misspelled extremely diversely, and 4) removing the plans not covered by this study, e.g. HDR plans. Processed data was then statistically analyzed using multivariate regression algorithms to quantitatively investigate the relationships between the input parameters, i.e. treatment site, technique, modality, and laterality, and the output parameters, i.e. MU/cGy radio, the total number of treatment beams and segments, and the beam gantry angles (only for lateral targets).Results:Mean values and standard deviations of the output parameters were calculated and tabulated per treatment site, technique and modality. Beam gantry angle range and mean gantry angle were analyzed per site for plans treating lateral targets.Conclusion:The results of this study, as the statistics of the major treatment plan parameters, could be useful to quantitatively evaluate the consistency of new treatment plans in either manual or automatic plan checking procedures.

Brachyverifier: An Automated System for Plan Quality Assurance in High-Dose-Rate Brachytherapy

To improve the quality assurance (QA) process of high-dose-rate brachytherapy (HDR) by automating the physics plan check process. BrachyVerifier is an application written in Java for the automation of plan QA in HDR. The program extracts information from the record-and-verify (R&V) data stored in an ARIA database (Varian Medical Systems, Inc., Palo Alto, CA) and from portable document format (PDF) files of (i) the plan printout from the Oncentra Brachytherapy planning system (Nucletron, an Elekta company, Elekta AB, Stockholm, Sweden), (ii) the written directive, and (iii) the catheter-length measurement report. Checks are customized to the type of plan, which is detected from the plan printout. If more than one type of plan is compatible with the plan printout, the user is prompted to choose the plan type. Check speed was assessed through a review of the time elapsed between plan approval and final plan check in the 43 tandem-and-ring fractions that occurred between July 2013 and November 2013. Statistical significance (p < 0.05) was assessed using an unpaired Student T-test. Checks specific to a given type of plan have been developed for the following types: tandem and ring, tandem and ring + interstitial needles, tandem and ovoids, tandem and ovoids + interstitial needles, tandem + interstitial needles, interstitial needles only (gynecologic and non-gynecologic), straight line cylinder, graphical cylinder and surface applicator. Other types of plans are processed under an “other” category, with no type-specific checks. BrachyVerifier was written and tested to perform the following checks:-Secondary calculation: the point doses in the plan printout are compared to the dose resulting from a 1D TG43 calculation. Tolerance is 5%.-Channel assignments: the channel assigned to each catheter is compared to our clinical practice for a given plan type (e.g., tandem to channel 3). Match must be exact.-Catheter lengths: the catheter lengths in the plan printout are compared to either a) the measurement report or b) the lengths that are expected for a given plan type (e.g., 1500 for tandem). Match must be exact.-Written directive compliance: the fractionation scheme, number of catheters and step size indicated in the written directive are compared to the plan printout. Match must be exact.-R&V check: the patient name, fractionation scheme and plan properties (such as TRAK value) in the plan printout are compared to the R&V system. TRAK tolerance is 1%. Three documents are generated: a summary sheet of all the checks being performed and their results, the results of the secondary calculation and a worksheet summarizing the plan information to be checked during treatment. Since its introduction into clinical use in September 2013, BrachyVerifier has been used to check 51 HDR plans. Mean physics check time for tandem and ring cases was 1217 seconds with BrachyVerifier (n = 23) and 1403 seconds without (n = 20; p = 0.045). BrachyVerifier has standardized the physics checks of HDR plans and produced a significant reduction in plan check time. The integration of BrachyVerifier into our clinical practice automated the checks on channel assignments, catheter lengths and prescription, thus reducing the possibility of missing errors that occurred during planning. BrachyVerifier has not eliminated the need for plan checking by a physicist. Moreover, checks such as correct attachment of transfer tubes and other safety-related checks cannot be automated via software. Although the exact implementation of BrachyVerifier is specific to our clinical practices and equipment, the general architecture is portable from one clinic to another.

Radiation oncology whiteboard: Data and workflow manager for enhanced communication and task management.

304 Background: Patient care within the radiation oncology clinic consists of several interdependent data driven procedures which must be executed in concert to ensure the best possible patient care. State of the art treatment in radiation oncology is largely computer driven, where patient care relies on the use of a typically vendor dependent, heterogeneous server environment. Traditional, peer-to-peer communication in such an environment introduces a fault intolerant workflow, where failed communication and data inconsistency issues may result in delayed or inappropriate patient treatment. Methods: The MGH Radiation Oncology Whiteboard is an intranet based application allowing for live monitoring of these patient care tasks available across the entire radiation oncology clinical domain. Patient events are recorded through interaction from various care-givers and data sources within the department, changing state as well as collecting event data from the accomplished task. Personnel may be alerted of state changes through automatic, customizable paging, email and live monitoring notification, triggering subsequent steps in the workflow chain. Currently the Whiteboard leverages data from multiple points in the clinic including: CT Simulation and data (4D), treatment planning written directive, dvh plan results, imrt and sbrt plan QA, daily and monthly linac QA, IORT coordination and calculation check, linear accelerator engineering log, patient treatment wait time (linac), remote treatment plan review. Results: Since introduction of the Whiteboard into the MGH clinic in 2008, task compliance in key areas, such as contour definition, radiation dose prescription, imrt QA measurement completion and pre-treatment plan checking has risen to levels well above 95%. Conclusions: Monitoring of patient care events improves QA by eliminating peer-to-peer communication and introducing complete transparency to the workflow communication chain. This approach enables all members of the staff to become active participants in the quality control process, every individual, through interaction with the Whiteboard, acting as an additional QC filter to improve patient care.