A Prospective, Multi-Institutional Study of Problematic Plan Detection during Physician Chart Rounds.

Abstract

We performed a multi-institutional prospective study to determine the detection rate of problematic treatment plans (PP) at physician chart rounds (CR), and to identify factors associated with PP detection. Curative intent PPs with simulated errors (representative of the most common targets of peer review) were generated. Two breast specialists selected twenty appropriate plans for inclusion and assigned them American Association of Physicists in Medicine (AAPM) Task Group 100 severity and detectability scores. The PPs were blinded and embedded at weekly virtual CR at 2 institutions over 12 months. At site A, both breast and lung cases were reviewed by a mix of breast and lung specialists during CR, and at site B, only breast cases were presented and reviewed by breast specialists. At both sites, breast plans were reviewed via slice-by-slice review in the treatment planning system (TPS), and both used a color-coded tool from the TPS to assess adherence to planning directives. Both sites had systematic approaches to case presentation (without a checklist). Site A was usually prospective CR, while site B was exclusively prospective. The following CR elements were recorded: PP detection, time of detection, length of CR, total number of cases presented, plan elements displayed, number and roles of attendees, and detector's role. Analysis was performed using simple statistics with chi-square testing. By PP error type classification, 55.0% pertained to "target volume delineation," 25% to "non-target volume delineation or normal tissue sparing," and 20.0% to "dose prescription or written directives." Detectability was rated ≤5 (<5% likelihood of going undetected) for 60% of PPs, and severity was rated ≥7 ("at least potentially serious toxicity or tumor underdose") for only 30% of PPs. CR lasted a median of 64 minutes at site A (IQR 55-82.5) and 70 minutes at site B (IQR 52.5-81.5). PPs were presented at a median of 34 minutes (IQR 22.5-43, site A) and 41.5 minutes (IQR 23.5-56, site B) after CR start. A median of 16 cases (IQR 13-19) at site A and 32 cases (IQR 25-34.5) at site B were presented per CR session, with a median of 1 PP (site A and B) presented per session (range 1-2). The median time spent per case was 4.0 minutes (Site A) and 2.2 minutes (Site B). The median number of attendings at CR was 4 for site A (range 2-6) and 6.5 for Site B (range 5-10). PP detection rate at site A was 20% (n = 4) and at site B was 70% (n = 14) (p = 0.001). Detections were made by an attending physician in 100% (site A, n = 4) and 92.9% (Site B, n = 13) of PP detections. There were no differences in detection rate by PP error type (p = 0.78), detectability (p = 0.60) or severity score (p = 0.68), or by time PP presented after CR start (p = 0.39). The effectiveness of PP detection at chart rounds can vary greatly between institutions. The study suggests possible areas for improvement but further study is needed to determine best practices.