Effect of an integrated continuing medical education (CME) and quality improvement (QI) program on radiation oncologist (RO) clinical practice

Abstract

There has been little RO-specific research in CME or QI program efficacy. TCI Singapore previously evaluated a chart audit with feedback (C-AWF) CME program in 2002. This prior program significantly improved RO behavior including documentation and communication, although had no impact on the already highly scoring RO performance. The CME program was subsequently modified to incorporate additional audit criteria, and complemented by a QI reminder system, simulation review (SR)-AWF and focused CME topic reviews. Our aim was to evaluate the refined CME/QI program for changes in RO behavior, performance, and adherence to departmental protocols/studies over the first 12 months of the program (April 2003-March 2004). 1. The CME/QI program a) C-AWF: Each fortnight, residents randomly audited 1 case per RO using departmental case notes, imaging, simulation/verification films and plans. Audit criteria were based on a validated instrument, and entered prospectively into a database. Behavior items related to documentation, QA and communication aspects, whilst performance related to management adequacy, including adherence to department protocols and studies. Behavior and performance scoring was presented and ranked by RO during a subsequent C-AWF meeting, with comprehensive discussion of each case among peers, incorporating individualized, enabling and educational feedback. b) SR-AWF: The fortnightly SR-AWF meeting rapidly evaluated patient management for all patients simulated during the prior week, via review of treatment charts, simulation/verification films and plans. Performance items alone were scored and recorded during the meeting, and educational/enabling feedback given. c) QI reminders (attached to “advice to simulator” sheets and treatment cards) targeting RO behavior and protocol/study adherence were mandated for completion prior to every TCI patient commencing radiotherapy. d) CME talks were targeted weekly based on self- and program-identified learning needs. 2. Evaluation RO behavior, performance and protocol/study adherence were evaluated by comparing the prospectively recorded audit results of patients reviewed in C-AWF and SR-AWF meetings during the first and second 6 months of the program. Actions generated from the AWF were also prospectively recorded and evaluated. 36 and 39 charts were evaluated during the first and second 6 months of the C-AWF, and 99 and 79 for the SR-AWF. Of patients audited, 29% had breast, 21% lung, 8% nasopharyngeal and 5% colorectal cancer. 58% were treated radically. Mean behavior significantly increased (12.7 to 13.6 out of 14, p = 0.0005). In addition, RO performance also improved over the same period (7.6 to 7.9 out of 8, p = 0.018). Protocol/study adherence significantly improved from 90.3% to 96.6% (p = 0.005). A total of 50 actions were generated in the form of targeted CME talks for knowledge or skills identified as deficient (14 actions), requirement for protocol refinement (11), systematic changes to RO practice (10), and remediation of deficient management of individual patients reviewed at the meetings (7, representing 3.0% of all patients reviewed). An integrated CME/QI program combining C-AWF, SR-AWF, QI reminders and targeted CME tutorials effectively improved targeted RO behavior and performance over a 12 month period. There was a corresponding increase in departmental protocol and study adherence. The program had additional benefits including the identification of learning needs to direct CME tutorials, the systematic change of suboptimal RO practice, and the alteration of deficient management of 3% of patients audited during the program