A randomized, controlled crossover study to discern the value of 360-degree versus traditional, faculty-only evaluation for performance improvement of anesthesiology residents

Abstract

360-degree evaluation, or multi-rater feedback, is a means of providing evaluation from a variety of stakeholders. The Accreditation Council for Graduate Medical Education (ACGME) lists 360-degree feedback as a recommended method for evaluating residents. Our study determines if 360-degree evaluation, as compared to traditional evaluation, affords anesthesiology residents greater potential for performance improvement. After IRB exemption and resident consent to participate, sixteen anesthesiology residents of various training levels at The George Washington University Medical Center were randomly assigned to receive either 360-degree evaluation or traditional evaluation. Mid-way through the study, the groups were crossed over. Three faculty members blinded to which type of evaluation each resident received evaluated all residents pre-study, midway, and at the conclusion of the study according the six core competencies set forth by the ACGME. The 360-degree study included evaluations by faculty (traditional), self, medical students, nursing staff and patients. Performance improvement in all core competencies demonstrated a trend toward greater scoring for residents who received early exposure to 360-degree evaluation compared to later exposure. Paired t-Tests demonstrated significance for resident performance improvement with early 360-degree evaluation versus early traditional evaluation for the Interpersonal and Communication Skills core competency over the entire study. Systems-based Practice and Practice-Based Learning and Improvement suggested improvements, but failed to reach statistical significance (P = 0.09, 0.07 respectively). Does a 360-degree evaluation, as compared to traditional, faculty-only evaluation afford anesthesiology residents greater potential for performance improvement? Sixteen residents were exposed to a prospective, single-blinded, crossover design study to determine the answer. For 360-degree evaluations, residents were rated according to the ACGME core competencies after adjusting to Program Director review of evaluations from self, faculty, nursing staff, medical students and patients.