Comparison of Resident Self-Evaluation to Standardized Patient Evaluators in a Multi-Institutional Objective Structured Clinical Examination: Objectively Measuring Residents' Communication and Counseling Skills.

Abstract

For the past 30 years, there has been a growing emphasis on communication and self-evaluation skills training in graduate medical education. This is reflected in the Next Accreditation System. The Objective Structured Clinical Examination (OSCE) is widely used in graduate medical education for assessing dimensions of interpersonal communication and counseling skills. The OSCEs may be developed to target challenging clinical scenarios difficult to capture in clinical practice and can be used as a medium for resident self-evaluation. The aims of the study were to evaluate residents' interpersonal, communication, and counseling skills using Kalamazoo Essential Elements Communication Checklist in 4 clinically challenging scenarios and to compare standardized patient (SP) evaluations to residents' self-evaluation by category of medical school. South East Michigan Center for Medical Education is a consortium of teaching hospitals. Member residents participate in 4 OSCEs as part of their postgraduate 1 curriculum. The OSCEs were developed to evaluate clinically relevant but difficult to capture scenarios including: (a) error disclosure/counseling an angry patient; (b) delivering bad news/end of life; (c) domestic violence; and (d) counseling a patient with colon cancer requesting alternative treatments. At the conclusion of each OSCE, SPs evaluated and residents self-evaluated their performance. Once evaluations were completed, SPs provided residents with feedback. Six member institutions and 344 residents participated during the 2014, 2015, and 2016 academic years. There were more international medical graduates (59%) than graduates of Liaison Committee for Medical Education-accredited medical schools. There were more males (62.2%) than females. Standardized patients rated residents higher than residents rated themselves in 2014 (<0.001), but not in 2015 or 2016. When combining all years and all residents, there was no correlation of SP and resident scores. Standardized patients rated female residents higher than female residents rated themselves (P < 0.0001). Male residents scored themselves similarly to the SPs, but male residents rated themselves higher than female residents rated themselves (P < 0.001). Standardized patient scores for male and female residents were not significantly different. Targeted OSCEs provide an objective format to evaluate residents in challenging clinical scenarios. Resident self-evaluations did not correlate with SPs. In addition, female residents rated themselves lower than male residents and lower than SPs. There is need to develop interventions and curricula to improve resident's self-evaluation skills and in particular address lower self-evaluation by female trainees.