Assessing Advanced Communication Skills via Objective Structured Clinical Examination: A Comparison of Faculty Versus Self, Peer, and Standardized Patient Assessors

Abstract

Construct: The construct addressed in this study is assessment of advanced communication skills among senior medical students. Background: The question of who should assess participants during objective structured clinical examinations (OSCEs) has been debated, and options discussed in the literature have included peer, self, standardized patient, and faculty assessment models. What is not known is whether same-level peer assisted learning can be utilized for formative assessment of advanced communication skills when no faculty, standardized patients, or other trained assessors are involved in providing feedback. If successful, such an educational model would optimize resource utilization and broaden the scope of topics that could be covered in formative OSCEs. Approach: The investigators developed a 4-station formative OSCE focused on advanced communication skills for senior medical students, and evaluated the concordance of assessment done by same-level peers, self, standardized patients, and faculty for 45 students. After each station, examinees completed a self-assessment checklist and received checklist-based assessment and verbal feedback from same-level peers only. Standardized patients completed checklist-based assessments outside the room, and faculty did so after the OSCE via video review; neither group provided direct feedback to examinees. The investigators assessed inter-rater agreement and mean difference scores on the checklists using faculty score as the gold standard. Findings: There was fair to good overall agreement among self, same-level peer, standardized patient, and faculty-assessment of advanced communication skills. Relative to faculty, peer and standardized patient assessors overestimated advanced communication skills, while self-assessments underestimated skills. Conclusions: Self and same-level peer-assessment may be a viable alternative to faculty assessment for a formative OSCE on advanced communication skills for senior medical students.