Using Elements from an Acute Abdominal Pain Objective Structured Clinical Examination (OSCE) Leads to More Standardized Grading in the Surgical Clerkship for Third-Year Medical Students

Abstract

There is poor reliability in the Likert-based assessments of patient interaction and general knowledge base for medical students in the surgical clerkship. The Objective Structured Clinical Examination (OSCE) can be used to assess these competencies. We hypothesize that using OSCE performance to replace the current Likert-based patient interaction and general knowledge base assessments will not affect the pass/fail rate for third-year medical students in the surgical clerkship. In this retrospective study, third-year medical student clerkship data from a three-station acute abdominal pain OSCE were collected from the 2009-2010 academic year. New patient interaction and general knowledge base assessments were derived from the performance data and substituted for original assessments to generate new clerkship scores and ordinal grades. Two-sided nonparametric statistics were used for comparative analyses, using an α = 0.05. Seventy third-year medical students (50.0% female) were evaluated. A sign test showed a difference in the original (4.45/5) and the new (4.20/5) median patient interaction scores (p < 0.01). A sign test did not show a difference in the original (4.00/5) and the new (4.11/5) median general knowledge base scores (p = 0.28). Nine clerkship grades changed between these different grading schemes (p = 0.045), with an overall agreement of 87.1% and a kappa statistic of 0.81. There were no differences in the pass/fail rate (p > 0.99). We conclude that there are no differences in pass/fail rate, but there is a more standardized distribution of patient interaction assessments and utilization of the full spectrum of possible passing grades. We recommend that the current patient interaction assessment for third-year medical students in the surgical clerkship be replaced with that found through trained standardized patients in this three-station acute abdominal pain OSCE.