Evaluation of the assessment and grading of medical students on a neurology clerkship

Abstract

To describe a clinical encounter (Bedside Examination Exercise [BEE]) used for assessment and teaching in the Massachusetts General Hospital neurology clerkship; to compare results of the BEE with the Harvard Medical School Subjective Evaluation Form (SEF) and National Board of Medical Examiners Shelf examination (Shelf); and to develop a grading system that assesses multiple skills and reflects proficiency. The BEE was administered to 409 students. Final grades were compared with those of 71 students evaluated with the SEF alone. We compared results on the SEF, BEE, and Shelf examination in another 132 students. A composite score was developed, weighted SEF 70%, BEE 15%, and Shelf 15%, to derive the final grade. The BEE helped limit grade inflation, but did not predict final grade determined by the SEF. Grades from the three test instruments had normal distributions, but different means and SDs: SEF 84% +/- 10.3%; BEE 83% +/- 9.3%; Shelf 69% +/- 8.4%. There was poor agreement among individual students between different tests, even within core competencies. The 70-15-15 composite score had a normal distribution, 81% +/- 8.5%. The Bedside Examination Exercise (BEE) is useful for assessing and teaching clinical skills. No single test instrument predicts results of another with acceptable accuracy. Use of complimentary assessment tools (BEE, Subjective Evaluation Form, and Shelf) lessens uncertainty in deriving the composite score, and facilitates evaluation of different attributes. The composite score enables a five-tier grading system that recognizes proficiency, rewards excellence, and provides meaningful feedback. This approach could be generalized to other clerkships.