Evaluation of the Clinical Assessment Project: A Computer-Based Multimedia Tool to Assess Problem-Solving Ability in Medical Students

Abstract

The purpose of this work was to describe Clinical Assessment, a computer-based multimedia patient simulation used to assess the problem-solving abilities of medical students and to evaluate its capacity to guide the assignment of course grade. This was a multisite reviewer-blinded comparison of course grades, National Board of Medical Examiners pediatric examination score, and Clinical Assessment scores at 3 pediatric clerkship sites of the Harvard Medical School. Participants included 470 students completing their pediatric clerkships. Each student's performance on < or = 4 Clinical Assessment patient case simulations was compared with National Board of Medical Examiners pediatric examination scores and course grades assigned by clerkship directors based on overall ward performance. Data from both the National Board of Medical Examiners pediatric "shelf" examination and the course grade were available for 411 students who completed > or = 1 Clinical Assessment case. There was a strong correlation between Clinical Assessment score and course grade when comparing students receiving honors versus satisfactory category course grades. Students who ordered more expensive or greater numbers of laboratory tests did not achieve greater diagnostic accuracy on Clinical Assessment. Clinical Assessment had a high positive predictive value for course grade: 95% of students scoring > or = 90% on Clinical Assessment achieved an honors category course grade. Because nearly all of the students who scored very well on Clinical Assessment received honors category course grades, future high scorers on this examination merit consideration for assigning a high course grade. A computer-based multimedia patient simulation assessment tool provides objective information that can complement a student's National Board of Medical Examiners score and course grade and may assist in evaluating clinical problem-solving ability.