Identification-Based Multiple-Choice Assessments in Anatomy can be as Reliable and Challenging as Their Free-Response Equivalents.

Abstract

Multiple-choice (MC) anatomy "spot-tests" (identification-based assessments on tagged cadaveric specimens) offer a practical alternative to traditional free-response (FR) spot-tests. Conversion of the two spot-tests in an upper limb musculoskeletal anatomy unit of study from FR to a novel MC format, where one of five tagged structures on a specimen was the answer to each question, provided a unique opportunity to assess the comparative validity and reliability of FR- and MC-formatted spot-tests and the impact on student performance following the change of test format to MC. Three successive year cohorts of health science students (n=1,442) were each assessed by spot-tests formatted as FR (first cohort) or MC (following two cohorts). Comparative question difficulty was assessed independently by three examiners. There were more higher-order cognitive skill questions and more of the course objectives tested in the MC-formatted tests. Spot-test reliability was maintained with Cronbach's alpha reliability coefficients≥0.80 and 80% of the MC items of high quality (having point-biserial correlation coefficients>0.25). These results also demonstrated guessing was not an issue. The mean final score for the MC-formatted cohorts increased by 4.9%, but did not change for the final theory examination that was common to all three cohorts. Subgroup analysis revealed that the greatest change in spot-test marks was for the lower-performing students. In conclusion, our results indicate spot-tests formatted as MC are suitable alternatives to FR tests. The increase in mean scores for the MC-formatted spot-tests was attributed to the lower demand of the MC format.