Assessing the Assessment: Are Senior Summative OSCEs Measuring Advanced Knowledge, Skills, and Attitudes?

Abstract

The authors investigated adaptation of Bloom's and Simpson's taxonomies for the medical (student) setting, and using the adapted taxonomies to determine whether a summative objective structured clinical examination (OSCE) used at their medical school was assessing higher-order knowledge, skills, and attitudes. Two faculty members (including H.M.H.) adapted the taxonomies and used them to categorize (knowledge, skills, or attitudes) and rank (by level within the taxonomies) every item on every OSCE station checklist. Interrater reliability was moderate to high. Although there was a range of domains and levels within and across stations, on average every OSCE station was assessing learning behaviors at a lower level than expectations articulated in the school's goals for medical students' education. The adapted taxonomies were useful for assessing the domains and levels of behaviors measured on the summative OSCE, and they can also be used to modify existing checklists or to create new assessment instruments that meet the expectations articulated in a school's goals for medical students' education.