Performance statistics of structures: does the anatomical region or structure type impact item difficulty or discrimination on gross anatomy practical examinations?

Abstract

Student examination performance is likely influenced by a number of factors, including (but not limited to) student learning, student approaches to studying, teacher emphasis, and clarity of objectives. Gross anatomy practical examinations are somewhat unique in that assessment has traditionally been achieved through free‐response identification questions. Variation in student examination performance on free‐response items, both within and between anatomical regions, is often accounted for by the casual observation that certain regions and/or structures are more “difficult” than others, but that explanation can be subjective and is of limited use from a pedagogical standpoint. The goal of this study was to directly assess the influence that anatomical region and structure type have on gross anatomy practical examination performance on free‐response items by coding and analyzing several years of question item data (i.e. difficulty and point biserial). This information can help with quality improvement of gross anatomy laboratory teaching and assessment.Five years of examination data (N>500 items) were categorized by structure type (e.g. vessel, muscle, bone, nerve, organ, etc.) and anatomical region. Within‐structure patterns were also explored, such as branching level and physical size. Difficulty and point bi‐serial correlations (PBS) were calculated for each item and then compared across categories with Wilcoxon tests. Results indicate that, overall, vessels, nerves, and ligaments tend to be the most difficult structure types, while bones, muscles, and organs are the easiest. Evaluation of vessel branching revealed that main vessels are typically easier than subsequent branches (various p<0.05). The general size of a nerve did not have a significant influence on difficulty. These patterns were generally consistent within each anatomical region, although vessels in the head and neck region are significantly more difficult than nerves (p=0.0176). Limited differences in PBS were exhibited. These results provide insight into several factors that influence examination performance and are useful for improving assessments in the gross anatomy laboratory.