Vignette element analysis for automated generation of vignettes in pharmacy education

Abstract

Objectives: The aim of this study is to analyze and describe vignette elements and structure, and resulting difficulty, as a foundation for vignette construction in medical education, and ultimately automated vignette generation. Methods: Sixty-three vignettes representing a variety of disease states were sourced from the Objective Structured Clinical Experiences, published practice literature focused on pharmacy, and other training and assessment environments within the school. Three coders independently coded each vignette to identify underlying elements and structure. A consensus-building process was used by the coders to discuss and reconcile coding differences. Results: The coding process resulted in 36 vignette elements. The most common elements were age (n = 59, 93.6%), gender (n = 57, 90.5%), and medications (n = 54, 85.7%); others included race, medications, and chief complaint. Vignette structures and wording were found to be highly variable, with elements present in different magnitudes (range: 4–18 elements), being used with different descriptors, and given in different sequences. Conclusions: Vignette construction could benefit from further understanding of vignette structures and wording and their influence on the level of difficulty. This undertaking will allow educators to construct better vignettes for teaching and assessment to ensure that student performance accurately represents student knowledge and skills, rather than construct irrelevant variance due to vignette-level inconsistencies in content or structure. The defined elements and structure will also enable a systematic generation of vignettes for further consistency in teaching and assessment.