Evolutions in clinical reasoning assessment: The Evolving Script Concordance Test

Abstract

Introduction: Script concordance testing (SCT) is a method of assessment of clinical reasoning. We developed a new type of SCT case design, the evolving SCT (E-SCT), whereby the patient’s clinical story is “evolving” and with thoughtful integration of new information at each stage, decisions related to clinical decision-making become increasingly clear.Objectives: We aimed to: (1) determine whether an E-SCT could differentiate clinical reasoning ability among junior residents (JR), senior residents (SR), and pediatricians, (2) evaluate the reliability of an E-SCT, and (3) obtain qualitative feedback from participants to help inform the potential acceptability of the E-SCT.Methods: A 12-case E-SCT, embedded within a 24-case pediatric SCT (PaedSCT), was administered to 91 pediatric residents (JR: n = 50; SR: n = 41). A total of 21 pediatricians served on the panel of experts (POE). A one-way analysis of variance (ANOVA) was conducted across the levels of experience. Participants’ feedback on the E-SCT was obtained with a post-test survey and analyzed using two methods: percentage preference and thematic analysis.Results: Statistical differences existed across levels of training: F = 19.31 (df = 2); p < 0.001. The POE scored higher than SR (mean difference = 10.34; p < 0.001) and JR (mean difference = 16.00; p < 0.001). SR scored higher than JR (mean difference = 5.66; p < 0.001). Reliability (Cronbach’s α) was 0.83. Participants found the E-SCT engaging, easy to follow and true to the daily clinical decision-making process.Conclusions: The E-SCT demonstrated very good reliability and was effective in distinguishing clinical reasoning ability across three levels of experience. Participants found the E-SCT engaging and representative of real-life clinical reasoning and decision-making processes. We suggest that further refinement and utilization of the evolving style case will enhance SCT as a robust, engaging, and relevant method for the assessment of clinical reasoning.