Board #130 - Research Abstract Performance and Cognitive Load Among Novices Training on Simple vs. Complex Simulation Scenarios During Procedural Skills Training

Abstract

Hypothesis A common belief in healthcare simulation is that higher realism results in improved learning.1-2 Predicated on the concept of context dependency (CD),3 proponents of this view advocate for simulations that recreate the contextual realities of clinical practice,4 arguing such environments help learners develop, coordinate and transfer constituent skills from simulation to clinical care.5 However, the evidence supporting this belief is mixed.6 A potential explanation for these disparate findings is that although higher realism may improve transfer through the CD effect, for novices it may also lead to increased task complexity. In turn, higher complexity may increases novices’ cognitive load (CL) to a point that is detrimental for learning, resulting in reduced transfer.7 The purpose of this study was to test these competing hypotheses by examining the effect of scenario complexity and CD on performance and CL during procedural skill acquisition, retention and transfer. Methods Novice medical students (n = 38) were randomly assigned to training on a simple or complex Lumbar Puncture (LP) simulation (Fig 1A and 1B), consisting of four practice trials interspersed with controlled feedback (acquisition phase). After 10 days, novices completed one trial on their training scenario (retention phase). Finally, to test the competing effects of CD vs. high task complexity, novices completed one trial on a very complex hybrid simulation (Fig 1C)8 contextually similar to the complex scenario (transfer phase). On all trials, LP performance was assessed using a global rating scale (GRS)9 and number of sterility breaches, while CL was measured using subjective ratings of mental effort (SRME) and reaction time (RT) to a vibrotactile secondary task.10-11 Acquisition and retention scores were analyzed using mixed 2x4 and 2x2 ANOVA; transfer scores were analyzed using t-tests. Effect sizes are reported as Cohen’s f or d, p<0.05 was considered statistically significant. Results Both groups demonstrated improved LP performance and fewer sterility breaches from the beginning to the end of the acquisition phase (p = 0.001, f = 0.96 for GRS; p = 0.001, f = 0.50 for sterility) and maintained this at retention. Novices in the simple group demonstrated superior LP performance and fewer sterility breaches compared to those in the complex group during acquisition (p = 0.002, d = 1.13 for GRS; p = 0.001, d = 1.60 for sterility) and at retention (p = 0.001, d = 1.25 for GRS; p = 0.001, d = 1.72 for sterility). CL decreased faster for novices in the simple group during acquisition (p = 0.005, f = 0.36 for SRME and p = 0.011, f = 0.35 for RT) and the simple group maintained lower CL at retention (p = 0.001, d = 1.39 for SRME and p = 0.021, d = 0.81 for RT). On transfer to the very complex simulation scenario, no differences in CL or GRS scores were observed between the two groups, however the simple group continued to make fewer sterility breaches (p = 0.023, d = 0.80). Conclusion This study provides empirical data explaining why higher realism may result in equivocal transfer among novice learners. Lower task complexity was associated with improved procedural performance during skill acquisition and faster decline in CL, which was retained after a 1-week delay. However, with the exception of fewer sterility breaches, the benefits of training on a simpler scenario were eliminated upon transfer to a very complex hybrid simulation. The equivalent performance of both groups raises the possibility that both CD and CL can impact transfer of procedural skills during simulation-based training. These competing effects should be balanced by simulation instructional designers, for instance by using a progressive sequence of training that reduces cognitive load in the early phases of learning and exposes the learner to more complex scenarios later in training.[5]