Student Attainment of Proficiency in a Clinical Skill: The Assessment of Individual Learning Curves

Robert D Campbell,Daniel S J Pang,David J Biau,Kent G Hecker

doi:10.1371/journal.pone.0088526

Abstract

The aims of this study were to determine if the learning curve cumulative summation test (LC-CUSUM) can differentiate proficiency in placing intravenous catheters by novice learners, and identify the cause of failure when it occurred. In a prospective, observational study design 6 undergraduate students with no previous experience of placing intravenous catheters received standardized training by a board certified veterinary anesthesiologist in intravenous catheter placement technique. Immediately following training, each student attempted 60 intravenous catheterizations in a dog mannequin thoracic limb model. Results were scored as a success or failure based upon completion of four specific criteria, and where catheter placement failure occurred, the cause was recorded according to pre-defined criteria. Initial acceptable and unacceptable failure rates were set by the study team and the LC-CUSUM was used to generate a learning curve for each student. Using 10% and 25% acceptable and unacceptable failure rates, 3 out of 6 students attained proficiency, requiring between 26 to 48 attempts. Applying 25% and 50% acceptable and unacceptable failure rates, 5 of 6 students obtained proficiency, requiring between 18 and 55 attempts. Wide inter-individual variability was observed and the majority of failed catheterisation attempts were limited to two of the four pre-defined criteria. These data indicate that the LC-CUSUM can be used to generate individual learning curves, inter-individual variability in catheter placement ability is wide, and that specific steps in catheter placement are responsible for the majority of failures. These findings may have profound implications for how we teach and assess technical skills.

Highlights

Attainment of proficiency in a technical skill, such as intravenous (IV) catheterization, is often based on an arbitrary distinction such as experience, or instructor observation of one or more successful attempts
Statistical process control charts, originally designed for use in the manufacturing industry to assess production processes have recently been adapted to quantitatively assess attainment and maintenance of proficiency in medicine, in surgery and anesthesia. [1,6,7] The cumulative summation (CUSUM) method tests the hypothesis that a process is deviating from a target of adequate performance, or that a process remains within an acceptable limit
Clinical proficiency has historically been determined by utilizing subjective criteria; for example, after an arbitrary number of procedural attempts, or after a certain amount of time has passed with assumed proficiency (Rush et al, 2011)

Summary

Introduction

Attainment of proficiency in a technical skill, such as intravenous (IV) catheterization, is often based on an arbitrary distinction such as experience, or instructor observation of one or more successful attempts. [8] If a process is deemed to be ‘out of control’, an intervention takes place, such as a period of re-training The limitation of this technique during an initial period of learning is that the learner will be frequently assessed as out of control due to poor performance, triggering an unacceptable failure rate. Biau and Porcher modified the CUSUM test to provide monitoring and assessment of when a process reaches an in control state, creating the LC-CUSUM (the cumulative summation test for learning curve). This approach considers that a process is out of control initially (as a result of initial trainee performance), and indicates when the process is in control (adequate trainee performance achieved). This approach considers that a process is out of control initially (as a result of initial trainee performance), and indicates when the process is in control (adequate trainee performance achieved). [9]

Objectives

Methods

Results

Conclusion