Measuring the impact of simulation debriefing on the practices of interprofessional trauma teams using natural language processing

Abstract

BackgroundNatural language processing (NLP) may be a tool for automating trauma teamwork assessment in simulated scenarios. MethodsUsing the Trauma Nontechnical Skills Assessment (T-NOTECHS), raters assessed video recordings of trauma teams in simulated pre-debrief (Sim1) and post-debrief (Sim2) trauma resuscitations. We developed codes through directed content analysis and created algorithms capturing teamwork-related discourse through NLP. Using a within subjects pre-post design (n = 150), we compared changes in teams' Sim1 versus Sim2 T-NOTECHS scores and automatically coded discourse to identify which NLP algorithms could identify skills assessed by the T-NOTECHS. ResultsAutomatically coded behaviors revealed significant post-debrief increases in teams' simulation discourse: Verbalizing Findings, Acknowledging Communication, Directed Communication, Directing Assessment and Role Assignment, and Leader as Hub for Information. ConclusionsOur results suggest NLP can capture changes in trauma team discourse. These findings have implications for the expedition of team assessment and innovations in real-time feedback when paired with speech-to-text technology.