P138: Using electronic health record data to assess emergency medicine trainees independent and interdependent performance: a qualitative perspective on measuring what matters

Abstract

Introduction: Competency-based medical education (CBME) affirms that trainees will receive timely assessments and effective feedback about their clinical performance, which has inevitably raised concerns about assessment burden. Therefore, we need ways of generating assessments that do not rely exclusively on faculty-produced reports. The main object of this research is to investigate how data already collected in the electronic health record (EHR) might be meaningfully and appropriately used for assessing emergency medicine (EM) trainees independent and interdependent clinical performance. This study represents the first step in exploring what EHR data might be utilized to monitor and assess trainees clinical performance Methods: Following constructivist grounded theory, individual semi-structured interviews were conducted with 10 EM faculty and 11 EM trainees, across all postgraduate years, to identify EHR performance indicators that represent EM trainees independent and interdependent clinical actions and decisions. Participants were presented with a list of performance indicators and asked to comment on how valuable each would be in assessing trainee performance. Data analysis employed constant comparative inductive methods and occured throughout data collection. Results: Participants created, refined, and eliminated performance indicators. Our main result is a catalogue of clinical performance indicators, described by our participants, as reflecting independent and/or interdependent EM trainee performance that are believed to be captured within the EHR. Such independent indicators include: number of patients seen (according to CTAS levels), turnaround time between when a patient is signed up for and first orders are made, number of narcotics prescribed. Meanwhile, interdependent indicators include, but are not limited to, length of stay, bounce-back rates, ordering practices, and time to fluids. Conclusion: Our findings document a process for developing EM trainee report cards that incorporate the perspectives of clinical faculty and trainees. Our work has important implications for distinguishing between independent and interdependent clinical performance indicators.