The Use of “War Games” to Evaluate Performance of Students and Residents in Basic Clinical Scenarios: A Disturbing Analysis

Abstract

"Failure to Rescue" is a term applied to clinical issues that, if unrecognized or improperly treated, lead to adverse outcomes. We examined the cognitive components of rescue through the use of a "War Games" simulator format. Our hypothesis was that junior and senior medical students would be less able than interns and residents to detail the actions needed to assess, intervene, and stabilize patients. Medical students and residents rotating on the trauma and surgical intensive care unit service participated. Twelve scenarios were created to focus on basic floor emergencies. Scores were assigned for clinical actions ordered. The scenarios were validated by two critical care attending physicians, and these scores were used as the expert group. Scores were assigned by two examiners, and the average of the grades in each area was used. The scores are a ratio of actual to possible correct responses in each section, and in the entire exercise. Subjects were divided into third-year medical students (MS3), fourth-year students (MS4), first-year residents (PGY1), residents beyond their first year (PGY2+), and experts. There were 20 subjects and 5 experts (n = 85) in each group for a total of 140 simulated cases examined. On initial evaluation, MS4 and PGY2+ performed significantly worse than expert, and MS3 and PGY1 performed similarly to experts. On secondary evaluation, all groups performed significantly worse than the expert group. In determining the diagnosis, only MS3 differed significantly from the experts. On follow-up, and in total score, all performed significantly worse than the experts. All groups had significant deficits in cognitive performance compared with experts in the areas of secondary evaluation, follow-up of the presenting problem, and total performance in simple clinical scenarios. We must design educational systems that rapidly enhance the cognitive performance of students and residents before they are left to independently diagnose and intervene in life-threatening clinical situations.