Simulation Testing of Pediatric Rapid Response Teams: Can Simulation Be Used to Determine the Best Team Structure?

Abstract

See related article, p 258 See related article, p 258 Medical simulation and rapid response teams (RRTs) are 2 subjects whose application and effectiveness have been criticized. However, both are increasingly being integrated and expanded into the healthcare environment, including institutions that care for children. Added to this predicament is the constant effort to restructure responsibilities among healthcare providers including within medical response teams. Furthermore, there are no studies using simulation scenarios to compare the performance of RRTs led by intensivists-in-training and nurse practitioners (NPs). In this volume of The Journal, Fehr et al1Fehr J.J. McBride M.E. Boulet J.R. Murray D.J. The simulation-based assessment of pediatric rapid response teams.J Pediatr. 2017; 188 (258–62.e1)Abstract Full Text Full Text PDF PubMed Scopus (3) Google Scholar studies the use of simulation to assess the performance of RRTs. Medical simulation can be used for many functions including personnel training and assessment as well as system evaluations.2Eppich W. Howard V. Vozenilek J. Curran I. Simulation-based team training in healthcare.Simul Healthc. 2011; 6: S14-9Crossref PubMed Scopus (94) Google Scholar Medical simulation is typically used for formative purposes, which emphasize future improvement of the individual, course, and/or system. However, medical simulation can also be used for high-stake, summative testing. In general, limitations of simulation include developing realistic and relevant scenarios, learner and leadership buy-in of the process and resources, and ultimately creating a culture of safety for the learners. A big debate within medical simulation is the challenge of extrapolating data obtained in the simulated environment into the clinical setting. In addition, tools used in medical simulation to evaluate the effectiveness of training typically incorporate 1, if not several, of the 4 levels in the Kirkpatrick Model.3Cox T. Seymour N. Stefanidis D. Moving the needle: simulation's impact on patient outcomes.Surg Clin North Am. 2015; 95: 827-838Abstract Full Text Full Text PDF PubMed Scopus (53) Google Scholar The most common Kirkpatrick levels used in medical simulation include level 1 (reaction) and level 2 (learning). Achievement of level 1 could be obtained by surveying the learner group regarding their reactions to training whereas level 2 could be obtained by assessing the knowledge of each learner by the administration of a post-test. The ideal strategy in the medical simulation community would be to generate evidence at Kirkpatrick level 4 (results). Kirkpatrick level 4 would include demonstrating targeted results (ie, patient outcomes) as a direct result of medical simulation training. Establishing these types of direct outcomes becomes very challenging and complex to substantiate. Because of the previously stated concerns, it is equally very interesting and difficult to study the issue of using medical simulation to determine the best team structure. There has been continued interest in the use of RRTs within the healthcare system. This is despite the evidence being unclear about their effectiveness. The lack of RRT effectiveness can be due to several factors including inconsistent activation criteria, various medical conditions, varying team compositions, and variability in training. Common RRT activation criteria are variable but typically include objective measures (ie, changes in vital signs) as well as subjective concerns from staff and family. The use of a RRT allows prompt patient evaluation by more skilled providers, which may result in reassurance or escalation of care. The widespread acceptance of RRTs has been based on previous evidence that these teams improve clinical outcomes, lower costs, and the Institute for Healthcare Improvement recommendation to implement rapid response systems as a key part of the 100 000 Lives Campaign.4Berwick D.M. Calkins D.R. McCannon C.J. Hackbarth A.D. The 100,000 lives campaign: setting a goal and a deadline for improving health care quality.JAMA. 2006; 295: 324-327Crossref PubMed Scopus (520) Google Scholar However, conflicting reports make RRT adoption challenging such as 1 study5Kotsakis A. Lobos A.T. Parshuram C. Gilleland J. Gaiteiro R. Mohseni-Bod H. et al.Implementation of a multicenter rapid response system in pediatric academic hospitals is effective.Pediatrics. 2011; 128: 72Crossref PubMed Scopus (82) Google Scholar reporting a significant reduction in pediatric intensive care unit (PICU) mortality rate after readmission from a medical or surgical unit, which is in sharp contrast to another report of an observational study6Joffe A.R. Anton N.R. Burkholder S.C. Reduction in hospital mortality over time in a hospital without a pediatric medical emergency team: limitations of before-and-after study designs.Arch Pediatr Adolesc Med. 2011; 165: 419Crossref PubMed Scopus (46) Google Scholar in a children's hospital that did not employ a RRT but still identified a significant reduction in mortality over the equivalent time period in which other pediatric centers decreased mortality in association with RRT implementation. In addition, the most recent American Heart Association's Pediatric Advanced Life Support guidelines7de Caen A.R. Berg M.D. Chameides L. Gooden C.K. Hickey R.W. Scott H.F. et al.Part 12: pediatric advanced life support: 2015 American Heart Association guidelines update for cardiopulmonary resuscitation and emergency cardiovascular care.Circulation. 2015; 132: S526-42Crossref PubMed Scopus (360) Google Scholar report inconclusive evidence for the use of RRTs and recommend that pediatric RRTs “may be considered in facilities where children with high-risk illnesses are cared for on general in-patient units.” Therefore, it would be valuable to determine which RRT configurations are the most effective, which is the basis for the study by Fehr et al.1Fehr J.J. McBride M.E. Boulet J.R. Murray D.J. The simulation-based assessment of pediatric rapid response teams.J Pediatr. 2017; 188 (258–62.e1)Abstract Full Text Full Text PDF PubMed Scopus (3) Google Scholar Fehr et al1Fehr J.J. McBride M.E. Boulet J.R. Murray D.J. The simulation-based assessment of pediatric rapid response teams.J Pediatr. 2017; 188 (258–62.e1)Abstract Full Text Full Text PDF PubMed Scopus (3) Google Scholar evaluated 2 distinct RRT structures to determine if the performance of the group differed among teams led by intensivists-in-training vs NPs. It should be noted that there is limited evidence regarding the use of NPs and physician assistants, particularly for children, within the acute and critical care settings.8Kleinpell R.M. Ely E.W. Grabenkort R. Nurse practitioners and physician assistants in the intensive care unit: an evidence-based review.Crit Care Med. 2008; 36: 2888-2897Crossref PubMed Scopus (199) Google Scholar This is relevant to all institutions because various staffing modeling are being used to provide appropriate personnel to care for our patients effectively despite staffing and financial challenges. The authors developed the simulation scenarios from review of critical events within their institution that triggered actual RRT calls. Ten simulation scenarios were subsequently developed that met several curriculum objectives including a wide age range, diverse medical conditions among cardiac, respiratory, and neurologic categories, and a significant degree of medical complexity. The age range for the simulation patients was from 24 days to 10 years. The conditions covered medical and surgical disorders including asthma exacerbation, supraventricular tachycardia, cardiomyopathy, opioid overdose, respiratory failure, seizure disorder, airway foreign body, septic shock, and increased intracranial pressure. The scenarios were designed to necessitate the use of established algorithmic treatment plans such as Pediatric Advanced Life Support as well as using nonstandardized therapies because of atypical clinical presentations and having incorrect initial diagnostic information provided to the learners. After development by the authors, the scenarios were reviewed and pilot tested by members of their institution's RRT. Scoring rubrics were developed to assess the global performance of the RRT during each scenario. This rubric was specific to each scenario and resulted in an overall global score from 1 (unsatisfactory) to 9 (superior). Subject volunteers were obtained from the authors' institution, which had the following backgrounds: PICU nurses, respiratory therapists, PICU intensivists-in-training, and NPs. These teams compromise the typical RRT at the authors' institution. Each team consisted of 3 members: a PICU nurse, respiratory therapist, and a PICU intensivist-in-training or an NP. All subjects in this study participated in an introductory simulation scenario on asthma, which was not included in the evaluation. The remaining 9 scenarios were divided into sets of 3, which included 1 scenario per set with incomplete information that required additional investigation by the RRT to identify the correct medical condition. These 3 challenging scenarios included undiagnosed or incorrectly diagnosed congestive heart failure, airway foreign body, and coarctation of the aorta. From a curriculum development perspective, I especially value the deliberate use of incorrect information in the scenario to mimic realistic situations encountered within the clinical environment. Six teams were led by an NP, and 11 were led by a PICU intensivist-in-training. The PICU intensivist-in-training was a fifth postgraduate year, 9 of the 11 times; this would typically correspond to a second year PICU fellow. The NPs were currently performing in an NP role for an average of 4.8 years and previously in a nursing role for an average of 15 years. All simulation scenarios occurred within a simulation center, and the sessions were video recorded. Each team completed 2 randomly assigned sets of scenarios (a total of 6 scenarios) plus the introductory session on asthma. These 17 teams were evaluated for a total of 102 study scenarios. Debriefing occurred only after completing the introductory session on asthma and the 6 scenarios. Key action items and a global performance score was assigned by 1 of the authors. The global score for each RRT was based on efficiency as well as adherence of the medical care to each specific scenario checklist. Analysis of the RRT performance used the overall global scores. Both types of RRTs promptly and effectively managed most of the simulated scenarios. The authors report the largest differences among the 2 teams particularly during the increased intracranial pressure ([mean ± SD for intensivist-in-training {7.43 ± 0.79}] vs [mean ± SD for NP {5.00 ± 1.73}]) and coarctation of the aorta ([intensivist-in-training {6.57 ± 1.40}] vs [NP {4.33 ± 1.53}]) scenarios. However, the overall global performance for all 9 scenarios among both groups was similar (intensivist-in-training [7.44 ± 1.25]) vs (NP [6.75 ± 1.46]). The global scores for the 9 scenarios ranged from the lowest for coarctation of the aorta (5.9 ± 1.70) to the highest for supraventricular tachycardia (8.30 ± 0.80). The reported overall reliability of the assessment (r = 0.57) implies that no significant judgements should be made regarding either RRT. The overall global scores made it difficult to determine the specific factor(s), which resulted in the variation among the groups. The authors conclude several limitations of this study while also emphasizing the many benefits from the use of medical simulation for RRT training.9Paige J.T. Garbee D.D. Brown K.M. Using simulation in interprofessional education.Surg Clin North Am. 2015; 95: 751-766Abstract Full Text Full Text PDF PubMed Scopus (34) Google Scholar The relatively low number of learners, the limited amount of simulation scenarios, and the diverse clinical as well as simulation experiences of the team members limits the interpretation of the results. However, significant benefits from this study include another training opportunity for exposure to critical medical conditions using an active learning format as well as enhancing the culture for the use of medical simulation to promote teamwork. In my experience, medical simulation can be used as an additional tool to effectively introduce and start the discussions on how to best train our providers to enhance their future performance. In addition, many of the systematic approaches used to manage these specific clinical dilemmas within the simulated venue can be frequently generalized for use in other situations within the actual clinical environment. Future studies related to this topic could be developed to evaluate other medical conditions to perhaps address additional conditions commonly seen (ie, nonpatients with medical conditions within a pediatric setting such as the parent with a new-onset seizure). Studies could also be developed to assess additional pediatric RRT structures (such as physician assistant vs NP or PICU intensivist-in-training versus pediatric hospitalist). If the main question is if medical simulation can be used as an effective evaluation tool in general, the answer appears to be yes. If the question is modified to ask if medical simulation can be used to determine the best team structure for pediatric RRTs, the answer is less clear. I would like to commend Fehr et al1Fehr J.J. McBride M.E. Boulet J.R. Murray D.J. The simulation-based assessment of pediatric rapid response teams.J Pediatr. 2017; 188 (258–62.e1)Abstract Full Text Full Text PDF PubMed Scopus (3) Google Scholar in expanding the application of medical simulation from the traditional, formative focused training modality for use in the formal evaluation of team framework. I think the overall take home message is that medical simulation and RRTs are both here to stay and only expanding in the opportunities available to enhance our educational activities, shape our institutions, and promote better care to our patients. The Simulation-Based Assessment of Pediatric Rapid Response TeamsThe Journal of PediatricsVol. 188PreviewTo create scenarios of simulated decompensating pediatric patients to train pediatric rapid response teams (RRTs) and to determine whether the scenario scores provide a valid assessment of RRT performance with the hypothesis that RRTs led by intensivists-in-training would be better prepared to manage the scenarios than teams led by nurse practitioners. Full-Text PDF