Board 166 - Program Innovations AbstractA Utilization Focused Evaluation of Simulation within the Emergency Triage Assessment and Treatment (ETAT) Program in Malawi (Submission #969)

Abstract

Introduction/Background Malawi has among the highest pediatric mortality rates in the world, exceeding 120 deaths/1000 admissions in some hospitals.1,2 Fifty to eighty two percent of these deaths occur within 48 hours of admission2,3 and many are attributable to deficiencies in the care received by critically ill children, which may in part be due to inadequate health worker training.2 Recent introduction of educational programs, such as Emergency Triage Assessment and Treatment (ETAT) have reduced mortality by 10% at some centers.3 As ETAT incorporates elements of simulation, national interest in developing simulation training capacity has grown. At the request of the Malawi Ministry of Health (MMoH), members of the International Pediatric Simulation Society (IPSS) have evaluated ETAT, to delineate strengths and weaknesses in simulation pedagogy within the program, and identify opportunities and threats to the development of simulation-based education in the country. Methods An eight person multidisciplinary team of simulation experts from IPSS travelled to Malawi in May 2013 to conduct the evaluation. A utilization focused evaluation framework known as the Context, Input, Process, Product (CIPP)4 model was adopted to guide the process. For each CIPP element, multiple data sources were collected, including field notes and interviews with stakeholders completed during site visits to the MMoH, central and district hospitals, rural healthcare centers and both medical and nursing training colleges; direct observations of an ETAT course; and follow up interviews with faculty and participants. Borrowing on the SWOT (strengths, weaknesses, opportunities and threats) matrix,5 data were organized as drivers (strengths and opportunities) or barriers (weaknesses and threats). Our evaluation revealed that although simulation is incorporated as an educational tool within ETAT, it may be significantly underutilized. Evaluation of context identified primary drivers to be buy in from the MMoH for national scale-up of ETAT and support from faculty for revising the curriculum to align with simulation best practices. Barriers included high patient volumes and staff shortages, limiting time for faculty and participants to attend ETAT training. However, this was also identified as an opportunity to incorporate in-situ simulation into ETAT. The evaluation of input identified access to simulation materials (e.g. mannequins, animal models and patients for ‘clinical practice’) to meet educational needs as a driver. Conversely, the increasing number of trainees and limited number of trainers were identified as barriers. Drivers identified during process evaluation included passionate faculty keenly interested in developing their simulation skills, opportunities for interprofessional education and team training (given ETAT is delivered in an interdisciplinary fashion) and dedicated moments for simulation training within the course. Barriers included lack of faculty training in simulation pedagogy, resulting in limited scenario based training, no debriefing and failure to facilitate deliberate practice.6 Finally, product evaluation revealed that participants perceived ETAT training significantly improved their skills. However, severe clinical resource shortages, resulting in a mismatch between what participants are taught and what they can deliver was identified as a significant barrier to subsequent improvement in pediatric outcomes. Results: Conclusion Recent evidence suggests training health workers through educational programs incorporating simulation significantly impacts pediatric mortality, supporting arguments for capacity development of simulation in Malawi. Our evaluation reveals faculty development and enhancement of simulation pedagogy within ETAT are the most pressing needs in this regard. This may be facilitated through a ‘train the trainers’ program focused on best practices in simulation.7 We are currently developing such a program, with anticipated rollout in 2014. Subsequent evaluation of its impact on the delivery and effectiveness of future ETAT courses is planned. Once a highly trained cadre of simulation educators has been established, development of programs beyond ETAT (e.g. in-situ simulation in healthcare facilities) may be explored. However, in such low resource settings, educational content must be appropriately matched to the realities of clinical practice.