Assessment In Emergency Medicine Research Articles

Current use of simulation for EPA assessment in emergency medicine.

Approximately five years ago, the Royal College emergency medicine programs in Canada implemented a competency-based paradigm and introduced the use of Entrustable Professional Activities (EPAs) for assessment of units of professional activity to assess trainees. Many competency-based medical education (CBME) based curricula, involve assessing for entrustment through observations of EPAs. While EPAs are frequently assessed in clinical settings, simulation is also used. This study aimed to characterize the use of simulation for EPA assessment. A study interview guide was jointly developed by all study authors and followed best practices for survey development. A national interview was conducted with program directors or assistant program directors across all the Royal College emergency medicine programs across Canada. Interviews were conducted over Microsoft Teams, interviews were recorded and transcribed, using Microsoft Teams transcribing service. Sample transcripts were analyzed for theme development. Themes were then reviewed by co-authors to ensure they were representative of the participants' views. A 64.7% response rate was achieved. Simulation has been widely adopted by EM training programs. All interviewees demonstrated support for the use of simulation for EPA assessment for many reasons, however, PDs acknowledged limitations and thematic analysis revealed certain themes and tensions for using simulation for EPA assessment. Thematic analysis revealed six major themes: widespread support for the use of simulation for EPA assessment, concerns regarding the potential for EPA assessment to become a "tick- box" exercise, logistical barriers limiting the use of simulation for EPA assessment, varied perceptions about the authenticity of using simulation for EPA assessment, the potential for simulation for EPA assessment to compromise learner psychological safety, and suggestions for the optimization of use of simulation for EPA assessment. Our findings offer insight for other programs and specialties on how simulation for EPA assessment can best be utilized. Programs should use these findings when considering using simulation for EPA assessment.

Association of preclinical blood glucose with hospitalization rate and in-hospital mortality: A single-center retrospective cohort study.

Critical illness is often accompanied by elevated blood glucose, which generally correlates with increased morbidity and mortality. Prehospital blood glucose (PBG) level might be a useful and easy-to-perform tool for risk assessment in emergency medicine. This retrospective single-center cohort study was designed to analyze the association of prehospital glucose measurements with hospitalization rate and in-hospital mortality. Records of 970 patients admitted to a university hospital by an emergency physician were analyzed. Patients with a PBG ≥140mg/dL (G1, n=394, equal to 7.8mmol/L) were compared with patients with a PBG<140mg/dL (G2, n=576). Multivariable logistic regression models were used to correct for age, prediagnosed diabetes, and sex. Five hundred thirty-four patients (55%) were hospitalized. In comparison to normoglycemic patients, hyperglycemic patients were more likely to be hospitalized with an adjusted odds ratio (OR) of 1.48 (95% confidence interval [CI] 1.11-1.97), more likely to be admitted to the intensive care unit (ICU) with an adjusted OR of 1.74 (95% CI 1.31-2.31) and more likely to die in the hospital with an adjusted OR of 1.84 (95% CI 0.96-3.53). Hospitalized hyperglycemic patients had a median length of stay of 6.0 days (interquartile range [IQR] 8.0) compared to 3.0 days (IQR 6.0) in the normoglycemic group (P<0.001). In the subgroup analysis of cases without known diabetes, patients with PBG ≥140mg/dL were more likely to be hospitalized with an adjusted OR of 1.49 (95% CI 1.10-2.03) and more likely to be admitted to ICU/intermediate care with an adjusted OR of 1.80 (95% CI 1.32-2.45), compared to normoglycemic patients. Elevated PBG ≥140mg/dL was associated with a higher hospitalization risk, a longer length of stay, and a higher mortality risk and may therefore be included in risk assessment scores.

Implementation of the SIMPL (Society for Improving Medical Professional Learning) performance assessment tool in the emergency department: A pilot study.

Feedback and assessment are difficult to provide in the emergency department (ED) setting despite their critical importance for competency-based education, and traditional end-of-shift evaluations (ESEs) alone may be inadequate. The SIMPL (Society for Improving Medical Professional Learning) mobile application has been successfully implemented and studied in the operative setting for surgical training programs as a point-of-care tool that incorporates three assessment scales in addition to dictated feedback. SIMPL may represent a viable tool for enhancing workplace-based feedback and assessment in emergency medicine (EM). We implemented SIMPL at a 4-year EM residency program during a pilot study from March to June 2021 for observable activities such as medical resuscitations and related procedures. Faculty and residents underwent formal rater training prior to launch and were asked to complete surveys regarding the SIMPL app's content, usability, and future directions at the end of the pilot. A total of 36/58 (62%) of faculty completed at least one evaluation, for a total of 190 evaluations and an average of three evaluations per faculty. Faculty initiated 130/190 (68%) and residents initiated 60/190 (32%) evaluations. Ninety-one percent included dictated feedback. A total of 45/54 (83%) residents received at least one evaluation, with an average of 3.5 evaluations per resident. Residents generally agreed that SIMPL increased the quality of feedback received and that they valued dictated feedback. Residents generally did not value the numerical feedback provided from SIMPL. Relative to the residents, faculty overall responded more positively toward SIMPL. The pilot generated several suggestions to inform the optimization of the next version of SIMPL for EM training programs. The SIMPL app, originally developed for use in surgical training programs, can be implemented for use in EM residency programs, has positive support from faculty, and may provide important adjunct information beyond current ESEs.

Diagnostic and prognostic test assessment in emergency medicine: likelihood and diagnostic odds ratios

Emergency physicians use diagnostic and prognostic tests on a daily basis to assess for life-threatening illness and to inform clinical decisions. Current and new tests must be scientifically evaluated for...

P042: Workplace-based assessment in emergency medicine: how do physicians use entrustment anchors?

Introduction: Competency based medical education (CBME) has triggered widespread utilization of workplace-based assessment (WBA) tools in postgraduate training programs. These WBAs predominately use rating scales with entrustment anchors, such as the Ottawa Surgical Competency Operating Room Evaluation (O-SCORE). However, little is known about the factors that influence a supervising physician's decision to assign a particular rating on scales using entrustment anchors. This study aimed to identify the factors that influence supervisors’ ratings of trainees using WBA tools with entrustment anchors at the time of assessment and to explore the experiences with and challenges of using entrustment anchors in the emergency department (ED). Methods: A convenience sample of full-time emergency medicine (EM) faculty were recruited from two sites within a single academic Canadian EM hospital system. Fifty semi-structured interviews were conducted with EM physicians within two hours of completing a WBA for an EM trainee. Interviews were audio-recorded, transcribed verbatim, and independently analyzed by two members of the research team. Themes were stratified by trainee level, rating and task. Results: Interviews involved 73% (27/37) of all EM staff and captured assessments completed on 83% (37/50) of EM trainees. The mean WBA rating of studied samples was 4.34 ± 0.77 (2 to 5), which was similar to the mean rating of all WBAs completed during the study period. Overall, six major factors were identified that influenced staff WBA ratings: amount of guidance required, perceived competence through discussion and questioning, trainee experience, clinical context, past experience working with the trainee, and perceived confidence. The majority of staff denied struggling to assign ratings. However, when they did struggle, it involved the interpretation of WBA anchors and their application to the clinical context in the ED. Conclusion: Several factors appear to be taken into account by clinical supervisors when they make decisions regarding the particular rating that they will assign a trainee on a WBA that uses entrustment anchors. Not all of these factors are specific to that particular clinical encounter. The results from this study further our understanding on the use of entrustment anchors within the ED and may facilitate faculty development regarding WBA completion as we move forward in CBME.

Harnessing the power of simulation for assessment: Consensus recommendations for the use of simulation-based assessment in emergency medicine.

To address the increasing demand for the use of simulation for assessment, our objective was to review the literature pertaining to simulation-based assessment and develop a set of consensus-based expert-informed recommendations on the use of simulation-based assessment as presented at the 2019 Canadian Association of Emergency Physicians (CAEP) Academic Symposium on Education. A panel of Emergency Medicine (EM) physicians from across Canada, with leadership roles in simulation and/or assessment, was formed to develop the recommendations. An initial scoping literature review was conducted to extract principles of simulation-based assessment. These principles were refined via thematic analysis, and then used to derive a set of recommendations for the use of simulation-based assessment, organized by the Consensus Framework for Good Assessment. This was reviewed and revised via a national stakeholder survey, and then the recommendations were presented and revised at the consensus conference to generate a final set of recommendations on the use of simulation-based assessment in EM. We developed a set of recommendations for simulation-based assessment, using consensus-based expert-informed methods, across the domains of validity, reproducibility, feasibility, educational and catalytic effects, acceptability, and programmatic assessment. While the precise role of simulation-based assessment will be a subject of continued debate, we propose that these recommendations be used to assist educators and program leaders as they incorporate simulation-based assessment into their programs of assessment.

P166: The chief resident incubator - a virtual community of practice

Introduction: The Emergency Medicine Chief Resident Incubator is a year-long curriculum for chief residents that aims to provide participants with a virtual community of practice, formal administrative training, mentorship, and opportunities for scholarship. Methods: The Chief Resident Incubator was designed by Academic Life in Emergency Medicine (ALiEM; www.aliem.com) a digital health professions education organization in 2015, following a needs assessment in emergency medicine. A 12-month curriculum was created using constructivist social learning theory, with specific learning objectives that reflected 11 key administrative or professional development domains deemed important to chief residents. The topics covered included interviewing skills, contract negotiations, leadership, coaching, branding, conflict resolution, and ended with a focus on wellness and career longevity. A Core Leadership Team and Virtual Mentors were recruited to lead each annual iteration of the curriculum. The Incubator was implemented as a virtual community of practice using Slack©, a messaging and digital communication platform. Ancillary technology such as Google Hangout on Air© and Mailchimp© were used to facilitate learner engagement with the curriculum. Three in person networking events were hosted at three large emergency medicine and education conferences with special medical education guests. Outcomes include chief resident participation rates, Slack© activity, Google Hangout© web analytics, newsletter email engagement, and scholarship. We also incorporated a hidden curriculum throughout the year with multiple online publications, competitions for guest grand round presentations, and incorporation of digital technologies in medical education. Results: A total of 584 chief residents have participated over the first 3 years of the Chief Resident Incubator; this includes chief residents from over 212 residency programs across North America. Over 27,000 messages have been shared on Slack© (median 214 per week). A total of 32 Google Hangouts© have occurred over the course of the inaugural Incubator including faculty mentorship from Dr. Rob Rogers, Dr. Dara Kass and Dr. Amal Mattu. A monthly newsletter was distributed to the participants with an opening rate of 59%. Scholarship included 26 published academic blog posts, 2 open access In-Training exam prepbooks, a senior level online curriculum with 9 published modules and 3 book club reviews. Conclusion: The Chief Resident Incubator is a virtual community of practice that provides longitudinal training and mentorship for chief residents. This Incubator framework may be used to design similar professional development curricula across various health professions using an online digital platform.

Programmatic Assessment in Emergency Medicine: Implementation of Best Practices.

Programmatic assessment is the intentional collection of key data from multiple sources for both assessment of learning and assessment for learning. We developed a system of programmatic assessment (PA) to identify competency progression (summative) and assessment for learning to assist residents in their formative development. The programmatic assessment was designed iteratively from 2014 through 2016. All assessments were first categorized by competency domain and source of assessment. The number of assessment modalities for each competency domain was collected. These multisource assessments were then mapped by program leadership to the milestones to develop a master PA blueprint. A resident learning management system provided the platform for aggregating formative and summative data, allowing residents and faculty ongoing access to guide learning and assessment. A key component of programmatic assessment was to support resident integration of assessment information through feedback by faculty after shifts and during monthly formal assessments, semiannual resident reviews, and summative judgments by the Clinical Competency Committee. Through the PA, the 6 competency domains are assessed through multiple modalities: patient care (22 different assessments), professionalism (18), systems-based practice (17), interprofessional and communication skills (16), medical knowledge (11), and practice-based learning and improvement (6). Each assessment provides feedback to the resident in various formats. Our programmatic assessment has been utilized for more than 2 years with iterative improvements. The implementation of programmatic assessment allowed our program to organize diverse, multisourced feedback to drive both formative and summative assessments.

Assessing Team Leadership in Emergency Medicine: The Milestones and Beyond.

Team leadership is a critical skill for emergency medicine physicians that directly affects team performance and the quality of patient care. There exists a robust body of team science research supporting team leadership conceptual models and behavioral skill sets. However, to date, this work has not been widely incorporated into health care team leadership education. This narrative review has 3 aims: (1) to synthesize the team science literature and to translate important concepts and models to health care team leadership; (2) to describe how team leadership is currently represented in the health care literature and in the Accreditation Council for Graduate Medical Education Milestones for emergency medicine; and (3) to propose a novel, evidence-based framework for the assessment of team leadership in emergency medicine. We conducted a narrative review of the team science and health care literature. We summarized our findings and identified a list of team leadership behaviors that were then used to create a framework for team leadership assessment. Current health care team leadership measurement tools do not incorporate evidence-based models of leadership concepts from other established domains. The emergency medicine milestones include several team leadership behaviors as part of a larger resident evaluation program. However, they do not offer a comprehensive or cohesive representation of the team leadership construct. Despite the importance of team leadership to patient care, there is no standardized approach to team leadership assessment in emergency medicine. Based on the results of our review, we propose a novel team leadership assessment framework that is supported by the team science literature.

The use of ultrasound for haemodynamic assessment in emergency medicine

L’exploration hemodynamique par echographie du coeur, des poumons et de la veine cave inferieure en medecine d’urgence permet d’ameliorer la demarche diagnostique et guide les mesures therapeutiques, en particulier devant un etat de choc non traumatique. Il a ete demontre qu’elle etait superieure a l’examen clinique et a la radio thoracique sur le plan diagnostique. Elle est en mesure d’analyser ses composantes principales, hypovolemique, obstructive par tamponnade, coeur pulmonaire aigu ou defaillance ventriculaire gauche. Une conference de consensus internationale a recemment defini les objectifs, les moyens, les principaux resultats ainsi que les limites de cet examen pratique par des non-cardiologues. Les indices a acquerir en bidimensionnel et ceux plus complets faisant appel aux modes Doppler sont decrits avec leurs interpretations. Ils permettent ainsi de realiser un diagnostic physiopathologique, un traitement adapte et une surveillance evolutive afin d’en apprecier l’efficacite. Un algorithme diagnostique est propose a partir des donnees de la litterature.

A National Faculty Development Needs Assessment in Emergency Medicine.

Emergency physicians who work in academic settings enjoy an expanding number of roles beyond that of the skilled clinician. Faculty development (FD) encompasses the broad range of activities that institutions use to renew skill-sets and assist faculty members in these multiple roles. This study seeks to define the current FD needs and interests of Canadian academic emergency physicians (AEPs). An online survey was administered to 943 AEPs in eight centers across Canada to determine their current FD activities, provide a detailed understanding of their FD needs and interests, elucidate the perceived barriers to and motivation for engaging in FD, and identify preferred methods of delivery for FD activities. This national, cross-sectional survey was completed by 336 respondents. It shows that need for FD is universally high, particularly in traditional domains of scholarship, leadership and education (79%, 80%, 87% overall interest, respectively). However, the study also suggests that there is increasing need for FD in areas where current participation is lowest, namely research and social accountability (12% and 13% more interest, respectively). Senior and junior faculty evince equivalent overall FD interest (p>0.05), whereas female AEPs expressed greater overall FD needs in leadership (1.82 vs 1.44 activities, p=0.003) than males. Continued participation in FD activities is best promoted by offering relevant topics, at convenient times and locations. This study reports the first comprehensive national FD needs assessment of Canadian academic emergency physicians.

Resident as Teacher: General needs Assessment in Emergency Medicine

Introduction Increasingly, residents are involved in the education of medical students, their peers and other health professionals. This study focused on the general needs assessment of the emergency medicine (EM) residents in developing a resident-as-teacher curriculum. Methods An anonymous and voluntary survey, which consisted of multiple choice and free text questions, was sent to all emergency physicians (EPs) working in the 7 restructured hospitals in Singapore. Results A total of 68 (60%) EPs completed the online survey. Overall, 19 (28%) EPs have undergone formal training in teaching. Majority of the EPs felt that it was important for the residents to acquire competent teaching skills at the end of their residency training. All of the EPs were in consensus that there should be formal instruction to the residents on how to teach. The four themes which emerged from the barriers to developing teaching skills in residents were time constraints, lack of faculty development, attitudes of both faculty and residents and a non-conducive work environment with high service requirements. Conclusions To develop the resident as an all rounded Emergency Physician, structured training and formal instruction for the developing of teaching and feedback skills must be included in the residency training. However in order for the new curriculum to be successful and achieve its primary objective of developing the residents' competencies in teaching, barriers hindering both the faculty and residents have to be addressed. Finally, faculty development needs to be done to equip the faculty with the appropriate teaching and feedback tools. (Hong Kong j.emerg.med. 2015;22:226-234)

Past and future of emergency medicine education and training

Past and future of emergency medicine education and training

The Current State of Core Competency Assessment in Emergency Medicine and a Future Research Agenda: Recommendations of the Working Group on Assessment of Observable Learner Performance

In 2012, the Accreditation Council for Graduate Medical Education (ACGME) introduced the Next Accreditation System (NAS) for residency program accreditation. With implementation of the NAS, residents are assessed according to a series of new emergency medicine (EM)-specific performance milestones, and the frequency of assessment reporting is increased. These changes are driving the development of new assessment tools for the NAS that can be feasibly implemented by EM residency programs and that produce valid and reliable assessment data. This article summarizes the recommendations of the writing group on assessment of observable learner performance at the 2012 Academic Emergency Medicine consensus conference on education research in EM that took place on May 9, 2012, in Chicago, Illinois. The authors define an agenda for future assessment tool research and development that was arrived at by consensus during the conference.

Administrative assessment in emergency medicine: a tool of impact

Background The Department of emergency medicine was established one year ago. One of our main goals is to ensure correct initial treatment, stabilisation and referral to the correct speciality, of all patients admitted to the hospital. In optimising the process of triage and referral, all young doctors have to consult a coordinating doctor (i.e. a more experienced doctor) before initialising treatment, deciding on a diagnosis and referring to another speciality. This coordinating function is essential in the treatment of the patients and the patient flow in the emergency department. In order to improve the two coordinating doctors’ (i.e. the authors) referral skills, a study monitoring the clinical outcome of all patients treated in a certain period was performed.

Outcome Assessment in Emergency Medicine—A Beginning: Results of the Council of Emergency Medicine Residency Directors (CORD) Emergency Medicine Consensus Workgroup on Outcome Assessment

This article is designed to serve as a guide for emergency medicine (EM) educators seeking to comply with the measurement and reporting requirements for Phase 3 of the Accreditation Council for Graduate Medical Education (ACGME) Outcome Project. A consensus workshop held during the 2006 Council of Emergency Medicine Residency Directors (CORD) "Best Practices" conference identified specific measures for five of the six EM competencies--interpersonal communication skills, patient care, practice-based learning, professionalism, and systems-based practice (medical knowledge was excluded). The suggested measures described herein should allow for ease in data collection and applicability to multiple core competencies as program directors incorporate core competency outcome measurement into their EM residency training programs.

Mental status assessment in emergency medicine

Mental status assessment in emergency medicine

Clinical Assessment in Emergency Medicine

The Accreditation Council for Graduate Medical Education (ACGME) has challenged all residencies with a new paradigm—to teach and evaluate residents based on six core competencies. One of these core competencies is clinical assessment. Standardized patients and direct observation are the most promising for emergency medicine educators to use to assess this competency. There is much room for research and national standardization of methods.

Clinical assessment in emergency medicine.

The Accreditation Council for Graduate Medical Education (ACGME) has challenged all residencies with a new paradigm-to teach and evaluate residents based on six core competencies. One of these core competencies is clinical assessment. Standardized patients and direct observation are the most promising for emergency medicine educators to use to assess this competency. There is much room for research and national standardization of methods.

Assessment of emergency medicine: a comparison of an experimental objective structured clinical examintion with a practical examination

An emergency medicine and trauma programme was implemented at Ben Gurion University Medical School in Israel. Clinical performance assessment of the first-year course in emergency medicine and trauma was done using a practical examination (PE). In the continuous process of critically reviewing the course objectives and assessment methods the objective structured clinical examination (OSCE) was chosen, for the first time in our medical school, as a tool for course development as well as evaluation of the existing PE. Seven experimental OSCE stations were designed which covered some of the course and practical examination topics. Twenty-six first-year medical students have taken both examinations concurrently. Twenty-three students answered an attitude questionnaire regarding both examinations. Results have indicated that the OSCE provided additional and crucial information on students' deficiencies in clinical performance which were not available from the PE. Those differences were probably due to realistic OSCE station content, highly simulated set-ups, and the objectivity of the examiner, all of which have contributed to a more challenging examination, as compared to the PE. The OSCE in emergency medicine and trauma seemed to have a relatively high level of acceptance by both staff and students. In our opinion it seems that the OSCE is a better tool for first-year level final assessment in emergency medicine and trauma. is a better tool for first-year level final assessment