Royal College Emergency Medicine Research Articles

Current use of simulation for EPA assessment in emergency medicine.

Approximately five years ago, the Royal College emergency medicine programs in Canada implemented a competency-based paradigm and introduced the use of Entrustable Professional Activities (EPAs) for assessment of units of professional activity to assess trainees. Many competency-based medical education (CBME) based curricula, involve assessing for entrustment through observations of EPAs. While EPAs are frequently assessed in clinical settings, simulation is also used. This study aimed to characterize the use of simulation for EPA assessment. A study interview guide was jointly developed by all study authors and followed best practices for survey development. A national interview was conducted with program directors or assistant program directors across all the Royal College emergency medicine programs across Canada. Interviews were conducted over Microsoft Teams, interviews were recorded and transcribed, using Microsoft Teams transcribing service. Sample transcripts were analyzed for theme development. Themes were then reviewed by co-authors to ensure they were representative of the participants' views. A 64.7% response rate was achieved. Simulation has been widely adopted by EM training programs. All interviewees demonstrated support for the use of simulation for EPA assessment for many reasons, however, PDs acknowledged limitations and thematic analysis revealed certain themes and tensions for using simulation for EPA assessment. Thematic analysis revealed six major themes: widespread support for the use of simulation for EPA assessment, concerns regarding the potential for EPA assessment to become a "tick- box" exercise, logistical barriers limiting the use of simulation for EPA assessment, varied perceptions about the authenticity of using simulation for EPA assessment, the potential for simulation for EPA assessment to compromise learner psychological safety, and suggestions for the optimization of use of simulation for EPA assessment. Our findings offer insight for other programs and specialties on how simulation for EPA assessment can best be utilized. Programs should use these findings when considering using simulation for EPA assessment.

The relationship between empathy and the quality of the educational environment in Canadian emergency medicine residents.

PurposeEmpathy and quality of educational environment appear to be inversely correlated with burnout but the relationship between the two is largely unknown. Our primary objective was to examine the relationship between postgraduate educational environment and empathy. Secondary objectives included impact of gender, residency year and on- versus off-service context on levels of empathy and educational environment.MethodsA modified Dillman approach was used to conduct an email survey of Canadian Royal College Emergency Medicine residents in June 2020. The survey instrument included: demographic data, Toronto Empathy Questionnaire (TEQ) and Scan of Postgraduate Educational Environment Domains (SPEED). Logistic and linear regressions evaluated the association between TEQ and SPEED, and mean SPEED scores and covariates, respectively.ResultsResponse rate was 38% (138/363) with representation from all programs. Respondents were mean 30 years of age, 59% men and 25%, 20%, 18%, 24%, and 13% in postgraduate year (PGY) 1–5, respectively. There was no statistically significant association between high/low TEQ scores and mean SPEED score (p = 0.97). There were no statistically significant associations between any of the covariates and high/low TEQ scores (gender, p = 0.21; PGY, p = 0.58; on-versus off-service, p = 0.46) or mean SPEED score (gender, p = 0.95; PGY, p = 0.48; on- versus off-service, p = 0.07). Emergency medicine residents rated their educational environment on average 3.44 (+/- 0.43) out of four. 39 of 134 residents were found to have low empathy.ConclusionThere was no association between empathy and educational environment. Further research is needed to elucidate modifiable factors contributing to the development of low empathy in emergency medicine residents.

Pre-departure and Post-elective Requirements for Global Health Electives: Survey of Canadian Royal College Emergency Medicine Programs.

Objectives: Global Health (GH) electives offer unique learning opportunities; however, risks to trainees and host populations should be minimized through pre-departure training and post-elective debriefing. In a 2016 study, only three Canadian residency programs mandated such training, although specific data on Emergency Medicine (EM) programs is lacking. This study aimed to identify GH elective requirements and perceived training gaps among EM programs.Methods: We conducted two email surveys (one each for EM program directors [PDs] and residents) regarding training requirements and perceived gaps for GH electives. We also contacted university postgraduate medical education (PGME) and GH offices, via their online publicized emails, to assess university-wide requirements and resources.Results: Nine PDs responded, with 78% reporting having residents participate in GH electives. Many PDs (67%) believed residents were moderately prepared for GH electives, while 33% felt they were unprepared to some degree. Forty seven out of an estimated 380 EM residents responded with 35% having completed a GH elective during residency. Of those, only one (6%) reported feeling very prepared, and 43% believed there was a need to improve trainings. Uncertainty around training requirements was reported, and residents identified challenges faced on electives, as well as priority topics for training. Responses from PGME and GH offices indicated that pre-departure training and post-elective debriefing were required or available at more universities than was indicated by the PD and resident respondents. However university requirements varied widely, with some exclusively requiring basic travel information and Health and Safety checklists or modules. The disparate responses indicate that residents and PDs may either be unaware of university requirements or not utilize available training resources for GH electives.Conclusions: Although Canadian EM residents participate in GH electives, the majority of training programs do not require pre-departure training or post-elective debriefing. PDs and residents report varying levels of preparedness, and residents acknowledge a variety of challenges during GH electives. This information can be used to inform pre-departure training and post-elective debriefing and encourage EM residents to access available university-wide training.

LO79: The variable landscape of resident selection: A study of Canadian Royal College emergency medicine training programs

Introduction: Little is known about how Royal College emergency medicine (RCEM) residency programs are selecting their residents. This creates uncertainty regarding alignment between our current selection processes and known best practices and results in a process that is difficult to navigate for prospective candidates. We seek to describe the current selection processes of Canadian RCEM programs. Methods: An online survey was distributed to all RCEM program directors and assistant directors. The survey instrument included 22 questions consisting of both open-ended (free text) and closed-ended (Likert scale) elements. Questions sought qualitative and quantitative data from the following 6 domains; paper application, letters of reference, elective selection, interview, rank order, and selection process evaluation. Descriptive statistics were used. Results: We received responses from 13/14 programs for an aggregate response rate of 92.9%. A candidate's letter of reference was identified as the single most important item from the paper application (38.5%). Having a high level of familiarity with the applicant was considered to be the most important characteristic of a reference letter author (46.2%). Respondents found that providing a percentile rank of the applicant was useful when reviewing candidate reference letters. Once the interview stage is reached, 76.9% of programs stated that the interview was weighted at least as heavily as the paper application; 53.8% weighted the interview more heavily. Once final candidate scores are established for both the paper application and the interview, 100% of programs indicated that further adjustment is made to the rank order list. Only 1/13 programs reported ever having completed a formal evaluation of their selection process. Conclusion: The information gained from this study helps to characterize the landscape of the RCEM residency selection process. We identified significant heterogeneity between programs with respect to which application elements were most valued. Canadian emergency medicine residency programs should re-evaluate their selection processes to achieve improved consistency and better alignment with selection best practices.

MP18: Pre-departure and post-elective requirements for global health electives: a survey of Canadian Royal College emergency medicine programs

Introduction: Participation in Global Health (GH) electives can improve resourcefulness, cultural and ethical insight, and personal development. Risks to trainees, hosts and institutions may be minimized through pre-departure and post-elective training. In 2016 such training was mandatory in only 3 Canadian residency programs, however there is no published data specific to Canadian Emergency Medicine (EM) programs. This study sought to identify current GH elective requirements and related perceived gaps among Royal College EM programs. Methods: We conducted two cross-sectional surveys in 2019 (one each for Royal College EM PDs and residents) via email regarding training requirements for GH electives. Additionally, a survey link was distributed in the CAEP EM resident newsletter. We also contacted university PGME and/or global health offices to understand and collate university-wide requirements and resources. Results: Nine PDs responded, with 78% reporting having 1-5 residents participate in GH electives yearly. Many PDs were unsure of the requirements surrounding GH electives; two reported that pre-departure training was required, while none reported requiring post-departure debriefs. Overall, 67% of PDs felt that their residents were moderately prepared for GH electives and 33% felt they were unprepared to some degree. Thirty-three percent believed that improvements should be made to either pre-departure training or both pre- and post-departure training, while 56% were unsure if improvements were needed. Forty-seven out of an estimated 380 residents responded. Thirty-five percent of residents had completed a GH elective during residency. Of residents who participated in a GH elective, only one (6%) reported feeling very prepared; 18 residents (43%) reported there was a need to improve trainings. Residents reported a number of challenges during electives (lack of resources, inadequate supervision, safety issues) and identified priority topics for training. Conclusion: Although EM residents are participating in GH electives, the majority of EM residency programs do not require pre- or post-departure training. Some PDs and residents report varying levels of preparedness, and residents acknowledge a variety of challenges during GH electives. This information can be used to inform pre-departure/post-elective GH training or to encourage EM residents to access university-wide training.

P025: Improving senior resident engagement at academic core rounds

Introduction: Royal College Emergency Medicine (EM) trainees at the University of Ottawa participate in weekly Academic Full Days (AFD) that consist of didactic activities, simulation-based learning, and core content sessions referred to as Core Rounds (CR). Despite CR being intentioned for all EM trainees, an attendance attrition has been noted as trainees progress towards their senior (SR) years (PGY3-5). The objectives of this study were to (1) identify barriers to SR trainee CR attendance and (2) identify areas for CR improvement. Methods: An on-line survey was administered to SR EM trainees (PGY3-5, n = 28) and recent graduates from our program (practice year 1-2, n = 20) to explore perceptions of the value of AFDs, CR attendance barriers, and areas for CR improvement. The survey consisted of 5-point Likert scales and free-text responses. Quantitative responses were analyzed using Microsoft Excel. Free-text responses were analyzed qualitatively using thematic analysis. Each free-text response was reviewed independently by two investigators (JML, MCL) and underwent line-by-line coding. Through joint discussions, the codes from each response were synthesized and themes were identified. Results: Of the 48 trainees and attendings surveyed, 32 responded (response rate 67%). Most respondents (90%) stated they benefited from SR trainee attendance when they were at a junior (JR) level. The majority perceived they benefited less from CR as a SR trainee compared to when they were a JR trainee (85%). Further, 87% responded that CR were not tailored to a SR level, and that they would attend more frequently if sessions were geared to their level (81%). From our thematic analysis, three themes emerged relating to SR trainee absenteeism: 1) CR quality, 2) External Factors (eg. trainee fatigue) and 3) Malalignment with trainees’ own education plan. We also identified three themes relating to areas for CR improvement: 1) CR content, 2) CR format and 3) SR trainee involvement. Conclusion: Respondents indicated a benefit to having SR trainee presence at CR. This study identified barriers to SR resident attendance at CR and areas for improvement. With the transition to competency based medical education it is critical that trainees engage in effective educational experiences, especially as the RCPSC does not mandate AFDs for EM training in this new curriculum. A culture-change initiative and CR reformat is now underway at our institution with planned post-implementation analysis.

MP56: A National survey of burnout and mentorship programs amongst Royal College Emergency Medicine residents

Introduction: In recent years, there has been growing interest in the field of physician wellness and burnout. Past research has shown that the prevalence of burnout is non-uniform between specialties and is most prevalent amongst emergency medicine physicians. Additionally, burnout can be observed amongst individuals early in their medical careers, including medical students and residents. To date, there is no national perspective of burnout amongst Canadian Royal College of Emergency Medicine (EM) residents. Our study looks to provide a national survey of burnout in this population as well as characterize mentorship programs at training sites. Methods: An anonymous electronic survey was e-mailed to Canadian EM residents via local program directors. Characteristics of mentor-mentee relationships and quality of residents’ mentorship experiences were assessed on a 6-point Likert scale. The Maslach Burnout Inventory – Human Services Survey (MBI-HSS) for medical personnel was used to assess burnout on three dimensions (emotional exhaustion, depersonalization and personal accomplishment). Burnout was dichotomized as present or absent if the MBI criteria are met (emotional exhaustion score > 26 or depersonalization score > 9 or personal accomplishment <34). Results: To date, 52 responses have been collected. Respondents are primarily male (63%) and in their PGY year 1-3 (71%). Responses were collected from 6/14 (43%) of eligible programs. 84% of residents currently had an emergency medicine mentor. Of these, 8% were dissatisfied with their residency's mentorship program and 55% were satisfied/very satisfied. 72% of residents met the threshold for burnout in at least one dimension of the MBI (3 dimensions = 17%; 2 dimensions = 17%; 1 dimension = 38%) and 13% cited considering suicide during their training. Conclusion: Results thus far suggest significant burnout amongst Royal College of Emergency Medicine residents. Alarmingly, 13% of responders cited having contemplated suicide during their training. These results point to an important opportunity to better support EM residents during their training to improve wellness and reduce burnout. Our findings suggest a high prevalence of residents with established mentors and future analyses will examine the correlation between mentorship characteristics and resident burnout levels.

LO55: Signal & noise – do professionalism concerns impact decision-making of competence committees?

Introduction: Competence committees (CCs) struggle with incorporating professionalism issues into resident progression decisions. This study examined how professionalism concerns influence individual faculty decisions about resident progression using simulated CC reviews. Methods: In 2017, the investigators conducted a survey of 25 program directors of Royal College emergency medicine residency training programs in Canada and those faculty members who are members of the CCs (or equivalent) at their home institution. The survey contained twelve resident portfolios, each containing formative and summative information available to a CC for making progression decisions. Six portfolios outlined residents progressing as expected and six were not progressing as expected. Further, a professionalism variable (PV) was added to six portfolios, evenly split between those residents progressing as expected and not. Participants were asked to make progression decisions based on each portfolio. Results: Raters were able to consistently identify a resident needing an educational intervention versus those who did not. When a PV was added, the consistency among raters decreased by 34.2% in those residents progressing as expected, versus increasing by 3.8% in those not progressing as expected (p = 0.01). Conclusion: When using an unstructured review of a simulated resident portfolio, individual reviewers can better discriminate between trainees progressing as expected when professionalism concerns are added. Considering this, educators using a competence committee in a CBME program must have a system to acquire and document professionalism issues to make appropriate progress decisions.

MP23: Giving medical students what they deserve - a rigorous, equitable and defensible CaRMS selection process

Innovation Concept: The fairness of the Canadian Residency Matching Service (CaRMS) selection process has been called into question by rising rates of unmatched medical students and reports of bias and subjectivity. We outline how the University of Saskatchewan Royal College emergency medicine program evaluates CaRMS applications in a standardized, rigorous, equitable and defensible manner. Methods: Our CaRMS applicant evaluation methods were first utilized in the 2017 CaRMS cycle, based on published Best Practices, and have been refined yearly to ensure validity, standardization, defensibility, rigour, and to improve the speed and flow of data processing. To determine the reliability of the total application scores for each rater, single measures intraclass correlation coefficients (ICCs) were calculated using a random effects model in 2017 and 2018. Curriculum, Tool or Material: A secure, online spreadsheet was created that includes applicant names, reviewer assignments, data entry boxes, and formulas. Each file reviewer entered data in a dedicated sheet within the document. Each application was reviewed by two staff physicians and two to four residents. File reviewers used a standardized, criterion-based scoring rubric for each application component. The file score for each reviewer-applicant pair was converted into a z-score based on each reviewer's distribution of scores. Z-scores of all reviewers for a single applicant were then combined by weighted average, with the group of staff and group of residents each being weighted to represent half of the final file score. The ICC for the total raw scores improved from 0.38 (poor) in 2017 to 0.52 (moderate) in 2018. The data from each reviewer was amalgamated into a master sheet where applicants were sorted by final file score and heat-mapped to offer a visual aid regarding differences in ratings. Conclusion: Our innovation uses heat-mapped and formula-populated spreadsheets, scoring rubrics, and z-scores to normalize variation in scoring trends between reviewers. We believe this approach provides a rigorous, defensible, and reproducible process by which Canadian residency programs can appraise applicants and create a rank order list.

P105: Charting in the electronic medical record: Perspectives of Emergency Medicine residents

Introduction: The literature reveals that residents spend significant amounts of time working with and charting in electronic medical records (EMR). As adoption of EMRs accelerates among emergency medicine (EM) departments, postgraduate programs will need to adapt curricula related to communication in the patient record. In order to make targeted changes, clinician-educators need a better understanding of how the documentation practices of trainees develop and change over residency, as well as the challenges they face in effectively charting. We gathered the perspectives of EM residents on data entry in the EMR to identify opportunities for such change. Methods: We recruited residents from all five years of the Royal College EM residency program at Queen's University and conducted focus groups from August to October 2018. Data collection was audio recorded and later transcribed. Line-by-line coding was performed independently by both AR and NP. A final codebook was validated by ZH. The codebook was then thematically analyzed to identify and characterize themes from the data. The study was approved by the Queen's University Health Sciences Research Ethics Board. Results: 15 EM residents participated. Groups discussed similar challenges with charting, including time constraints, ensuring sufficient, but appropriate detail, variable preceptor expectations, and an inability to draw diagrams. All residents noted formal teaching of the SOAP note framework during medical school and reported receiving an introductory EMR session. Groups highlighted the importance of feedback, especially from physicians with medicolegal experience. They also described more informal learning strategies, including receiving tips from preceptors during shifts and reading the notes of others. They also reported that changes in their documentation practices as junior and senior residents were largely due to a graduation of responsibility and medicolegal considerations. Conclusion: Our results suggest there is a lack of formal postgraduate training for EM residents with respect to documentation in the EMR with reliance on informal teaching and feedback. Future work should explore opportunities to address this gap with various educational strategies, including the development of specific objectives, application of consistent expectations, modelling of excellent chart notes in teaching, and instruction by preceptors with medicolegal experience.

P093: Performing the balancing act: emergency medicine physicians’ multifaceted roles and their influence on trainee assessment

Introduction: Competency-based workplace assessments are important in clinical training. However, feedback and assessment are still often perceived as unsatisfactory, particularly in busy settings such as emergency departments. Currently, little is known about how attending staff physicians sense of self may interface with the processes they use to assess and give feedback to trainees. We aimed to understand how attendings perceive their roles when tasked with conducting assessments and providing feedback to trainees. Methods: We conducted semi-structured, individual interviews with attendings (n=16) who used McMAP (McMaster Modular Assessment Program), a workplace-based assessment system at McMaster Universitys Royal College Emergency Medicine program. Attendings were recruited using snowball sampling. Data were interpreted using thematic analysis, sensitized to the dramaturgical lens and rater cognition frameworks. Results: Attendings identified themselves using three distinct but intimately connected roles when assessing trainee performance: the doctor that ensures the safety and well-being of patients; the coach (educator) that empowers, guides, and supports the next generation of medical doctors; and the assessor that formally assesses a trainees progression through the residency program. These roles are influenced by clinical training and experience, teaching experience and context. Conclusion: The ways in which attendings assess and provide feedback to trainees involve a complex and dynamic process that is influenced by their perceived roles as a doctor, coach, and assessor. Understanding the way attendings view and juggle their roles may provide insight into potentially new approaches to assessment and feedback. Results and implications will be discussed.

LO40: Describing CCFP(EM) programs in Canada: a national survey of program directors

Introduction: Enhanced skills training in emergency medicine (EM) for family physicians (CCFP(EM)) has existed since the 1970s. Accreditation standards define what every program must and should have, yet little is known on what is currently done across Canada. Our objectives were to: 1) describe major components of CCFP(EM) programs; and 2) determine how programs incorporate these components into their curriculum. Methods: A rigorous development process included expert content development and in-person pilot testing using Royal College Emergency Medicine Program Directors. An electronic survey questionnaire comprised of 63 questions was administered to all 17 CCFP(EM) program directors using a modified Dillman technique. Non-responders were sent a reminder email every 2 weeks over a 6-week period and an in-person reminder was given to non-responders at a face to face meeting 4 weeks after the initial survey was sent in June 2016. Results: All 17/17 (100%) program directors responded. There was considerable variation in administrative structure and financial support for each program. All programs provided ultrasound courses for basic skills (trauma, abdominal aortic aneurysm, intrauterine pregnancy). Variation exists for offering independent ultrasound certification (77%), advanced scanning (18%) and protected academic time for scanning (53%). All programs utilize high fidelity simulation. Some programs use in situ simulation (18%) and hold a simulation boot camp (41%). Most centres required an academic project, most commonly a quality assurance project (53%) and/or a critical appraisal of the literature (59%). Publication or national conference presentations were required by 12% of programs. Competency based assessments use simulation (88%) and direct observations (53%). Only 24% of programs have a transition to practice curriculum. All programs maintain strong connections to family medicine. Conclusion: This study demonstrates diverse structures of CCFP(EM) programs across Canada. Programs are similar regarding the provision of ultrasound, simulation and protected teaching time. Variation exists in administrative structure and financial resources of each program, academic project requirements, and how programs perform competency based assessments.

LO42: Ice Cream Rounds: the adaptation and implementation of a peer-support wellness rounds in an emergency medicine residency training program

Introduction/Innovation Concept: Emergency Medicine (EM) is a specialty that requires physicians to deal with acutely ill patients in a fast-paced environment, which can create stress, mental exhaustion and burnout. Continually changing working teams in the Emergency Department does not always allow appropriate debriefing for difficult patient encounters and outcomes on shift. To address these challenges, we sought to adapt and implement a peer-support rounds called ‘Ice Cream Rounds’ used in some Pediatric training programs for an EM training program. Methods: CCFP and Royal College EM residents were surveyed to determine interest and need for Ice Cream Rounds. Of the 31/50 respondents, 87% (26/31) identified their co-residents as their main source of support after difficult patient encounters and 71% (22/31) felt that current opportunities to debrief after difficult experiences were only “sometimes” or “rarely” adequate. Overall, 84% (26/31) were interested in attending Ice Cream Rounds. Residents expressed that they did not want staff present for Ice Cream Rounds so two residents (SCS and TK) obtained training to lead peer-support sessions from The Faculty of Medicine Wellness Program. Attendance at rounds was voluntary and the EM program provided funding for refreshments. Two Ice Cream Rounds were piloted. Attendance and feedback was recorded from pilot sessions. Curriculum, Tool, or Material: Resident-only, peer-run confidential debriefing sessions. Sessions were voluntary and lasted one hour. Approximately 20-30/50 residents attended each Ice Cream Rounds. Discussions were confidential but include topics such as difficult patient encounters, poor patient outcomes, challenges in residency, and ethical issues. In response to positive attendance and feedback, the EM program provided 3-4 one-hour protected time slots with a stipend for refreshments for future academic years. Comments from residents consistently reaffirmed that Ice Cream Rounds was a helpful forum to discuss important issues with colleagues and provided a safe and confidential resource to help cope with residency challenges. Conclusion: We adapted, implemented, and evaluated a novel Peer-Support Wellness Rounds for debriefing resident issues and difficult patient encounters in a EM training program. To our knowledge this is the first Canadian initiative to implement such rounds in an EM training program. We believe that this template can be easily adopted by any EM training program and will effectively address wellness challenges faced by residents during their training.

Resident Development via Progress Testing and Test-Marking: An Innovation and Program Evaluation.

IntroductionSince 2008, the McMaster University Royal College Emergency Medicine residency training program has run practice Short Answer Question (SAQ) examinations to help residents test their knowledge and gain practice in answering exam-style questions. However, marking this type of SAQ exam is time-consuming.MethodsTo help address this problem, we require that senior residents help mark at least one exam per year alongside faculty members. Examinees’ identities are kept anonymous by assigning a random number to each resident, which is only decoded after marking. Aggregation of marks is done by faculty only. The senior residents and faculty members all share sequential marking of each question. Each question is reviewed, and exemplar “best practice” answers are discussed. As novel/unusual answers appear, instantaneous fact-checking (via textbooks, or the internet) and discussions occur allowing for real-time modification to the answer keys as needed.ResultsA total of 22 out of 37 residents (post graduate year 1 to post graduate year 5 (PGY1 to PGY5)) participated in a recent program evaluation focus group. This evaluation showed that residents feel quite positive about this process. With the anonymization process, residents do not object to their colleagues seeing and marking their answers. Senior residents have found this process informative and have felt that this process helps them gain insight into better “examsmanship.”ConclusionsInvolving residents in marking short-answer exams is acceptable and perceived as useful experience for improving exam-taking skills. More studies of similar innovations would be required to determine to what extent this may be the case.

LO093: A national needs assessment survey for the development of a quality improvement and patient safety curriculum for Canadian emergency medicine residents

Introduction: The Canadian Medical Education Directives for Specialists (CanMEDS) framework defines the competencies that postgraduate medical education programs must cover for resident physicians. The 2015 iteration of the CanMEDS framework emphasizes Quality Improvement and Patient Safety (QIPS), given their role in the provision of high value and cost-effective care. However, the opinion of Emergency Medicine (EM) program directors (PDs) regarding the need for QIPS curricula is unknown, as is the current level of knowledge of EM residents in QIPS principles. We therefore sought to determine the need for a QIPS curriculum for EM residents in a Canadian Royal College EM program. Methods: We developed a national multi-modal needs assessment. This included a survey of all Royal College EM residency PDs across Canada, as well as an evaluative assessment of baseline QIPS knowledge of 30 EM residents at the University of Toronto (UT). The resident evaluation was done using the validated Revised QI Knowledge Application Tool (QIKAT-R), which evaluates an individual’s ability to decipher a systematic quality problem from short clinical scenarios and to propose change initiatives for improvement. Results: Eight of the 13 (62%) PDs responded to the survey, unanimously agreeing that QIPS should be a formal part of residency training. However, challenges identified included the lack of qualified and available faculty to develop and teach QIPS material. 30 of 30 (100%) residents spanning three cohorts completed the QIKAT-R. Median overall score was 11 out of 27 points (IQR 9-14), demonstrating the lack of poor baseline QIPS knowledge amongst residents. Conclusion: QIPS is felt to be a necessary part of residency training, but the lack of available and qualified faculty makes developing and implementing such curriculum challenging. Residents at UT consistently performed poorly on a validated QIPS assessment tool, confirming the need for a formal QIPS curriculum. We are now developing a longitudinal, evidence-based QIPS curriculum that trains both residents and faculty to contribute to QI projects at the institution level.

LO089: Out of province elective restrictions: implications for Royal College Emergency Medicine training

Introduction / Innovation Concept: Several provinces (AB, SK and QC) have recently introduced restrictions to out of province (OOP) electives. Concurrently, enhanced competency training is a prominent part of RCPSC Emergency Medicine (EM) programs (Thoma et al., 2015). We present the implications of OOP elective restrictions on RCPSC-EM training and education. The revised 2008 RCPSC-EM requirements specify a minimum of 6 months devoted to achieving a particular expertise pertinent to the practice of EM. The most restrictive policies permit up to 3 months OOP during the 5-year residency. This limits residents’ ability to pursue enhanced competency training opportunities outside their training site. Enhanced training might be a graduate degree, fellowship or clinical year designed by the resident and program director. Enhanced training can help achieve specific career goals, meet the needs of the institution where the resident will practice, and contribute to the growth and development of EM in Canada. Methods: New OOP policies are evaluated using the Health Reform Analysis (HRA) and SWOT (Strengths, Weaknesses, Opportunities and Threats) analysis methodologies. Stated and implied reasons for reform are evaluated and stakeholder perspectives (health system authorities, partner universities, resident physicians and the general public) provided. Curriculum, Tool, or Material: The material includes previous out of province elective policies and recent reforms. Conclusion: Policies for the 4th year EM elective time are variable across universities. This has resulted in inconsistent approval of residents’ requests for OOP enhanced training. Thus, enhanced training that might be approved at one site, may not be at another. Several test cases already exist and will be presented. This data has not been previously collated or reported to our knowledge. Varied interpretation of newly emerging policies has implications for the consistency, equity, and future of EM residency training in Canada.

A national survey of Canadian emergency medicine residents' comfort with geriatric emergency medicine.

Geriatric patients represent a large and complex subgroup seen in emergency departments (EDs). Competencies in geriatric emergency medicine (EM) training have been established. Our objectives were to examine Canadian postgraduate year (PGY)-5 EM residents' comfort with the geriatric EM competency domains, assess whether Canadian EM residents become more comfortable through residency, and determine whether geriatric educational exposures are correlated with resident comfort with geriatric EM. A national, cross-sectional study of PGY-1 and PGY-5 Royal College EM residents was conducted to determine their comfort in geriatric EM clinical competency domains. Residents reported their level of comfort in satisfying each competency domain using a seven-point Likert scale. Residents were also asked about the location of their medical education as well as the type and number of different geriatric exposures that they had received to date. Of the 141 eligible residents from across Canada, 77% (109) consented to participate. None of the PGY-1 EM residents and 34% (14) of PGY-5 EM residents reported that they were comfortable with all eight geriatric EM competency domains. PGY-5 EM residents were significantly more comfortable than PGY-1 EM residents. Residents reported a highly variable range of geriatric educational exposures obtained during training. No relationship was found between resident-reported comfort and the nature or number of geriatric exposures that they had received. Current Royal College EM residency training in Canada may not be adequately preparing graduates to be comfortable with defined competencies for the care of older ED patients.

The McMaster Modular Assessment Program (McMAP)

To assess resident competence, generalist programs such as emergency medicine (EM), which cover a broad content and skills base, require a substantial number of work-based assessments (WBAs) that integrate qualitative and quantitative data. The McMaster Modular Assessment Program (McMAP), implemented in McMaster University's Royal College EM residency program in 2011-2012, is a programmatic assessment system that collects and aggregates data from 42 WBA instruments aligned with EM tasks and mapped to the CanMEDS competency framework. These instruments incorporate task-specific checklists, behaviorally anchored task-specific and global performance ratings, and written comments. They are completed by faculty following direct observation of residents during shifts. The rotation preceptor uses aggregated data to complete an end-of-rotation report for each resident in the form of a qualitative global assessment of performance. The quality of end-of-rotation reports-as measured by comparing report quality one year prior to and one year after McMAP implementation using the Completed Clinical Evaluation Report Rating tool-has improved significantly (P < .001). This may be a result of basing McMAP's end-of-rotation reports on robust documentation of performance by multiple raters throughout a rotation rather than relying on a single faculty member's recall at rotation's end as in the previous system. By aligning theory-based assessment instruments with authentic EM work-based tasks, McMAP has changed the residency program's culture to normalize daily feedback. Next steps include determining how to handle "big data" in assessment and delineating policies for promotion decisions.

Outcome Feedback within Emergency Medicine Training Programs: An Opportunity to Apply the Theory of Deliberate Practice?

Outcome feedback is the process of learning patient outcomes after their care within the emergency department. We conducted a national survey of Canadian Royal College emergency medicine (EM) residents and program directors to determine the extent to which active outcome feedback and follow-up occurred. We also compared the perceived educational value of outcome feedback between residents and program directors. We distributed surveys to all Royal College-accredited adult and pediatric EM training programs using a modified Dillman method. We analyzed the data using student's t-test for continuous variables and Fisher's exact test for categorical variables. We received 210 completed surveys from 260 eligible residents (80.8%) and 21 of 24 program directors (87.5%) (overall 81.3%). Mandatory active outcome feedback was not present in any EM training program for admitted or discharged patients (0/21). Follow-up was performed electively by 89.4% of residents for patients admitted to the hospital, and by 44.2% of residents for patients discharged home. A majority of residents (76.9%) believed that patient follow-up should be mandatory compared to 42.9% of program directors (p=0.002). The perceived educational value of outcome feedback was 5.8/7 for residents and 5.1/7 for program directors (difference 0.7; p=0.002) based on a seven-point Likert scale (1=not important; 7=very important). While Canadian EM training programs do not mandate follow-up, it is performed electively by the majority of residents surveyed. Residents place a significantly greater educational value on outcome feedback than their program directors, and believe that follow-up should be a mandatory component of EM residencies.

191: Pediatric Pain Management: the Emergency Medicine Residents' Perspectives

Pain is a component of up to 80% of all emergency department (ED) visits. Oligoanalgesia, the under-treatment of pain, is a recognized problem in the ED and can have numerous detrimental effects for the pediatric patient. Surveying emergency medicine residents, our primary objectives were to describe a) the extent and type of training received in pediatric acute pain management, b) their management approach to common painful pediatric presentations, c) their level of comfort in assessing and treating acute pain, d) their perceived facilitators and barriers to optimally managing pain, and e) their attitudes towards managing such pain in the ED. A descriptive, cross-sectional survey was administered to all Royal College Emergency Medicine (EM) and Pediatric Emergency Medicine (PEM) residents, the Canadian College of Family Physician's EM residents, Clinical Fellows and PEM Fellows at three Canadian universities. A novel survey tool was created and distributed from May to June 2013. Surveys were completed online, using a modified Dillman approach. Our response rate was 46% (56 of 122). Forty-five percent (25 of 55) of residents had not received any training for pediatric pain assessment. All years of residents were uncomfortable with the assessment of pain in 0- to 2-year-olds (P=0.07). For patients aged two to 12 years, level of comfort with pain assessment increased with years of training (P=0.02). When assessing pain in children with disabilities, 83% (45 of 54) of residents were ‘extremely’ or ‘somewhat’ uncomfortable; 65% (47 of 55) of residents had not received any training on how to assess pain for such patients. Sixty-nine percent (38 of 55) of residents had received training on how to treat pediatric pain. All residents were more comfortable using pain medication for a nine-year-old, as compared to a one-year-old (oral oxycodone P=0.00003, oral morphine P=0.00002, IV morphine P=0.004). Ninety-eight percent (54 of 55) of residents felt that it was ‘extremely’ or ‘somewhat’ important to receive education about pediatric pain management. Their preferred methods to learn children's pain management were through role modeling (61%) and lectures (57%). Top challenges to optimal pediatric pain management were non-verbal patients, patients with disabilities, and young infants. Pediatric pain management education is important to EM residents. Residents recall receiving sub-optimal training on this important topic. They are especially uneasy in the management of younger children and those with disabilities. This study's findings can be used to inform post-graduate curriculum planning, in order to optimize pediatric pain assessment and treatment education.