Resident Performance Research Articles

Abstract 307: In-Situ Simulation-Based Booster ACLS Training to Increase the Confidence and Performance of Internal Medicine Residents as Code Blue Leaders - A Quality Improvement Project

Background: In the United States, nearly 290,000 adult patients develop in-hospital cardiac arrest (IHCA) every year. Delivery of guideline-based resuscitation has improved patient outcomes. During 2022, our hospital had 68 inpatient i.e., approximately 1 code/resident. Residents have reported desire for more exposure. The 2020 AHA Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care recommended: 1) booster training, 2) in-situ simulation-based resuscitation training, and 3) specific teamwork and leadership training for healthcare providers. We aimed to implement such training to achieve an improvement in resident confidence and performance as code blue leaders. Methods: Pre-intervention, residents completed a 5-item 0-100 continuous scale to assess confidence in major code leader skills (Fig 1-A). Interventions included: 1) Calling residents to patient rooms to run simulation-based codes, and 2) a didactic session that discussed common errors during codes. Time to key ACLS steps was recorded during the simulations. At the end of the project, residents completed the post-intervention survey of confidence and balancing measures. Results: 13 residents led simulation-based codes, 20 attended the didactic session, and 25 completed the survey. Resident confidence in the defined skills increased by an average of 30%. (P<0.05) (Fig 1-A). Residents who led codes after the didactic session had shorter times to key ACLS steps (Fig 1-B). Balancing measures included resident-reported satisfaction and perceived disruption to clinical workflow (100% and 0% of responses, respectively). Conclusions: In-situ simulation-based booster ACLS training was associated with an increase in the confidence and performance of IM residents as code blue leaders. Incorporation of such training in IM residency curriculum should be considered as a practical application to sustain the gains of this QI initiative.

Evaluation of a Remote Curriculum to Improve Internal Medicine Resident Knowledge and Confidence in Oncologic Emergency Identification and Management

Background Oncologic emergencies (OEs) are life-threatening complications of malignancy that can present in multiple clinical settings and necessitate early recognition and intervention. Most medical resident training is experiential, but variability in exposure can yield gaps in medical knowledge that could result in delayed patient care. Remote curricula are commonly used to fill these gaps and reduce classroom burden on trainees. Given their time-sensitive nature, OEs are a prime target to augment clinical experience with a supplementary curriculum to prepare residents for new scenarios. This study evaluates whether an original, remote curriculum improves internal medicine resident knowledge of and confidence in recognition and management of OEs. Methods With expert faculty input, we developed a remote curriculum for first-year internal medicine residents at a tertiary care academic medical center. The curriculum includes 6 OEs: tumor lysis syndrome (TLS), hypercalcemia (hyperCa), neutropenic fever (NF), hyperviscosity syndrome and leukostasis (HVS/LS), superior vena cava (SVC) syndrome, and malignant spinal cord compression (CC). OEs were selected based upon prominence in the literature, morbidity and mortality, need for urgent management, and association with malignancy. We created an online lesson in Qualtrics for each OE. Each lesson was designed to be completed in 15 minutes via computer or smartphone. Lessons were delivered by email every 10 days, with reminders every 5 days. Learning objectives for each OE lesson were to 1) identify concerning signs and symptoms, 2) order appropriate workup, and 3) provide prompt management in the appropriate clinical setting. Lessons included clinical questions and evidence-based information delivery contextualized by case presentations. We evaluated the curriculum's efficacy with a randomized, controlled study. First-year residents were randomly assigned to two groups (“Intern A” and “Intern B”). We built assessments of knowledge and confidence with expert faculty input. Confidence in identifying and managing each OE was assessed by Likert scale (“1 - completely confident,” “5 - not at all confident”). Knowledge was evaluated with 3 multiple choice questions per OE, for a maximum knowledge score of 18. After both groups completed the pre-test, Intern A received the curriculum. After curriculum completion, both groups completed the post-test. All interns were required to complete the curriculum, but research participation was optional. Residents consented to participation prior to each test administration. All lessons and research activities were anonymous. Analysis was completed in GraphPad Prism 9 using the unpaired t-test or one-way ANOVA with post-hoc testing, as appropriate. Results Thirty-nine internal medicine residents completed the curriculum. Twenty were assigned to Intern A and 19 to Intern B. Intern A had 10 pre-test respondents (50.0%) with a mean knowledge score of 9.13 (50.7% correct), and Intern B had 13 pre-test respondents (68.4%) with a mean knowledge score of 9.35 (51.9% correct). There were no differences by OE-specific score. Baseline confidence was similar between groups (Table 1). Comparison of mean knowledge scores is shown in Figure 1. Intern A and B had 9 (45.0%) and 10 (52.6%) post-test respondents, respectively. Both Intern A (mean 14.2; 95% CI, 12.8 to 15.7; 78.9% correct) and Intern B (mean 12.2; 95% CI, 10.5 to 13.9; 67.8% correct) significantly improved their knowledge scores compared to prior, though Intern A had a more marked effect size (Intern A mean difference 5.02; 95% CI, 2.84 to 7.21; P<0.0001; Intern B mean difference 2.66; 95% CI, 0.660 to 4.66; P=0.006). Intern A confidence significantly improved from the pre-test in both the identification and management of HVS/LS and SVC syndrome. Intern A confidence was significantly better than Intern B confidence in management of SVC syndrome at the time of the post-test (Table 1). Conclusions Baseline intern OE knowledge scores were sub-optimal. Intern participation in a remote OE curriculum improved knowledge acquisition to a greater degree than experiential learning alone. Confidence in identification and management of OEs with low pre-existing confidence, such as HVS/LS and SVC syndrome, improved in the remote curriculum. Further study should evaluate knowledge retention over time, resident clinical performance, and patient outcomes.

How does current paediatrics residency selection criteria correlate with residency performance?

The selection process for potential residents needs to be reviewed regularly and assessed if effective in selecting the best-fit residents who can achieve academic and professional excellence. Objective measures must take precedence over subjective criteria to reduce selection bias while ensuring transparency and accountability. However, the predictors of an ideal resident and his/her performance during residency training have been a great challenge to identify as part of the selection process. The use of examination results from medical school examination, licensing examinations such as the United States Medical Licensing Examination,1,2,3 and structured interviews4 was reported to correlate positively with doctor’s performances. A Canadian study also reported that the presence of scholarly activity did not affect match outcome, though this is variable for different programmes.5 Competitive programmes like paediatrics have a vested interest in selecting the most suitable applicants who will excel as paediatric residents and emerge as holistic, high-performing paediatricians in their field.6

A Prospective Study Measuring Resident and Faculty Contour Concordance: A Potential Tool for Quantitative Assessment of Residents’ Performance in Contouring and Target Delineation in Radiation Oncology Residency

Purpose/Objective(s)Accurate target delineation (ie, contouring) is essential for radiation treatment planning and radiotherapy efficacy. As a result, improving the quality of target delineation is an important goal in the education of radiation oncology residents. The purpose of this study was to track the concordance of radiation oncology residents’ contours with those of faculty physicians over the course of 1 year to assess for patterns. Materials/MethodsResidents in postgraduate year (PGY) levels 2 to 4 were asked to contour target volumes that were then compared to the finalized, faculty physician–approved contours. Concordance between resident and faculty physician contours was determined by calculating the Jaccard concordance index (JCI), ranging from 0, meaning no agreement, to 1, meaning complete agreement. Multivariate mixed-effect models were used to assess the association of JCI to the fixed effect of PGY level and its interactions with cancer type and other baseline characteristics. Post hoc means of JCI were compared between PGY levels after accounting for multiple comparisons using Tukey’s method. ResultsIn total, 958 structures from 314 patients collected during the 2020-2021 academic year were studied. The mean JCI was 0.77, 0.75, and 0.61 for the PGY-4, PGY-3, and PGY-2 levels, respectively. The JCI score for PGY-2 was found to be lower than those for PGY-3 and PGY-4, respectively (all P < .001). No statistically significant difference of JCI score was found between the PGY-3 and PGY-4 levels. The average JCI score was lowest (0.51) for primary head and/or neck cancers, and it was highest (0.80) for gynecologic cancers. ConclusionsTracking and comparing the concordance of resident contours with faculty physician contours is an intriguing method of assessing resident performance in contouring and target delineation and could potentially serve as a quantitative metric, which is lacking currently, in radiation oncology resident evaluation. However, additional study is necessary before this technique can be incorporated into residency assessments.

Impact of deep learning on radiologists and radiology residents in detecting breast cancer on CT: a cross-vendor test study

To investigate the effect of deep learning on the diagnostic performance of radiologists and radiology residents in detecting breast cancers on computed tomography (CT). In this retrospective study, patients undergoing contrast-enhanced chest CT between January 2010 and December 2020 using equipment from two vendors were included. Patients with confirmed breast cancer were categorised as the training (n=201) and validation (n=26) group and the testing group (n=30) using processed CT images from either vendor. The trained deep-learning model was applied to test group patients with (30 females; mean age=59.2±15.8 years) and without (19 males, 21 females; mean age=64±15.9 years) breast cancer. Image-based diagnostic performance of the deep-learning model was evaluated with the area under the receiver operating characteristic curve (AUC). Two radiologists and three radiology residents were asked to detect malignant lesions by recording a four-point diagnostic confidence score before and after referring to the result from the deep-learning model, and their diagnostic performance was evaluated using jackknife alternative free-response receiver operating characteristic analysis by calculating the figure of merit (FOM). The AUCs of the trained deep-learning model on the validation and test data were 0.976 and 0.967, respectively. After referencing with the result of the deep learning model, the FOMs of readers significantly improved (reader 1/2/3/4/5: from 0.933/0.962/0.883/0.944/0.867 to 0.958/0.968/0.917/0.947/0.900; p=0.038). Deep learning can help radiologists and radiology residents detect breast cancer on CT.

Sex influences how attendings evaluate surgical residents in the operating room

BackgroundThe aim of this study was to evaluate the influence of sex on facultys’ perception of resident autonomy and performance. MethodsAutonomy/performance/complexity evaluations performed by faculty of categorical general surgery residents (2015–2021) were analyzed. Comparisons of scores by faculty and resident sex were performed. ResultsA total of 10967 paper/electronic evaluations were collected. Female attendings rated female residents significantly lower in autonomy when compared to males (2.75 vs 2.91, p ​= ​0.0037). There was no significant difference in autonomy ratings for male versus female residents when evaluated by a male attending (2.93 vs 2.96, p ​= ​0.054) but male attendings did rate female residents significantly lower in autonomy at the highest complexities (2.37 vs 2.50, p ​= ​0.012). ConclusionThe data suggests a unique interaction between attending and resident sex. A periodic evaluation of evaluations within one's program may provide invaluable implicit bias insight and should be considered.

Influence of the embedded participant on learners’ performance during high-fidelity simulation sessions in healthcare

BackgroundThe embedded participant (EP) plays a key role during a full scale/high-fidelity simulation (HFS) session. He/she is expected to guide the learner towards the achievement of the educational objectives of the scenario. However, his/her influence on learners’ performance stands undetermined and this effect remains briefly addressed in the literature. This study primarily aims to assess whether the EP could have an influence on the performance of the learner during a HFS scenario. The secondary aim was to establish an inventory of the EP practices in France.MethodsThis retrospective study was conducted in Lyon Claude Bernard University Health Simulation Centre (France). Anaesthesia and critical care residents in postgraduate years 1 to 5 who were scheduled for their HFS sessions during the 2016 to 2021 academic years were included. Two investigators independently evaluated the resident performance regarding both technical and non-technical skills from video recordings. In addition, a nationwide survey was sent out by email through the networks of the Francophone Healthcare Simulation Society (SoFraSimS, Société Francophone de Simulation en Santé) to collect information on EP practices in French-speaking Simulation centres.ResultsFrom a total of 344 HFS videos analysed, a cohort of 19 experienced EPs was identified. The EPs had an influence on the technical and non-technical performances of the learners. The 147 responses to the survey showed that predefined rules of EP good practice existed in only 36% of the simulation centres and 65% of respondents believed that specific EP training would be justified.ConclusionThe EP can exert an influence on the performance of the learners during HFS. For acting as an EP, a great variability of practices and a lack of training are reported by professionals working in simulation centres. These results suggest that more attention must be paid to EP training and attitudes during simulation, especially if summative simulations are extensively developed.

A survey of knowledge, attitude, and performance of Kerman residents on segregation and recycling of household solid wastes during COVID-19 period

Background: Awareness, knowledge, and attitude towards waste management, environmental factors of waste disposal sites and citizens’ as well as waste generators’ behaviors should be considered for implementing a waste segregation plan at the source. Methods: The statistical population of this cross-sectional study includes 150 000 households living in Kerman. The sample size is calculated as 444 individuals using a stratified sampling method considering a 15% withdrawal probability. The correlation between Kerman citizens’ demographic data (age, gender, mean income, and household head’s educational level) and behavioral patterns (knowledge and awareness, attitude and behavior) regarding waste management and environmental factors of waste disposal sites was evaluated using SPSS software. Results: Examining the variables indicates that attitude, environmental factors, behavior, awareness, and knowledge have the highest mean and standard deviation, respectively. The highest mean score of awareness and knowledge was observed among individuals who earned 40 million IRR and higher income (2.55±0.24) and those who had a PhD degree (2.63±0.27). The results show that one unit increase in the score of awareness and knowledge, attitude and environmental factors could increase the mean behavior score. Conclusion: Teaching the households waste-related environmental issues and how to properly separate recyclable waste as well as providing correct information regarding the current (COVID-19 period) and future plans and policies of the waste management organization, and developing suitable motivational mechanisms could improve the performance level of a society.

Enhancing residents’ neonatal resuscitation competency through team-based simulation training: an intervention educational study

BackgroundNeonatal resuscitation training in a simulated delivery room environment is a new paradigm in pediatric medical education. The purpose of this research is to highlight team-based simulation as an effective method of teaching neonatal resuscitation to senior pediatric residents.MethodsIn an intervention educational study, we evaluated the impact of team-based simulation training in the development of neonatal resuscitation. A team consisting of a three-person group of senior pediatric residents performed neonatal resuscitation on a low-fidelity newborn simulator based on the stated scenario. Video-based structured debriefing was performed and followed by the second cycle of scenario and debriefing to evaluate the feasibility of conducting team-based simulation training in a lesser-resourced environment. Evaluation criteria included megacode scores which is a simulation performance checklist, pre-and post-test scores to evaluate residents’ knowledge and confidence, the survey checklist as a previously developed questionnaire assessing residents’ satisfaction, and debriefing from live and videotaped performances. Four months after the end of the training course, we measured the behavioral changes of the residents by conducting an OSCE test to evaluate post-training knowledge retention. Mean ± SD was calculated for megacode, satisfaction (survey checklist), and OSCE scores. Pre- and post-program gains were statistically compared. The first three levels of Kirkpatrick’s training effectiveness model were used to evaluate the progress of the program.ResultsTwenty-one senior residents participated in the team-based simulation. The mean ± SD of the megacode score was 35.6 ± 2.2. The mean ± SD of the overall satisfaction score for the evaluation of the first level of the Kirkpatrick model was 96.3 ± 3.7. For the evaluation of the second level of the Kirkpatrick model, the pre-posttest gain in overall confidence score had a statistically significant difference (P = 0.001). All residents obtained a passing grade in OSCE as an evaluation of the third level.ConclusionsTeam-based simulation training in neonatal resuscitation improves the knowledge, skills, and performance of pediatric residents and has a positive effect on their self-confidence and leadership skills. There is still a need to investigate the transfer of learning and abilities to real-life practice, and further research on cost-effectiveness and impact on patient outcomes is warranted.

Development of a CFD model indicating the quantitative relationship among reactor dimension, bed flow unevenness, and performance for VOCs biofilters

ABSTRACT This study presents a Computational Fluid Dynamics (CFD) based biofiltration model to investigate the airflow distribution and the impact of bed flow unevenness (BFU) on the removal of Volatile Organic Compounds (VOCs) in biofilters. The biofiltration model consists of a gas flow sub-model and a VOCs removal sub-model, which were validated by pilot-scale experiments. The model was used to examine the quantitative relationship among reactor dimensions, including the width to height ratio of the filter bed and empty bed residence time (EBRT), BFU, and performance for VOCs biofilters. Simulation results demonstrate that the flow unevenness index (FUI) of the packing layer changes from 0.06 to 0.48 m2‧s−1 with reactor dimension changes. With an increase in the width to height ratio at a constant EBRT, FUI increases, BFU changes, and flow velocity fluctuation on the cross-section becomes larger, leading to a reduction of about 10% in VOCs removal efficiency. Concentration distribution of VOCs becomes uneven in the horizontal direction. At a constant width to height ratio of the filter bed, an increase in EBRT causes an increase in FUI, leading to a decrease in VOCs removal efficiency. When the width to height ratio is 0.5, velocity fluctuation of filter bed cross-section is small, the concentration of VOCs decreases evenly across the filter bed layer, and FUI is at a low level (0.06–0.11 m2‧s−1). Implication: In this manuscript, a biofiltration model of VOCs biofilter based on CFD has constructed and validated. And the manuscript gave the quantitative relationship among reactor dimension, bed flow unevenness and performance for VOCs biofilters for the first time. This study can lead to enhanced VOCs removal efficiency and improved overall performance of biofilters in practical engineering applications.

Assessing Endoscopic Suture Performance of Gynecology and Obstetrics Residents Following Methodic Training.

To evaluate the performance of residents in gynecology and obstetrics before and after practicing laparoscopic sutures, to establish when the training shows the best results, in addition to comparing whether being in different years of residency influences this progression. A prospective cohort study involving 32 medical residents evaluated with a pretest to establish their previous knowledge in laparoscopic suture. This test consisted of knotting two wires, one made of polypropylene and the other of polyglactin, with a blocking sequence of five semi-knots. We set a 30-minute limit to complete the task. Then, the residents held four training meetings, focusing on suture, Gladiator rule, knot, and symmetries, in addition to executing blocking sequences. A second test to establish progress was performed. Regarding the time spent to make the stiches using polyglactin wire, a statistically significant time improvement (p < 0.01) was observed, with a 10.67-minute pretraining median (mean 12.24 minutes) and a 2.53-minute posttraining median (mean 3.25 minutes). Regarding the stitches with polypropylene wire, a statistically significant time improvement (p < 0.05) was also observed, with a 9.38-minute pretraining median (mean 15.43 minutes) and a 3.65-minute posttraining median (mean 4.54 minutes). A total of 64.2% of the residents had been able to make the knot with polypropylene previously. One hundred percent were able to complete the task in the posttest. Model training using the Gladiator rule for laparoscopic suture improves the knotting time with statistically similar performance, regardless of the year of residency, after systematic training.

308 Barriers to Early Adoption of Novel Program to Train and Evaluate Resident Performance as Trauma Team Leaders

308 Barriers to Early Adoption of Novel Program to Train and Evaluate Resident Performance as Trauma Team Leaders

Deep Learning for Detection of Pneumothorax and Pleural Effusion on Chest Radiographs: Validation Against Computed Tomography, Impact on Resident Reading Time, and Interreader Concordance.

To study the performance of artificial intelligence (AI) for detecting pleural pathology on chest radiographs (CXRs) using computed tomography as ground truth. Retrospective study of subjects undergoing CXR in various clinical settings. Computed tomography obtained within 24 hours of the CXR was used to volumetrically quantify pleural effusions (PEfs) and pneumothoraxes (Ptxs). CXR was evaluated by AI software (INSIGHT CXR; Lunit) and by 3 second-year radiology residents, followed by AI-assisted reassessment after a 3-month washout period. We used the area under the receiver operating characteristics curve (AUROC) to assess AI versus residents' performance and mixed-model analyses to investigate differences in reading time and interreader concordance. There were 96 control subjects, 165 with PEf, and 101 with Ptx. AI-AUROC was noninferior to aggregate resident-AUROC for PEf (0.82 vs 0.86, P < 0.001) and Ptx (0.80 vs 0.84, P = 0.001) detection. AI-assisted resident-AUROC was higher but not significantly different from the baseline. AI-assisted reading time was reduced by 49% (157 vs 80 s per case, P = 0.009), and Fleiss kappa for Ptx detection increased from 0.70 to 0.78 ( P = 0.003). AI decreased detection error for PEf (odds ratio = 0.74, P = 0.024) and Ptx (odds ratio = 0.39, P < 0.001). Current AI technology for the detection of PEf and Ptx on CXR was noninferior to second-year resident performance and could help decrease reading time and detection error.

105 Intersectional Inequities in Emergency Medicine Resident Performance Assessments by Race and Sex

105 Intersectional Inequities in Emergency Medicine Resident Performance Assessments by Race and Sex

The effects of resident work hours on well-being, performance, and education: A review from a Japanese perspective.

This article examines the impact of working-hour restrictions on the well-being, performance, and education of medical residents in Japan. Despite Japan's plan to introduce new regulations for resident working hours by 2024, there is still an ongoing debate regarding their appropriateness. This review provides a comprehensive overview of the current regulations of resident working hours worldwide, with a specific focus on weekly hours. The varying regulations are highlighted, including the 80-hour-per-week regulation in the United States and the 48-h-per-week regulation in the European Union influencing other regions. The article also discusses the effectiveness of working-hour restrictions on residents' mental health, with shorter working hours having potentially greater benefits. However, the impacts on medical safety and resident education are mixed, and further reduction in working hours must be carefully considered to avoid adverse effects. The planned changes to working-hour limits for residents in Japan offer a unique opportunity to gain new evidence on the impact of such regulations, which will be of interest to policymakers and researchers worldwide.

Intersectional Disparities in Emergency Medicine Residents’ Performance Assessments by Race, Ethnicity, and Sex

Previous studies have demonstrated sex-specific disparities in performance assessments among emergency medicine (EM) residents. However, less work has focused on intersectional disparities by ethnoracial identity and sex in resident performance assessments. To estimate intersectional sex-specific ethnoracial disparities in standardized EM resident assessments. This retrospective cohort study used data from the Association of American Medical Colleges and the Accreditation Council for Graduate Medical Education Milestones (Milestones) assessments to evaluate ratings for EM residents at 128 EM training programs in the US. Statistical analyses were conducted in June 2020 to January 2023. Training and assessment environments in EM residency programs across comparison groups defined by ethnoracial identity (Asian, White, or groups underrepresented in medicine [URM], ie, African American/Black, American Indian/Alaska Native, Hispanic/Latine, and Native Hawaiian/Other Pacific Islander) and sex (female/male). Mean Milestone scores (scale, 0-9) across 6 core competency domains: interpersonal and communications skills, medical knowledge, patient care, practice-based learning and improvement, professionalism, and system-based practice. Overall assessment scores were calculated as the mean of the 6 competency scores. The study sample comprised 128 ACGME-accredited programs and 16 634 assessments for 2708 EM residents of which 1913 (70.6%) were in 3-year and 795 (29.4%) in 4-year programs. Most of the residents were White (n = 2012; 74.3%), followed by Asian (n = 477; 17.6%), Hispanic or Latine (n = 213; 7.9%), African American or Black (n = 160; 5.9%), American Indian or Alaska Native (n = 24; 0.9%), and Native Hawaiian or Other Pacific Islander (n = 4; 0.1%). Approximately 14.3% (n = 386) and 34.6% (n = 936) were of URM groups and female, respectively. Compared with White male residents, URM female residents in 3-year programs were rated increasingly lower in the medical knowledge (URM female score, -0.47; 95% CI, -0.77 to -0.17), patient care (-0.18; 95% CI, -0.35 to -0.01), and practice-based learning and improvement (-0.37; 95% CI, -0.65 to -0.09) domains by postgraduate year 3 year-end assessment; URM female residents in 4-year programs were also rated lower in all 6 competencies over the assessment period. This retrospective cohort study found that URM female residents were consistently rated lower than White male residents on Milestone assessments, findings that may reflect intersectional discrimination in physician competency evaluation. Eliminating sex-specific ethnoracial disparities in resident assessments may contribute to equitable health care by removing barriers to retention and promotion of underrepresented and minoritized trainees and facilitating diversity and representation among the emergency physician workforce.

Assessment of Resident and AI Chatbot Performance on the University of Toronto Family Medicine Residency Progress Test: Comparative Study.

Large language model (LLM)-based chatbots are evolving at an unprecedented pace with the release of ChatGPT, specifically GPT-3.5, and its successor, GPT-4. Their capabilities in general-purpose tasks and language generation have advanced to the point of performing excellently on various educational examination benchmarks, including medical knowledge tests. Comparing the performance of these 2 LLM models to that of Family Medicine residents on a multiple-choice medical knowledge test can provide insights into their potential as medical education tools. This study aimed to quantitatively and qualitatively compare the performance of GPT-3.5, GPT-4, and Family Medicine residents in a multiple-choice medical knowledge test appropriate for the level of a Family Medicine resident. An official University of Toronto Department of Family and Community Medicine Progress Test consisting of multiple-choice questions was inputted into GPT-3.5 and GPT-4. The artificial intelligence chatbot's responses were manually reviewed to determine the selected answer, response length, response time, provision of a rationale for the outputted response, and the root cause of all incorrect responses (classified into arithmetic, logical, and information errors). The performance of the artificial intelligence chatbots were compared against a cohort of Family Medicine residents who concurrently attempted the test. GPT-4 performed significantly better compared to GPT-3.5 (difference 25.0%, 95% CI 16.3%-32.8%; McNemar test: P<.001); it correctly answered 89/108 (82.4%) questions, while GPT-3.5 answered 62/108 (57.4%) questions correctly. Further, GPT-4 scored higher across all 11 categories of Family Medicine knowledge. In 86.1% (n=93) of the responses, GPT-4 provided a rationale for why other multiple-choice options were not chosen compared to the 16.7% (n=18) achieved by GPT-3.5. Qualitatively, for both GPT-3.5 and GPT-4 responses, logical errors were the most common, while arithmetic errors were the least common. The average performance of Family Medicine residents was 56.9% (95% CI 56.2%-57.6%). The performance of GPT-3.5 was similar to that of the average Family Medicine resident (P=.16), while the performance of GPT-4 exceeded that of the top-performing Family Medicine resident (P<.001). GPT-4 significantly outperforms both GPT-3.5 and Family Medicine residents on a multiple-choice medical knowledge test designed for Family Medicine residents. GPT-4 provides a logical rationale for its response choice, ruling out other answer choices efficiently and with concise justification. Its high degree of accuracy and advanced reasoning capabilities facilitate its potential applications in medical education, including the creation of exam questions and scenarios as well as serving as a resource for medical knowledge or information on community services.

Burnout and fatigue amongst internal medicine residents: A cross-sectional study on the impact of alternative scheduling models on resident wellness.

Fatigue and burnout are prevalent among resident physicians across Canada. Shifts exceeding 24 hours are commonly purported as detrimental to resident health and performance. Residency training programs have employed strategies towards understanding and intervening upon the complex issue of resident fatigue, where alternative resident scheduling models have been an area of active investigation. This study sought to characterize drivers and outcomes of fatigue and burnout amongst internal medicine residents across different scheduling models. We conducted cross-sectional surveys were among internal medicine resident physicians at the University of Alberta. We collected anonymized socioeconomic demographics and medical education background, and estimated associations between demographic or work characteristics and fatigue and burnout outcomes. Sixty-nine participants competed burnout questionnaires, and 165 fatigue questionnaires were completed (response rate of 48%). The overall prevalence of burnout was 58%. Lower burnout prevalence was noted among respondents with dependent(s) (p = 0.048), who identified as a racial minority (p = 0.018), or completed their medical degree internationally (p = 0.006). The 1-in-4 model was associated with the highest levels of fatigue, reported increased risk towards personal health (OR 4.98, 95%CI 1.77-13.99) and occupational or household harm (OR 5.69, 95%CI 1.87-17.3). Alternative scheduling models were not associated with these hazards. The 1-in-4 scheduling model was associated with high rates of resident physician fatigue, and alternative scheduling models were associated with less fatigue. Protective factors against fatigue are best characterized as strong social supports outside the workplace. Further studies are needed to characterize the impacts of alternative scheduling models on resident education and patient safety.

Comparison of Cardiac Auscultation Features on Four Different Simulation Mannequins Performed by Pediatric Residents.

Cardiac murmurs are a common problem in pediatric clinical practice. Studies demonstrated low accuracy in detecting and diagnosing various cardiac murmurs at all levels of medical training. So, supplementary training methods started to evolve, including simulation for auscultation skills training. Over the years, mannequins have evolved with different types of technology. Therefore, we decided to compare cardiac auscultation accuracy among high-fidelity mannequins as the primary objective and compare the performance of various postgraduate-level residents as a secondary objective. Pediatric residents at King Abdulaziz University Hospital were given a lecture on the basics of cardiac auscultation and then requested to auscultate four mannequins, namely SimJumior® (Laerdal Medical, Stavanger, Norway), SimBaby™ (Laerdal Medical), Pediatric HAL® (Gaumard Scientific, Miami, FL, USA), and Cardiac Patient Simulator K-Plus (Kyoto Kagaku Co. Ltd., Kyoto, Japan). The accuracies of murmur type, diagnosis, and auscultation time were compared. Results: A total of 56 pediatric residents were enrolled. Median murmur accuracy ranged from 50% to 53% (p-value 0.79), and median diagnosis accuracy ranged from 33% to 36% (p-value 0.77), with a nonsignificant difference between mannequins. Comparing resident levels in all mannequins, median murmur accuracy ranged from 49% to 56% (p-value 0.70), and median diagnosis accuracy ranged from 29% to 41% (p-value 0.09). While the median average auscultation time was between 41 and 50 seconds (p-value 0.34). Auscultation skills can be taught through simulation on any mannequin used in this comparison, not necessarily the cardiac one. For better accuracy, future comparisons might include more advanced cardiac mannequins based on cardiac auscultation expertise (i.e., consultant level). The introduction of an auscultation program from the undergraduate level throughout the training process and monitoring of these skills are mandated.

Telemedicine-Guided Two-Incision Lower Leg Fasciotomy Performed by Combat Medics During Tactical Combat Casualty Care: A Feasibility Study.

During tactical combat casualty care, life- and limb-saving procedures might also be performed by combat medics. This study assesses whether it is feasible to use a head-mounted display (HMD) to provide telemedicine (TM) support from a consulted senior surgeon for combat medics when performing a two-incision lower leg fasciotomy. Nine combat medics were randomized into groups to perform a two-incision lower leg fasciotomy. One group used the Vuzix M400 and the second group used the RealWear HMT-1Z1. A third, control, group received no guidance. In the Vuzix M400 group and RealWear HMT-1Z1 group, a senior surgeon examined the results after the two-incision lower leg fasciotomy was finished to assess the release of compartments, possible collateral damage, and performance of the combat medics. In the control group, these results were examined by a surgical resident with expertise in two-incision lower leg fasciotomies. The resident's operative performance questionnaire was used to score the performance of the combat medics. The telehealth usability questionnaire was used to evaluate the usability of the HMDs as perceived by the combat medics. Combat medics using an HMD were considered competent in performing a two-incision lower leg fasciotomy (Vuzix: median 3 [range 0], RealWear: median 3 [range 1]). These combat medics had a significantly better score in their ability to adapt to anatomical variances compared to the control group (Vuzix: median 3 [range 0], RealWear: median 3 [range 0], control: median 1 [range 0]; P = .018). Combat medics using an HMD were faster than combat medics in the control group (Vuzix: mean 14:14 [SD 3:41], RealWear: mean 15:42 [SD 1:58], control: mean 17:45 [SD 2:02]; P = .340). The overall satisfaction with both HMDs was 5 out of 7 (Vuzix: median 5 [range 0], RealWear: median 5 [range 1]; P = .317). This study shows that it is feasible to use an HMD to provide TM support performance from a consulted senior surgeon for combat medics when performing a two-incision lower leg fasciotomy. The results of this study suggest that TM support might be useful for combat medics during tactical combat casualty care when performing life- and limb-saving procedures.