In-service Exam Research Articles

Association Between Resident In-Service Exam Scores by Postgraduate Year and Subsequent Board Qualifying Exam.

Prior work notes the AUA In-Service Exam (ISE) percentile ranking of chief residents correlates with the American Board of Urology Qualifying Exam (QE) performance. We present a 5-year analysis of resident performance on the ISE and subsequent QE to determine if earlier time points in training may identify those needing additional educational support. Participant ISE scores over a 5-year period from 2014 to 2018 and subsequent QE scores in 2019 were recorded. Pearson's correlation coefficient measured the association between percentage questions correct for each ISE year and QE. Youden Index calculated the optimal cut point for yearly ISE percentage correct that would predict scoring greater than the lowest quartile and decile on the QE. Median percent questions correct on ISE increased over postgraduate year (PGY) 1 (47%), PGY2 (56.5%), and PGY3 (70%) but remained stable thereafter (PGY4-PGY5) at approximately 70%. Median QE percent correct in 2019 was 66% (SD 7.6%). Correlation of percent questions correct between ISE and QE improved from 0.31 to 0.53 over training duration. The lowest decile and quartile percent correct scores on the QE were 56% and 60%, respectively. Percent correct ISE score predicting performance above the lowest decile 2019 QE score increased from 38% in PGY1 to 57% in PGY2 and leveled off after PGY3 (∼70%). Similar observations were noted with lowest quartile QE score. Scoring approximately 70% of questions correct on the ISE during PGY3 and later years was associated with a low risk of failing the QE. Such information provides benchmarks for residency programs to offer targeted educational content for at-risk candidates.

Prompt engineering to increase GPT3.5’s performance on the Plastic Surgery In-Service Exams

This study assesses ChatGPT's (GPT-3.5) performance on the 2021 ASPS Plastic Surgery In-Service Examination using prompt modifications and Retrieval Augmented Generation (RAG). ChatGPT was instructed to act as a "resident," "attending," or "medical student," and RAG utilized a curated vector database for context. Results showed no significant improvement, with the "resident" prompt yielding the highest accuracy at 54%, and RAG failing to enhance performance, with accuracy remaining at 54.3%. Despite appropriate reasoning when correct, ChatGPT's overall performance fell in the 10th percentile, indicating the need for fine-tuning and more sophisticated approaches to improve AI's utility in complex medical tasks.

Performance of three artificial intelligence (AI)-based large language models in standardized testing; implications for AI-assisted dental education.

The emerging rise in novel computer technologies and automated data analytics has the potential to change the course of dental education. In line with our long-term goal of harnessing the power of AI to augment didactic teaching, the objective of this study was to quantify and compare the accuracy of responses provided by ChatGPT (GPT-4 and GPT-3.5) and Google Gemini, the three primary large language models (LLMs), to human graduate students (control group) to the annual in-service examination questions posed by the American Academy of Periodontology (AAP). Under a comparative cross-sectional study design, a corpus of 1312 questions from the annual in-service examination of AAP administered between 2020 and 2023 were presented to the LLMs. Their responses were analyzed using chi-square tests, and the performance was juxtaposed to the scores of periodontal residents from corresponding years, as the human control group. Additionally, two sub-analyses were performed: one on the performance of the LLMs on each section of the exam; and in answering the most difficult questions. ChatGPT-4 (total average: 79.57%) outperformed all human control groups as well as GPT-3.5 and Google Gemini in all exam years (p < .001). This chatbot showed an accuracy range between 78.80% and 80.98% across the various exam years. Gemini consistently recorded superior performance with scores of 70.65% (p = .01), 73.29% (p = .02), 75.73% (p < .01), and 72.18% (p = .0008) for the exams from 2020 to 2023 compared to ChatGPT-3.5, which achieved 62.5%, 68.24%, 69.83%, and 59.27% respectively. Google Gemini (72.86%) surpassed the average scores achieved by first- (63.48% ± 31.67) and second-year residents (66.25% ± 31.61) when all exam years combined. However, it could not surpass that of third-year residents (69.06% ± 30.45). Within the confines of this analysis, ChatGPT-4 exhibited a robust capability in answering AAP in-service exam questions in terms of accuracy and reliability while Gemini and ChatGPT-3.5 showed a weaker performance. These findings underscore the potential of deploying LLMs as an educational tool in periodontics and oral implantology domains. However, the current limitations of these models such as inability to effectively process image-based inquiries, the propensity for generating inconsistent responses to the same prompts, and achieving high (80% by GPT-4) but not absolute accuracy rates should be considered. An objective comparison of their capability versus their capacity is required to further develop this field of study.

Assessment of Artificial Intelligence Performance on the Otolaryngology Residency In-Service Exam.

This study seeks to determine the potential use and reliability of a large language learning model for answering questions in a sub-specialized area of medicine, specifically practice exam questions in otolaryngology-head and neck surgery and assess its current efficacy for surgical trainees and learners. All available questions from a public, paid-access question bank were manually input through ChatGPT. Outputs from ChatGPT were compared against the benchmark of the answers and explanations from the question bank. Questions were assessed in 2 domains: accuracy and comprehensiveness of explanations. Overall, our study demonstrates a ChatGPT correct answer rate of 53% and a correct explanation rate of 54%. We find that with increasing difficulty of questions there is a decreasing rate of answer and explanation accuracy. Currently, artificial intelligence-driven learning platforms are not robust enough to be reliable medical education resources to assist learners in sub-specialty specific patient decision making scenarios.

Gynecologic Oncology fellowship recruitment in the virtual era: An evaluation of program websites and survey of applicant preferences

ObjectiveVirtual Gynecologic Oncology fellowship recruitment has altered how candidates and programs exchange information. This study analyzes programs’ web-based content and the priorities of fellowship candidates. MethodsWeb-based materials of Gynecologic Oncology fellowship programs participating in the 2022 match were reviewed. An anonymous survey was emailed to applicants. Questions assessed importance of web-based materials on a Likert scale. Respondents were asked to rank factors from most to least important in their decisions to interview and rank programs. ResultsOf the 66 programs participating in the 2022 Gynecologic Oncology fellowship match, 62 (93.9%) had accessible websites. Over one-fourth (25.8%) of program websites did not list application requirements. Most (74.2%) websites contained requests for letters of recommendation, but fewer (48.4%) specified the preferred quantity or authorship. Residency in-service exam score requirement information was present on 61.3% of websites. Of 100 applicants invited to participate, 44 returned surveys (44% response rate). The median number of programs applied to was 60 (IQR 51–65). Web-based materials most important to candidates were application requirements and deadlines, letter of recommendation details, and in-service exam requirements. Interaction with faculty and program information received during interview days were among the most important factors in decisions to rank programs. ConclusionsGynecologic Oncology fellowship applicants surveyed in this study applied to nearly all participating fellowships. The content of web-based materials varies across program websites, particularly for application requirements, which applicants indicated as the most important electronically available material. Programs should have clear application requirements and provide clinical details on their websites.

ChatGPT Is Equivalent to First-Year Plastic Surgery Residents: Evaluation of ChatGPT on the Plastic Surgery In-Service Examination.

ChatGPT is an artificial intelligence language model developed and released by OpenAI (San Francisco, CA) in late 2022. The aim of this study was to evaluate the performance of ChatGPT on the Plastic Surgery In-Service Examination and to compare it to residents' performance nationally. The Plastic Surgery In-Service Examinations from 2018 to 2022 were used as a question source. For each question, the stem and all multiple-choice options were imported into ChatGPT. The 2022 examination was used to compare the performance of ChatGPT to plastic surgery residents nationally. In total, 1129 questions were included in the final analysis and ChatGPT answered 630 (55.8%) of these correctly. ChatGPT scored the highest on the 2021 exam (60.1%) and on the comprehensive section (58.7%). There were no significant differences regarding questions answered correctly among exam years or among the different exam sections. ChatGPT answered 57% of questions correctly on the 2022 exam. When compared to the performance of plastic surgery residents in 2022, ChatGPT would rank in the 49th percentile for first-year integrated plastic surgery residents, 13th percentile for second-year residents, 5th percentile for third- and fourth-year residents, and 0th percentile for fifth- and sixth-year residents. ChatGPT performs at the level of a first-year resident on the Plastic Surgery In-Service Examination. However, it performed poorly when compared with residents in more advanced years of training. Although ChatGPT has many undeniable benefits and potential uses in the field of healthcare and medical education, it will require additional research to assess its efficacy.

Residency Program Perspective on In-Service Exams.

Residency Program Perspective on In-Service Exams.

Predictive Value of In-Service Exams.

Predictive Value of In-Service Exams.

MP25-04 UROANKI: EXAMINING THE IMPACT OF SPACED-REPETITION ON UROLOGIC SURGICAL EDUCATION

You have accessJournal of UrologyCME1 Apr 2023MP25-04 UROANKI: EXAMINING THE IMPACT OF SPACED-REPETITION ON UROLOGIC SURGICAL EDUCATION Chanan Reitblat, Stephanie Daignault-Newton, Bryn Launer, Vadim Osadchiy, Abhishek Venkataramana, Jeff Wieder, Ganesh Palapattu, and Kate Kraft Chanan ReitblatChanan Reitblat More articles by this author , Stephanie Daignault-NewtonStephanie Daignault-Newton More articles by this author , Bryn LaunerBryn Launer More articles by this author , Vadim OsadchiyVadim Osadchiy More articles by this author , Abhishek VenkataramanaAbhishek Venkataramana More articles by this author , Jeff WiederJeff Wieder More articles by this author , Ganesh PalapattuGanesh Palapattu More articles by this author , and Kate KraftKate Kraft More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000003253.04AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: UroAnki is a free, peer-reviewed digital flashcard deck that leverages spaced repetition to help trainees master the fundamentals of Urology efficiently and effectively. The aim of this pilot study is to describe the implementation of UroAnki and to provide preliminary data on its adoption and impact on junior resident education. METHODS: In May 2022, Urology faculty and residents adapted Pocket Guide to Urology into a digital flashcard deck that tests common clinical scenarios junior residents are expected to evaluate and manage. Users were surveyed 3 months after implementation regarding usage patterns, perceived impact on self-efficacy, and feedback on areas of improvement. RESULTS: 227 out of 1169 (19.4%) UroAnki users responded to the survey, of which 114 (50%) identified as junior residents (PGY1-3). 89% of junior residents reported using UroAnki in the preceding month, with 62% reporting at least once a week usage. After studying the UroAnki cards, a majority of junior residents reported feeling better prepared for their clinical responsibilities (75%), on-the-spot questions posed by senior residents and faculty (83%), and the Urology in-service exam (66%) (Figure 1a). 80% of junior residents reported that studying UroAnki cards reduced stress and anxiety levels regarding building their fund of clinical knowledge (Figure 1a). Junior residents also ranked UroAnki as the most valuable resource for learning the fundamentals of Urology as compared to other commonly used educational resources (Figure 1b). CONCLUSIONS: UroAnki has been broadly adopted by Urology trainees. Among junior residents, UroAnki improved perceived self-efficacy, reduced stress and anxiety levels, and was ranked the most valuable resource for learning the fundamentals of Urology. Source of Funding: University of Michigan Urology Resident Education and Research Fund © 2023 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 209Issue Supplement 4April 2023Page: e343 Advertisement Copyright & Permissions© 2023 by American Urological Association Education and Research, Inc.MetricsAuthor Information Chanan Reitblat More articles by this author Stephanie Daignault-Newton More articles by this author Bryn Launer More articles by this author Vadim Osadchiy More articles by this author Abhishek Venkataramana More articles by this author Jeff Wieder More articles by this author Ganesh Palapattu More articles by this author Kate Kraft More articles by this author Expand All Advertisement PDF downloadLoading ...

14. Is Plastic Surgery Resident Learning and Assessment Consistent? A Comparison of American Society of Plastic Surgery Educational Network topics, In-Service Examinations, and Residency Core Curriculums

Background: The in-service exam is the only objective measure to compare plastic surgery (PS) residents’ knowledge. Yet, there is an ongoing debate surrounding the correlation of the test to a plastic surgeon’s success. The American Society of Plastic Surgeons (ASPS) has created an Education Network (EdNet) with an in-depth curriculum, deemed a “gold standard” on which many programs model their academic didactic sessions. However, no analysis has investigated how well the in-service examination, EdNet, and core curricula across residency programs align. This study aims to quantify differences between critical PS learning modalities to inform a more unified education plan for residents. Methods: EdNet questions, accessed through ASPS, corresponding to each course subcategory were tabulated. Next, two independent reviewers and a third arbitrator assigned each question from the 2018-2022 in-service exams to the EdNet subcategories. Finally, the curriculums for fifteen residency programs were obtained, and didactics for each curriculum were assigned to EdNet subcategories. Alignment between core curricula, in-service exam questions, and EdNet resident courses was tested with Pearson’s Chi-Square and Fisher’s Exact Tests. Results: 102 EdNet subcategories were identified, corresponding to nine major courses. 2,038 questions exist across all resident courses. 1170 in-service questions and 910 didactics were assigned to EdNet subcategories. Program core curricula taught thirty subcategories at significantly different frequencies than those presented in EdNet courses. The past five in-service examinations tested 28 subcategories at significantly different frequencies than those taught in the EdNet courses. Conclusion: This study helps answer two questions in PS resident education: do programs teach, and does the in-service exam test, the gold standard? The answer, in both cases, is “somewhat,” as nearly one-third of gold standard subcategories are taught and tested at significantly different frequencies. The gold standard may also require review. For example, while anatomy courses exist for the breast and trunk, none exist for the head and neck or extremities. Similarly, EdNet does not dedicate a category to lymphedema. Therefore, there is significant room to further align teaching and standardized testing in PS education to improve resident learning and better assess resident knowledge.

The Predictive Value of the Residency AOBFP In-Service Exam, Produced and Administered by ACOFP.

Board certification is acknowledged as the mainstay for ensuring quality physician-delivered health care within medical specialties. The American College of Osteopathic Family Physicians (ACOFP) administers the American Osteopathic Board of Family Physicians' (AOBFP) In-Service Examination (ISE) to provide residents and program directors with a formative examination to assess competency and preparation for successful completion of the AOBFP certifying examination (CE). Unique assessment processes are integral to monitoring development of the osteopathic family physician throughout training and into practice, and to verify their competency for the safety and protection of the public. This study sought to investigate whether performance on the AOBFP ISE predicted performance on the AOBFP CE, and thereby successfully equipped residents to safely enter medical practice. In 2020, data from 1,893 PGY-1 through PGY-3 residents (2016-2018), whose ISE scores could be matched with scores on the AOBFP initial board CE, were analyzed for this study. Correlations among ISE administrations across 3 years of postgraduate medical education were in the mid-to-high .6 range; the ISE scores correlated with CE scores in the mid .4 to high .5 range. Less reliable measures of positive predictive value were 0.99, and sensitivity was 0.91. Results suggest that ISE administrations during residency training are effective in developing remediation strategies for subsequent successful CE performance. The inclusion of osteopathic principles in the AOBFP CE necessitates inclusion of osteopathic content in resident training exams like ISE.

Can Customized YouTube Videos Improve ABSITE Performance?

The COVID-19 pandemic decreased in-person didactic sessions in our general surgery residency. We piloted a program to develop online tutorials posted to a YouTube channel designed to prepare our residents for the ABSITE examination. Study participants were General Surgery residents in our five-year ACGME accredited general surgery residency program who took the ABSITE in-service exam between 2019 and 2021. We compared raw scores and overall programmatic percentile scores for three academic years (2018-2019, 2019-2020, and 2020-2021). From academic year 2018-2019 to 2019-2020, before introducing the YouTube teaching platform, average raw scores and percentile scores among our general surgery residents remained unchanged (raw scores 64 - 65% (p = ns), percentile scores 39 to 37% (p = ns)). However, raw ABSITE scores increased significantly after introducing the YouTube teaching platform. A one-way ANOVA showed a statistically significant difference in raw ABSITE scores for AY 2020-2021 compared to the two prior years (F(2, 30) = [1.193], P < 0.01). Mean percentile ranking scores (program-wide) increased 15% from AY 2018-2019 to AY 2020-2021 (Figure1). Teaching via an online platform such as YouTube allows residents to review material at their own pace and schedule. It is other our hypothesis is that the online YouTube material presented in the Ventura Surgery School channel contributed, in part, to this relative improvement compared to resident peers in other institutions. Teaching via an online platform could be a valuable tool to educate surgical learners in our ever-changing teaching environment.

Two-Screen Virtual Board Game Didactic for Teaching Wilderness and Environmental Medicine Topics to Emergency Medicine Residents.

This game is appropriate for medical students, interns, junior and senior residents. The COVID-19 pandemic has forced a transition from in-person to virtual learning, and educators must innovate and adapt to keep learners engaged. One way to achieve this is through gamification.1 The authors employed a novel approach to gamification of virtual learning which engaged not only learners' computers but also their mobile phones. Learners worked in teams communicating by text message to answer ABEM board-style questions and occupy sites on the virtual board. By the end of this didactic, the learner will:1. Describe the basics of the presentation of each topic listed above2. Recall the basics of management of each topic listed above3. Improve learners' preparedness for the Emergency Medicine Inservice Exam and Written Board Examination. Wilderness and environmental medicine topics were selected from the list of topics covered on the ABEM boards. Questions were then written by the authors teaching these concepts. Learners were surveyed immediately following the session using an evaluation tool containing both Likert-scale questions on quality and applicability as well as open-ended questions on strengths and areas for improvement. The response rate to this survey was 100%. Feedback was overwhelmingly positive, with 19/20 respondents rating the sessions 5/5 for effectiveness and value of teaching compared with other sessions, and 1/20 rating the session 4/5. Nineteen out of twenty respondents rated the content as "highly relevant"; 1/20 rated it as "mostly relevant." Nineteen out of twenty respondents rated the session 5/5 for being engaging and holding their attention; 1/20 rated it as somewhat engaging. Learners rated the session as highly relevant and engaging. We hypothesize that by engaging two screens and forcing learners to work together by text message, we were able to turn what would normally be a distraction (texting co-residents during resident conference) into a tool for learning. Electrical injury, lightning strike, thermal burns, inhalational injury, chemical burns, acute radiation syndrome, snake bites, scorpion envenomation, stingray envenomation, jellyfish stings, black widow spider bites, mammalian bites, rabies, murine typhus, bear encounters, decompression sickness, arterial gas embolism, drowning, dehydration, heat stroke, exercise-associated hyponatremia, frostbite, hypothermia, CO poisoning, hydrogen sulfide poisoning, giardia.

PD33-04 THE EMPIRE IN-SERVICE REVIEW SERIES: AN INNOVATIVE VIRTUAL TOOL FOR UROLOGY RESIDENT EDUCATION

INTRODUCTION AND OBJECTIVE: The outbreak of COVID-19 led to a growth in virtual learning tools. Based on feedback from a prior EMPIRE lecture series, the authors sought to develop a topic and questionbased review for urology residents in preparation for the In-Service Exam (ISE). METHODS: The authors consulted the AUA Core Curriculum for a list of high-yield ISE topics. Faculty from the New York Section were selected to host a review on areas of expertise. Residents and faculty were notified via social media and email. Each session included a hybrid review of high yield facts within a subject domain in addition to a discussion of relevant questions from prior ISEs. All lectures were conducted via Zoom and then posted to the EMPIRE YouTube channel. A de-identified survey was sent out to participants at the conclusion of the series and results were analyzed with descriptive statistics. RESULTS: There were a total of 18 lectures delivered over 9 weeks. An average of 63 viewers attended each lecture on Zoom. An average of 1130 views were accrued for each lecture on YouTube. Since publication, a total of 20,948 views were accrued on our YouTube channel. Of these, 42% were viewed on a smartphone. Of 46 responses to the survey, 92% felt that the series increased their confidence for the ISE. Additionally, 97% of respondents felt that they would recommend the series to a colleague. CONCLUSIONS: This virtual series was a successful launch of a multi-institutional topic and question-focused ISE review course. Based on our utilization data, future resident education resources should allow for asynchronous access of the material on mobile platforms. Continued effort should be put forth to optimizing and exploring new frontiers in virtual education.

MP21-12 THE IMPACT OF MALE INFERTILITY FACULTY ON UROLOGY RESIDENCY TRAINING: WHAT ARE THE EFFECTS ON FERTILITY KNOWLEDGE, IN-SERVICE EXAM SCORES, AND SURGICAL CONFIDENCE?

You have accessJournal of UrologyInfertility: Epidemiology & Evaluation I (MP21)1 Sep 2021MP21-12 THE IMPACT OF MALE INFERTILITY FACULTY ON UROLOGY RESIDENCY TRAINING: WHAT ARE THE EFFECTS ON FERTILITY KNOWLEDGE, IN-SERVICE EXAM SCORES, AND SURGICAL CONFIDENCE? Kian Asanad, David Nusbaum, John Rodman, Gerhard Fuchs, and Mary Samplaski Kian AsanadKian Asanad More articles by this author , David NusbaumDavid Nusbaum More articles by this author , John RodmanJohn Rodman More articles by this author , Gerhard FuchsGerhard Fuchs More articles by this author , and Mary SamplaskiMary Samplaski More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000002006.12AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Given the lack of consistent male infertility faculty exposure in urology residency programs, we sought to understand the impact of having a dedicated reproductive urologist (RU) faculty on resident male infertility training. We surveyed urology residents to determine the effects on self-competence in male infertility knowledge, procedures, and In-Service Exam scores. METHODS: A 42-question anonymous survey was emailed to American urology residencies assessing exposure to RU faculty in the clinic and operating room, fertility knowledge in management of non-obstructive azoospermia (NOA), and self-competence in performing infertility procedures. Fisher’s Exact test was performed to examine the association between formal microsurgical training and self-reported male infertility knowledge and procedural competence. Multivariable linear regression analysis was used to identify significant predictors of higher In-Service Exam Infertility/Sexual Medicine subscores. RESULTS: Infertility comprises ≤10% of training in 75% of respondents. Residents exposed to RU faculty are more likely to self-rate their infertility understanding as “excellent” or “good” (p=0.04 and p=0.02, respectively) and 14.4-times more likely to feel confident performing infertility procedures, versus residents lacking faculty (p<0.001). Residents with formal microsurgical training have better self-rated global infertility understanding (p<0.001), NOA management (p=0.01), and competency performing infertility procedures (p<0.00). Those that rate their knowledge of NOA management as ‘excellent’ scored, on average, 65.8 points higher on the 2019 In-Service Exam Infertility/Sexual Medicine section than those that rate their knowledge as ‘awful’ (p-value=0.007 and <0.001, respectively). CONCLUSIONS: Infertility comprises a minority of urology residency training. Most residents (80.8%) plan to perform infertility procedures in practice, despite two-thirds lacking confidence performing these. Having an RU faculty and formal microsurgical training significantly improves residents’ surgical confidence, NOA management, and global understanding of male infertility. Better knowledge in management of NOA correlates with higher In-Service Exam Infertility/Sexual Medicine subscores. A structured educational curriculum may improve resident male infertility training. Source of Funding: None © 2021 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 206Issue Supplement 3September 2021Page: e350-e351 Advertisement Copyright & Permissions© 2021 by American Urological Association Education and Research, Inc.MetricsAuthor Information Kian Asanad More articles by this author David Nusbaum More articles by this author John Rodman More articles by this author Gerhard Fuchs More articles by this author Mary Samplaski More articles by this author Expand All Advertisement Loading ...

MP12-02 THE NORTHWESTERN UNIVERSITY NOVEL RESIDENCY CURRICULUM: THIRTEEN-YEAR EXPERIENCE WITH A FULL FIVE-YEAR UROLOGY OVERSIGHT PROGRAM

You have accessJournal of UrologyEducation Research II (MP12)1 Sep 2021MP12-02 THE NORTHWESTERN UNIVERSITY NOVEL RESIDENCY CURRICULUM: THIRTEEN-YEAR EXPERIENCE WITH A FULL FIVE-YEAR UROLOGY OVERSIGHT PROGRAM Lauren Folgosa Cooley, Dylan Isaacson, Kelly Ross, and Stephanie Kielb Lauren Folgosa CooleyLauren Folgosa Cooley More articles by this author , Dylan IsaacsonDylan Isaacson More articles by this author , Kelly RossKelly Ross More articles by this author , and Stephanie KielbStephanie Kielb More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000001985.02AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: In July 2006, the urology residency program at Northwestern introduced its novel curriculum. The curriculum emphasizes urology-directed PGY-1 and PGY-2 years, general surgery rotations determined by the program director (PD) based on educational rationale and six months of elective time. The curriculum was tested as a pilot then fully adopted and utilized for the next thirteen years. In this work, we present educational outcomes under the novel curriculum. METHODS: Rotation schedules and resident electives were recorded each year under the new curriculum. Operative case logs and AUA In-Service Examination scores were collected prospectively from 2001-2018. Residents and faculty rated satisfaction with the residency program on a five-point Likert scale from “poor” to “outstanding” from 2004-2018. Unpaired T-tests were used to compare continuous variables between the years before versus after curriculum introduction. RESULTS: From 2006-2020, residents completed 4 and 2 months of general surgery (GS) rotations in their PGY-1 and PGY-2 years, respectively. One-half of the PGY-1 GS rotations were changed for educational reasons by the PD every year, while PGY-2 rotations - transplant and advanced surgical critical care - were unchanged. Average cases logged per resident and In-Service Exam scores were comparable between groups, however residents had earlier exposure to the in-service exam as PGY-1 residents instead of PGY-2. The percentage of faculty and residents describing the program as “outstanding” increased from 50% in 2004-2005 to 82% in 2017-2018 (Figure 1). The most popular electives (total months) were plastic surgery (35.3), diagnostic radiology (33.1), international (20), medical oncology (13.3) and pathology (12.1). All residents matriculating under the new curriculum took and passed their written boards. CONCLUSIONS: After thirteen years with the novel curriculum, resident case numbers and In-Service Exam scores remained similar while faculty/resident satisfaction increased. Direct PD control of GS rotations allowed adjustments based on educational rationale. These results demonstrate that a urology-directed and flexible residency program can be instituted without compromising learner outcomes. Source of Funding: None © 2021 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 206Issue Supplement 3September 2021Page: e188-e189 Advertisement Copyright & Permissions© 2021 by American Urological Association Education and Research, Inc.MetricsAuthor Information Lauren Folgosa Cooley More articles by this author Dylan Isaacson More articles by this author Kelly Ross More articles by this author Stephanie Kielb More articles by this author Expand All Advertisement Loading ...

“Optimization of Surgical Resident Safety and Education During the COVID-19 Pandemic – Lessons Learned”

The COVID-19 pandemic has engendered rapid and significant changes in patient care. Within the realm of surgical training, the resultant reduction in clinical exposure and case volume jeopardizes the quality of surgical training. Thus, our general surgery residency program proceeded to develop a tailored approach to training that mitigates impact on resident surgical education and optimizes clinical exposure without compromising safety. Residents were engaged directly in planning efforts to craft a response to the pandemic. Following the elimination of elective cases, the in-house resident complement was effectively decreased to reduce unnecessary exposure, with a back-up pool to address unanticipated absences and needs. Personal protective equipment availability and supply, the greatest concern to residents, has remained adequate, while being utilized according to current guidelines. Interested residents were given the opportunity to work in designated COVID ICUs on a volunteer basis. With the decrease in operative volume and clinical duties, we shifted our educational focus to an intensive didactic schedule using a teleconferencing platform and targeted areas of weakness on prior in-service exams. We also highlighted critical COVID-19 literature in a weekly journal club to better understand this novel disease and its effect on surgical practice. The long-term impact of the COVID-19 pandemic on resident education remains to be seen. Success may be achieved with commitment to constant needs assessment in the changing landscape of healthcare with the goal of producing a skilled surgical workforce for public service.

Searching for an Alternative Method to Evaluate Surgical Knowledge: Does a Drawing and Labeling Quiz Predict In-Service Exam Performance and Enhance Learning?

The best methods to assess surgical knowledge are still debated. The authors used a non-multiple-choice test as a pre- and post-conference assessment to measure residents' knowledge gains with comparison to a standard summative assessment tool. At one didactic conference, plastic surgery residents at a single institution were given a pre-test of drawing and labeling structures in the extensor mechanism of the finger and within the carpal tunnel. The quiz was followed by a lecture on the same material and a subsequent post-test. Scores were correlated with in-service exam performance. Pre-test scores (n = 13) were positively correlated with postgraduate year (PGY) until PGY-3. Performance on labeling structures was higher than performance on the respective drawing prompt. Residents' ability to label structures increased more strongly with PGY than their ability to draw structures. The post-test (n = 8) demonstrated that teaching improves performance on labeling questions (pre-test score = 62%; post-test score = 87%). Improvement was observed across all PGYs. Pretest results were positively correlated with in-service exam performance. Our study suggests that a knowledge test focused on drawing and labeling structures given to surgical residents is a valid, nontraditional method for assessing resident knowledge. Such a quiz would offer programs an alternative method for regularly evaluating residents aside from in-service questions, in order to identify residents who may need targeted training for the in-service exam and to inform teaching plans.Additionally, residents could use quiz feedback to guide study efforts and prime conference-related learning.

Does the Removal of Textbook Reading from Emergency Medicine Resident Education Negatively Affect In-Service Scores?

IntroductionIn-service exam scores are used by residency programs as a marker for progress and success on board exams. Conference curriculum helps residents prepare for these exams. At our institution, due to resident feedback a change in curriculum was initiated. Our objective was to determine whether assigned Evidence-Based Medicine (EBM) articles and Rosh Review questions were non-inferior to Tintinalli textbook readings. We further hypothesized that the non-textbook assigned curriculum would lead to higher resident satisfaction, greater utilization, and a preference over the old curriculum.MethodsWe collected scores from both the allopathic In-training Examination (ITE) and osteopathic Emergency Medicine Residency In-service Exam (RISE) scores taken by our program’s residents from both the 2015–2016 and 2016–2017 residency years. We compared scores pre-curriculum change (pre-CC) to scores post-curriculum change (post-CC). A five-question survey was sent to the residents regarding their satisfaction, preference, and utilization of the two curricula.ResultsResident scores post-CC were shown to be non-inferior to their scores pre-CC for both exams. There was also no significant difference when we compared scores from each class post-CC to their respective class year pre-CC for both exams. Our survey showed significantly more satisfaction, utilization, and preference for this new curriculum among residents.ConclusionWe found question-based learning and Evidence-Based Medicine articles non-inferior to textbook readings. This study provides evidence to support a move away from textbook readings without sacrificing scores on examinations.

Trends and factors affecting the US adult hematology workforce: a mixed methods study

AbstractThe current demand for adult hematologists in the United States is projected to exceed the existing supply. However, no national study has systematically evaluated factors affecting the adult hematology workforce. In collaboration with the American Society of Hematology (ASH), we performed a mixed methods study consisting of surveys from the annual ASH In-Service Exam for adult hematology/oncology fellows from 2010 to 2016 (8789 participants); interviews with graduating or recently graduated adult hematology/oncology fellows in a single training program (8 participants); and 3 separate focus groups for hematology/oncology fellowship program directors (12 participants), fellows (12 participants), and clinicians (10 participants) at the 2016 ASH annual meeting. In surveys, the majority of fellows favored careers combining hematology and oncology, with more fellows identifying oncology, rather than hematology, as their primary focus. In interviews with advanced-year fellows, mentorship emerged as the single most important career determinant, with mentorship opportunities arising serendipitously, and oncology faculty perceived as having greater availability for mentorship than hematology faculty. In focus group discussions, hematology, particularly benign hematology, was viewed as having poorer income potential, research funding, job availability, and job security than oncology. Focus group participants invariably agreed that the demand for clinical care in hematology, particularly benign hematology, exceeded the current workforce supply. Single-subspecialty fellowship training in hematology and the creation of new clinical care models were offered as potential solutions to these workforce problems. As a next step, ASH is conducting a national, longitudinal study of the adult hematology workforce to improve recruitment and retention in the field.