In-service Examination Research Articles

Prompt engineering to increase GPT3.5’s performance on the Plastic Surgery In-Service Exams

This study assesses ChatGPT's (GPT-3.5) performance on the 2021 ASPS Plastic Surgery In-Service Examination using prompt modifications and Retrieval Augmented Generation (RAG). ChatGPT was instructed to act as a "resident," "attending," or "medical student," and RAG utilized a curated vector database for context. Results showed no significant improvement, with the "resident" prompt yielding the highest accuracy at 54%, and RAG failing to enhance performance, with accuracy remaining at 54.3%. Despite appropriate reasoning when correct, ChatGPT's overall performance fell in the 10th percentile, indicating the need for fine-tuning and more sophisticated approaches to improve AI's utility in complex medical tasks.

Predictors of standardized in-service examination performance and residency outcomes in a graduate periodontics program.

Predictors of standardized in-service examination performance and residency outcomes in a graduate periodontics program.

Analysis of the microsurgery fellowship match: 2019-2022 update.

A recent analysis of microsurgery fellowship match data published in 2019 demonstrated increased competition for available positions. With growing opportunities in the field, the authors hypothesize that the landscape for both applicants and programs has become more competitive. The aim of this study is to compare two periods of match data to inform residents and programs in microsurgery. Microsurgery fellowship match data was obtained from the San Francisco Match with approval by the American Society for Reconstructive Microsurgery for the years 2014-2022. Data were stratified into the categories of 2016-2018 and 2019-2022. Parameters assessed included: program and position fill rates, match rates, and in-service examination percentiles. Data were analyzed using Pearson's Chi-square tests and unpaired t-tests. The median number of participating programs and positions increased to 29 and 47 in 2019-2022, compared with 23 and 40 in 2016-2018. This coincided with a decrease in the number of applicants per position (1.3 [52-40] vs. 1.1 [50-47], p = .45). There was a significant increase in the match rate between groups (67.8% vs. 80.2%, p = .007). Recently, 2022 saw the lowest position fill rate on record, at 75.4% (40 of 53 positions filled), down from 85.3% (35 of 41) in 2018 (p = .35) and 95.6% (43 of 45) in 2019 (p = .006). Mean in-service examination percentiles for successfully matched applicants did not differ between (2016-2018) and (2019-2022) applicants. Recent years have seen a rise in the number of microsurgery fellowship training programs with a decline in the number of applicants. Accordingly, there has been an increased match rate for prospective applicants. Despite this, a pool of unmatched applicants and unfilled positions with training opportunities still remain. The reasons for which are likely multifactorial.

Performance of three artificial intelligence (AI)-based large language models in standardized testing; implications for AI-assisted dental education.

The emerging rise in novel computer technologies and automated data analytics has the potential to change the course of dental education. In line with our long-term goal of harnessing the power of AI to augment didactic teaching, the objective of this study was to quantify and compare the accuracy of responses provided by ChatGPT (GPT-4 and GPT-3.5) and Google Gemini, the three primary large language models (LLMs), to human graduate students (control group) to the annual in-service examination questions posed by the American Academy of Periodontology (AAP). Under a comparative cross-sectional study design, a corpus of 1312 questions from the annual in-service examination of AAP administered between 2020 and 2023 were presented to the LLMs. Their responses were analyzed using chi-square tests, and the performance was juxtaposed to the scores of periodontal residents from corresponding years, as the human control group. Additionally, two sub-analyses were performed: one on the performance of the LLMs on each section of the exam; and in answering the most difficult questions. ChatGPT-4 (total average: 79.57%) outperformed all human control groups as well as GPT-3.5 and Google Gemini in all exam years (p < .001). This chatbot showed an accuracy range between 78.80% and 80.98% across the various exam years. Gemini consistently recorded superior performance with scores of 70.65% (p = .01), 73.29% (p = .02), 75.73% (p < .01), and 72.18% (p = .0008) for the exams from 2020 to 2023 compared to ChatGPT-3.5, which achieved 62.5%, 68.24%, 69.83%, and 59.27% respectively. Google Gemini (72.86%) surpassed the average scores achieved by first- (63.48% ± 31.67) and second-year residents (66.25% ± 31.61) when all exam years combined. However, it could not surpass that of third-year residents (69.06% ± 30.45). Within the confines of this analysis, ChatGPT-4 exhibited a robust capability in answering AAP in-service exam questions in terms of accuracy and reliability while Gemini and ChatGPT-3.5 showed a weaker performance. These findings underscore the potential of deploying LLMs as an educational tool in periodontics and oral implantology domains. However, the current limitations of these models such as inability to effectively process image-based inquiries, the propensity for generating inconsistent responses to the same prompts, and achieving high (80% by GPT-4) but not absolute accuracy rates should be considered. An objective comparison of their capability versus their capacity is required to further develop this field of study.

Navigating Discordance: Assessing Varied Applications of the American Academy of Periodontology In-Service Examination in Postdoctoral Programs

Introduction The American Academy of Periodontology (AAP) In-Service Examination (ISE) is offered annually to residents in postdoctoral periodontal programs across the United States and Canada. The language in AAP published guidance supports both formative and summative uses of the exam, presenting discordance in how to use and interpret the AAPISE. It is important to clarify how programs use data, as formative assessments are low-stakes and aim to provide feedback to learners for improving their learning. On the other hand, summative assessments are used to make high-stakes decisions, such as program admission, graduation, and licensure. These two types of assessments serve different purposes and have distinct outcomes. This study aimed to explore and characterize how stakeholders utilize the AAPISE and its associated score reports. Methods Semi-structured interviews of periodontology program directors, department chairs, and residents were conducted in 2022. Data from these interviews were explored, coded, and thematically analyzed. Results A total of 16 interviews were conducted and analyzed: four chairs, eight program directors, and four residents representing the experience at 20 postdoctoral periodontal programs. Five major themes were identified regarding the use of the AAPISE: formative assessment, board preparation, program development, promoting a culture of achievement, and summative assessment. Conclusion This study's findings suggest varied uses of the AAPISE amongst postdoctoral periodontal program stakeholders, reflecting the variance within the AAP guidance. The discordant uses and guidance jeopardize the AAPISE’s potential to align with the 2018 Consensus Framework for Good Assessment elements. The authors propose recommendations to the AAP and stakeholders for future use.

Comparing Graduating Plastic Surgery Residents' Case Logs With Accreditation Council for Graduate Medical Education Requirements, Content at National Meetings, and In-Service Examination Test Items.

The importance of adaptable and up-to-date plastic surgery graduate medical education (GME) has taken on new meaning amidst accelerating surgical innovation and increasing calls for competency-based training standards. We aimed to examine the extent to which the procedures plastic surgery residents perform, as represented in case log data, align with 2 core standardized components of plastic surgery GME: ACGME (Accreditation Council for Graduate Medical Education) minimum procedure count requirements and the PSITE (Plastic Surgery In-Service Training Examination). We also examined their alignment with procedural representation at 2 major plastic surgery meetings. Nine categories of reconstructive and aesthetic procedures were identified. Three-year averages for the number of procedures completed in each category by residents graduating in 2019-2021 were calculated from ACGME national case log data reports. The ACGME procedure count minimum requirements were also ascertained. The titles and durations of medical programming sessions scheduled for Plastic Surgery The Meeting (PSTM) 2022 and the Plastic Surgery Research Council (PSRC) Annual Meeting 2022 were retrieved from online data. Finally, test items from the 2020 to 2022 administrations of the PSITE were retrieved. Conference sessions and test items were assigned to a single procedure category when possible. Percent differences were calculated for comparison. The distribution of procedures on plastic surgery resident case logs differs from those of the major mechanisms of standardization in plastic surgery GME, in-service examination content more so than ACGME requirements. Meeting content at PSTM and PSRC had the largest percent differences with case log data, with PSTM being skewed toward aesthetics and PSRC toward reconstructive head and neck surgery. The criteria and standards by which plastic surgery residents are evaluated and content at national meetings differ from the procedures they actually complete during their training. Although largely reflecting heterogeneity of the specialty, following these comparisons will likely prove useful in the continual evaluation of plastic surgery residency training, especially in the preparation of residents for the variety of training and practice settings they pursue.

Relationship Between National Residency Matching Program (NRMP) Rank Order and Otolaryngology Residency Performance.

The process of resident recruitment is costly, and our surgical residency program expends significant time on the resident selection process while balancing general duties and responsibilities. The aim of our study was to explore the relationship between otolaryngology-head and surgery (OHNS) residents' National Residency Matching Program (NRMP) rank-list position at our institution and their subsequent residency performance. Retrospective cohort study. Single site institution. We retrospectively reviewed 7 consecutive resident classes (2011-2017) at a single tertiary OHNS residency program. We reviewed each resident's absolute rank order in the NRMP matches. Measures of residency performance included overall faculty evaluation during postgraduate year 5 (PGY5), annual in-service examination scores (scaled score), and the number of manuscripts published in peer-reviewed journals. Correlations between NRMP rank order and subsequent residency performance were assessed using Spearman's rho correlation coefficients (ρ). Twenty-eight residents entered residency training between 2011 and 2017. The average rank position of the trainees during this study was 9.7 (range: 1-22). We found no significant correlation between rank order and faculty evaluation during PGY5 (ρ = 0.097, P = .625) or number of publications (ρ = -0.256, P = .189). Additionally, when assessing the association between rank order and annual Otolaryngology Training Examination-scaled scores, no statistically significant relationship was found between the 2 (P > .05). Our results showed that there were no significant correlations between OHNS rank order and various measures of success in residency training, which aligns with existing literature. Further investigation of this relationship should be conducted to ensure the applicability of our findings.

Predictors of standardized in-service examination performance and residency outcomes in a graduate periodontics program.

The objectives of this study were to assess the influence of learner- and education-related factors on standardized in-service examination performance and determine whether in-service examination scores predict residency outcomes. American Academy of Periodontology (AAP) In-service Examination (AIE) scores from 10 periodontics residency classes at a single center were recorded and compared against a panel of learner- and education-related variables using multiple linear regression models. Defined residency outcome measures were analyzed against AIE scores using binomial logistic regression. No evaluated learner- or education-related variable was a statistically significant predictor of AIE score in this study sample. Likewise, AIE score was not a statistically significant predictor of any assessed residency outcome. The AAP has performed a tremendous service to periodontics residents and programs by marshaling the leadership and expertise necessary to offer a professionally constructed assessment instrument. However, in the current study, no relationship could be identified between AIE score and any outcome, including first-attempt board certification. The AAP In-service Committee appears well situated to provide additional leadership focusing on exam implementation, which may enhance AIE value in competency decision making.

Artificial intelligence in dental education: ChatGPT's performance on the periodontic in-service examination.

ChatGPT's (Chat Generative Pre-Trained Transformer) remarkable capacity to generate human-like output makes it an appealing learning tool for healthcare students worldwide. Nevertheless, the chatbot's responses may be subject to inaccuracies, putting forth an intense risk of misinformation. ChatGPT's capabilities should be examined in every corner of healthcare education, including dentistry and its specialties, to understand the potential of misinformation associated with the chatbot's use as a learning tool. Our investigation aims to explore ChatGPT's foundation of knowledge in the field of periodontology by evaluating the chatbot's performance on questions obtained from an in-service examination administered by the American Academy of Periodontology (AAP). ChatGPT3.5 and ChatGPT4 were evaluated on 311 multiple-choice questions obtained from the 2023 in-service examination administered by the AAP. The dataset of in-service examination questions was accessed through Nova Southeastern University's Department of Periodontology. Our study excluded questions containing an image as ChatGPT does not accept image inputs. ChatGPT3.5 and ChatGPT4 answered 57.9% and 73.6% of in-service questions correctly on the 2023 Periodontics In-Service Written Examination, respectively. A two-tailed t test was incorporated to compare independent sample means, and sample proportions were compared using a two-tailed χ2 test. A p value below the threshold of 0.05 was deemed statistically significant. While ChatGPT4 showed a higher proficiency compared to ChatGPT3.5, both chatbot models leave considerable room for misinformation with their responses relating to periodontology. The findings of the study encourage residents to scrutinize the periodontic information generated by ChatGPT to account for the chatbot's current limitations.

Are We Teaching Evidence-Based and Inclusive Practices in Gender-Affirming Care? Perspectives From Plastic Surgery In-Service Examinations.

Background Resident physicians take annual in-service examinations (ISEs) as part of continuing medical education, which set educational priorities, allow for formative feedback, and guide preparation for final board examinations. Gender-affirming care is provided in many specialties but has been an underrepresented area in medical education. Plastic surgeons provide a large portion of gender-affirming surgical care. Educational gaps in standardized ISEs may contribute to ongoing health care disparities for transgender and gender diverse people. Objective To evaluate the quality of content pertaining to gender-affirming surgery (GAS) on plastic surgery ISEs. Methods Plastic surgery ISEs from years 2012 to 2020 were accessed online through the American Council of Academic Plastic Surgeons website in June 2022. All 5 gender diverse authors analyzed examinations for the presence of GAS questions; identified questions were analyzed for quantity, organization based on content category, affirming language, and accuracy against current guidelines. Results Of 1959 ISE questions available for review, 11 GAS questions were identified for a total frequency of 0.56%. Most GAS questions (6 of 11, 55%) were miscategorized. Inappropriate language, including misgendering of patients, occurred in 7 of 11 (64%) questions. No questions discussed GAS beyond chest or genital surgery, or common variations of these procedures. Transgender identities were represented as only binary, with no mention of nonbinary or gender-fluid individuals. Conclusions Our study illustrates that there are significant gaps in educational content pertaining to gender-affirming care on plastic surgery ISEs.

Evaluation of Artificial Intelligence-generated Responses to Common Plastic Surgery Questions.

Artificial intelligence (AI) is increasingly used to answer questions, yet the accuracy and validity of current tools are uncertain. In contrast to internet queries, AI generates summary responses as definitive. The internet is rife with inaccuracies, and plastic surgery management guidelines evolve, making verifiable information important. We posed 10 questions about breast implant-associated illness, anaplastic large lymphoma, and squamous carcinoma to Bing, using the "more balanced" option, and to ChatGPT. Answers were reviewed by two plastic surgeons for accuracy and fidelity to information on the Food and Drug Administration (FDA) and American Society of Plastic Surgeons (ASPS) websites. We also presented 10 multiple-choice questions from the 2022 plastic surgery in-service examination to Bing, using the "more precise" option, and ChatGPT. Questions were repeated three times over consecutive weeks, and answers were evaluated for accuracy and stability. Compared with answers from the FDA and ASPS, Bing and ChatGPT were accurate. Bing answered 10 of the 30 multiple-choice questions correctly, nine incorrectly, and did not answer 11. ChatGPT correctly answered 16 and incorrectly answered 14. In both parts, responses from Bing were shorter, less detailed, and referred to verified and unverified sources; ChatGPT did not provide citations. These AI tools provided accurate information from the FDA and ASPS websites, but neither consistently answered questions requiring nuanced decision-making correctly. Advances in applications to plastic surgery will require algorithms that selectively identify, evaluate, and exclude information to enhance the accuracy, precision, validity, reliability, and utility of AI-generated responses.

Bard Versus the 2022 American Society of Plastic Surgeons In-Service Examination: Performance on the Examination in Its Intern Year.

Bard is a conversational generative artificial intelligence (AI) platform released by Google (Mountain View, CA) to the public in May 2023. This study investigates the performance of Bard on the American Society of Plastic Surgeons (ASPS) In-Service Examination to compare it to residents' performance nationally. We hypothesized that Bard would perform best on the comprehensive and core surgical principles portions of the examination. Google's 2023 Bard was used to answer questions from the 2022 ASPS In-Service Examination. Each question was asked as written with the stem and multiple-choice options. The 2022 ASPS Norm Table was utilized to compare Bard's performance to that of subgroups of plastic surgery residents. A total of 231 questions were included. Bard answered 143 questions correctly corresponding to an accuracy of 62%. The highest-performing section was the comprehensive portion (73%). When compared with integrated residents nationally, Bard scored in the 74th percentile for post-graduate year (PGY)-1, 34th percentile for PGY-2, 20th percentile for PGY-3, 8th percentile for PGY-4, 1st percentile for PGY-5, and 2nd percentile for PGY-6. Bard outperformed more than half of the first-year integrated residents (74th percentile). Its best sections were the comprehensive and core surgical principle portions of the examination. Further analysis of the chatbot's incorrect questions might help improve the overall quality of the examination's questions.

Critical care medicine training in the age of COVID-19.

The COVID-19 pandemic caused the largest disruption to graduate medical education in modern history. The danger associated with SARS-CoV-2 necessitated a paradigm shift regarding the fundamental approach to the education of medical residents and fellows. Whereas prior work has examined the effect of the pandemic on residents' experiences during training, the effect of the pandemic on academic performance of critical care medicine (CCM) fellows is not well understood. This study examined the relationship between CCM fellow's lived experiences during the COVID-19 pandemic and performance on in-training examinations. This mixed-methods study consisted of a quantitative retrospective analysis of critical care fellows' in-training examination scores and a qualitative, interview-based phenomenological examination of fellows' experiences during the pandemic while training in a single large academic hospital in the American Midwest. Quantitative: Prepandemic (2019 and 2020) and intrapandemic (2021 and 2022) in-training examination scores were analyzed utilizing an independent samples t test to determine whether a significant change occurred during the pandemic. Qualitative: Individual semi-structured interviews were conducted with CCM fellows exploring their lived experiences during the pandemic and their perception of the effect on their academic performance. Transcribed interviews were analyzed for thematic patterns. These themes were coded and categorized, and subcategories were developed as indicated during the analysis. The identified codes were then analyzed for thematic connections and apparent patterns. Relationships between themes and categories were analyzed. This process was continued until a coherent picture could be assembled from the data to answer the research questions. Analysis was performed from a phenomenological perspective with an emphasis on interpretation of the data from the participants' perspectives. Quantitative: Fifty-one in-training examination scores from 2019 to 2022 were obtained for analysis. Scores from 2019 to 2020 were grouped as prepandemic scores, while scores from 2021 to 2022 were grouped as intrapandemic scores. Twenty-four prepandemic and 27 intrapandemic scores were included in the final analysis. A significant difference was found between mean total prepandemic and intrapandemic in-service examination scores (t 49=2.64, p=0.01), with mean intrapandemic scores being 4.5 points lower than prepandemic scores (95 % CI, 1.08-7.92). Qualitative: Interviews were conducted with eight CCM fellows. Thematic analysis of the qualitative interviews revealed three main themes: psychosocial/emotional effects, effects on training, and effects on health. The factors that most effected participants' perceptions of their training were burnout, isolation, increased workload, decreased bedside teaching, decreased formal academic training opportunities, decreased procedural experience, a lack of an external reference point for normal training in CCM, fear of spreading COVID-19, and neglect of personal health during the pandemic. In-training examination scores decreased significantly during the COVID-19 pandemic for CCM fellows in this study. The fellows in this study reported perceived effects of the pandemic on their psychosocial/emotional well-being, medical training, and health.

ChatGPT Is Equivalent to First-Year Plastic Surgery Residents: Evaluation of ChatGPT on the Plastic Surgery In-Service Examination.

ChatGPT is an artificial intelligence language model developed and released by OpenAI (San Francisco, CA) in late 2022. The aim of this study was to evaluate the performance of ChatGPT on the Plastic Surgery In-Service Examination and to compare it to residents' performance nationally. The Plastic Surgery In-Service Examinations from 2018 to 2022 were used as a question source. For each question, the stem and all multiple-choice options were imported into ChatGPT. The 2022 examination was used to compare the performance of ChatGPT to plastic surgery residents nationally. In total, 1129 questions were included in the final analysis and ChatGPT answered 630 (55.8%) of these correctly. ChatGPT scored the highest on the 2021 exam (60.1%) and on the comprehensive section (58.7%). There were no significant differences regarding questions answered correctly among exam years or among the different exam sections. ChatGPT answered 57% of questions correctly on the 2022 exam. When compared to the performance of plastic surgery residents in 2022, ChatGPT would rank in the 49th percentile for first-year integrated plastic surgery residents, 13th percentile for second-year residents, 5th percentile for third- and fourth-year residents, and 0th percentile for fifth- and sixth-year residents. ChatGPT performs at the level of a first-year resident on the Plastic Surgery In-Service Examination. However, it performed poorly when compared with residents in more advanced years of training. Although ChatGPT has many undeniable benefits and potential uses in the field of healthcare and medical education, it will require additional research to assess its efficacy.

51. Is Plastic Surgery Resident Learning and Assessment Consistent? A Comparison of American Society of Plastic Surgery Educational Network Topics, In-service Examinations, and Residency Core Curriculums

PURPOSE: The in-service examination objectively compares plastic surgery (PS) residents’ knowledge, while the ASPS’ Education Network (EdNet) is a “gold-standard” which programs use to design their curricula. This study aims to quantify the degree to which critical PS learning modalities align. METHODS: Questions from the EdNet resident courses and 2018-2022 in-service examinations were tabulated and assigned to EdNet topics. Then, curricula for fifteen PS residencies were assigned to EdNet topics. Topic alignment between curricula didactics, in-service examinations, and EdNet courses was tested with Pearson’s Chi-Square and Fisher’s Exact Tests. RESULTS: 2,038 questions corresponding to 102 topics were queried from EdNet. 1170 in-service questions and 910 curricula lectures were assigned to these topics. Program curricula taught thirty topics at significantly different frequencies than those taught by EdNet. The past five in-service examinations tested 28 topics at significantly different frequencies than those taught by EdNet. CONCLUSION: One-third of gold-standard EdNet topics are taught and tested at significantly different frequencies. However, the gold-standard itself may require review: no anatomy courses exist for the head and neck or extremities, and there is no course dedicated to lymphedema. Therefore, there is significant room to further align PS educational modalities to improve resident learning and assessment.

Detecting residents at risk of attrition – A Singapore pathology residency's experience

The SingHealth Pathology Residency Program (SHPRP) is a 5-year postgraduate training program in Singapore. We face the problem of resident attrition, which has a significant impact on the individual, program and healthcare providers. Our residents are regularly evaluated, using in-house evaluations as well as assessments required in our partnership with the Accreditation Council for Graduate Medical Education International (ACGME-I). We hence sought to determine if these assessments were able to distinguish residents who would attrite from residents who would graduate successfully. Retrospective analysis of existing residency assessments was performed on all residents who have separated from SHPRP and compared with residents currently in senior residency or graduated from the program. Statistical analysis was performed on quantitative assessment methods of Resident In-Service Examination (RISE), 360-degree feedback, faculty assessment, Milestones and our own annual departmental mock examination. Word frequency analysis of narrative feedback from faculty assessment was used to generate themes.Since 2011, 10 out of 34 residents have separated from the program. RISE, Milestone data and the departmental mock examination showed statistical significance in discriminating residents at risk of attrition for specialty-related reasons from successful residents. Analysis of narrative feedback showed that successful residents performed better in areas of organization, preparation with clinical history, application of knowledge, interpersonal communication and achieving sustained progress. Existing assessment methods used in our pathology residency program are effective in detecting residents at risk of attrition. This also suggests applications in the way that we select, assess and teach residents.

14. Is Plastic Surgery Resident Learning and Assessment Consistent? A Comparison of American Society of Plastic Surgery Educational Network topics, In-Service Examinations, and Residency Core Curriculums

Background: The in-service exam is the only objective measure to compare plastic surgery (PS) residents’ knowledge. Yet, there is an ongoing debate surrounding the correlation of the test to a plastic surgeon’s success. The American Society of Plastic Surgeons (ASPS) has created an Education Network (EdNet) with an in-depth curriculum, deemed a “gold standard” on which many programs model their academic didactic sessions. However, no analysis has investigated how well the in-service examination, EdNet, and core curricula across residency programs align. This study aims to quantify differences between critical PS learning modalities to inform a more unified education plan for residents. Methods: EdNet questions, accessed through ASPS, corresponding to each course subcategory were tabulated. Next, two independent reviewers and a third arbitrator assigned each question from the 2018-2022 in-service exams to the EdNet subcategories. Finally, the curriculums for fifteen residency programs were obtained, and didactics for each curriculum were assigned to EdNet subcategories. Alignment between core curricula, in-service exam questions, and EdNet resident courses was tested with Pearson’s Chi-Square and Fisher’s Exact Tests. Results: 102 EdNet subcategories were identified, corresponding to nine major courses. 2,038 questions exist across all resident courses. 1170 in-service questions and 910 didactics were assigned to EdNet subcategories. Program core curricula taught thirty subcategories at significantly different frequencies than those presented in EdNet courses. The past five in-service examinations tested 28 subcategories at significantly different frequencies than those taught in the EdNet courses. Conclusion: This study helps answer two questions in PS resident education: do programs teach, and does the in-service exam test, the gold standard? The answer, in both cases, is “somewhat,” as nearly one-third of gold standard subcategories are taught and tested at significantly different frequencies. The gold standard may also require review. For example, while anatomy courses exist for the breast and trunk, none exist for the head and neck or extremities. Similarly, EdNet does not dedicate a category to lymphedema. Therefore, there is significant room to further align teaching and standardized testing in PS education to improve resident learning and better assess resident knowledge.

21. Efficacy of Commonly Utilized Resources in Preparation for the Plastic Surgery In-Service Examination

Background: The Plastic Surgery In-Service examination is a critical aspect of trainee evaluation and board preparation yet is understudied1. There is currently no standardized study plan. Improved understanding of the efficacy of commonly utilized resources is useful for program and independent study plans. Purpose: This study seeks to assess what proportion of questions on the 2019-2021 examinations could be answered correctly using commonly utilized educational resources. Methods: Syllabi from previous In-Service examinations (2019-2021) were examined to determine if correct answers could be found within common educational resources consisting of: previous examination syllabi, Plastic Surgery by Neligan, Grabb and Smith’s Plastic Surgery, Green’s Operative Hand Surgery, the American Society of Plastic Surgeons (ASPS) Resident Education Curriculum (REC) and relevant cited continuing medical education (CME) articles from Plastic & Reconstructive Surgery Journal (PRS). Each of these resources were searched and categorized by whether or not the correct answer could be found within each resource. Results: A total of 750 questions underwent analysis. Overall, Neligan’s Plastic Surgery was the highest yield, comprising 57.6% of total averaged questions correct. This was followed by the REC which yielded 57.2%. The remaining resources from highest to lowest yield were as follows: Grabb & Smith’s Plastic Surgery (56.3%), previous year’s syllabi (33.7%), Green’s Operative Hand Surgery (30.1%), and CME articles (28%) respectively. When the exam was broken into subcategories (Comprehensive, Hand & Lower Extremity, Craniomaxillofacial, Breast & Cosmetic, and Core Surgical Principles), the order of highest to lowest yield did not vary significantly with the exception of the Hand & Lower Extremity section, for which Green’s Operative Hand Surgery contributed approximately 72% answers correct. Conclusion: Overall, Neligan’s Plastic Surgery and the Resident Education Curriculum were the highest yield resources for all topics averaged, with Green’s Operative Hand Surgery being the highest yield for the Hand and Lower Extremity section. Some resources, particularly the Resident Education Curriculum, may not be frequently utilized despite its high yield nature. This study serves to identify gaps in resident study material and is a tool to identify top resources when devising a study plan to achieve the best performance possible on the examination.

Resident Physician “Moonlighting” for On-call in Transfusion Medicine: Educational Impact and Preparedness for Practice

Abstract Introduction/Objective Resident physician moonlighting is controversial and uncommon in Pathology, usually limited to anatomic pathology grossing support. Due to an acute staffing shortage in our department, we offered senior AP/ CP residents in our program a “moonlighting” opportunity involving partnership with attendings to supervise junior residents for transfusion medicine on-call (TMO) . Given the conflicting claims of educational benefit and detriment of moonlighting, we sought to evaluate impact of TMO moonlighting activities on residents' educational experience and preparedness for clinical practice. Methods/Case Report Residents selected for moonlighting TMO were PGY-3/4, “exceeded expectations” on their transfusion medicine rotation and had taken resident TMO. After initial “boot camp” exercises involving common TMO scenarios, they were added to the attending on-call schedule with a fair financial incentive. They were the first resource and provided in-person support for the TMO resident. Attendings were available for consultation via a secure communications platform (phone and text). After one year, residents were surveyed regarding overall experience, sense of graduated responsibility, impact of financial incentive, preparedness for future practice, and impact on day- time service commitment. "Resident In-Service Examination" (RISE) score was used to estimate impact of TMO on medical knowledge. Results (if a Case Study enter NA) NA. Conclusion Four out of the five residents took the survey. All 4 responded “definite” for positive experience, feeling of graduated responsibility and preparedness for practice. Financial incentive was assessed favorably. No resident reported an adverse impact on their daytime service responsibility. The group median score on RISE transfusion medicine 2022 was 75th percentile with median increase in 10 percentile points. As pathology moves towards sub- specialty practice, working fluency in some areas such as TMO remains a necessity for smaller group practices. Our experience (albeit with a small group of residents) suggests that TMO moonlighting activities may better prepare the residents for this unmet need.

The Blepharoplasty Teacher: Kirigami Model for Learning Eyelid Anatomy.

Blepharoplasty is highly desired for both functional and aesthetic improvements, and it is a common operation performed for patients over the age of 65.1,2 The ability to master the complex periorbital anatomy foundational to blepharoplasty remains challenging for both students and trainees. The three-dimensional understanding of layered structures is hard to obtain through two-dimensional textbooks,3,4 and although the use of technology, including three-dimensional applications, has improved the breadth of anatomy resources, it is hindered by accessibility and cost.5 Therefore, we developed a low-tech, low-cost teaching model, the Blepharoplasty Teacher, that creatively uses the principles of kirigami, the art of paper folding and cutting, to improve education and knowledge retention of the eyelid anatomy critical to the operative approach in blepharoplasty. The Blepharoplasty Teacher device is printed double-sided on 8.5 × 11-inch paper. Figure 1 depicts a representative image. The entire Blepharoplasty Teacher is included as supplemental digital content with instructions for assembly. (See Document, Supplemental Digital Content 1, which demonstrates the Blepharoplasty Teacher, designed by Amy Zhong, M.A., and printed with permission from Mount Sinai Health System. Print on a double-sided sheet of paper and follow the instructions to create a booklet with seven picture panels by cutting and folding along the dotted lines. Cut along the dotted lines on the eyelid to expose each layer, https://links.lww.com/PRS/F444.) A randomized controlled trial was conducted to analyze the efficacy of the Blepharoplasty Teacher in improving education and knowledge retention.Fig. 1.: The Blepharoplasty Teacher, panel 1, printed with permission from Mount Sinai Health System.Study participants comprised 21 first-year and second-year medical students who had previously completed their medical school anatomy course. They completed a 10-question pretest about their knowledge and comfort with blepharoplasty anatomy, and then had 30 minutes to study from either the Blepharoplasty Teacher (n = 11) or a traditional textbook (n = 10). They subsequently completed a 10-question posttest about eyelid anatomy and their comfort level with this anatomy. The surveys contained anatomically based questions adapted from the American Council of Academic Plastic Surgeons in-service examination review material. (See Document, Supplemental Digital Content 2, which shows pretest and posttest questions adapted from the American Council of Academic Plastic Surgeons in-service examination review material, https://links.lww.com/PRS/F445.) Students who used the Blepharoplasty Teacher significantly improved their test scores by an average of 1.4 points out of 10 (p = 0.017), whereas students who used the textbook did not achieve significant improvement (p > 0.05) (Fig. 2). Eight students (73 percent) using the Blepharoplasty Teacher improved their score by at least 1 point as compared to three (30 percent) in the textbook group (p = 0.05). Six students (55 percent) who studied from the Blepharoplasty Teacher felt more comfortable with eyelid anatomy after the study as compared to one student (10 percent) using the textbook (p = 0.031). Nine students (82 percent) who used the Blepharoplasty Teacher would recommend it to a colleague.Fig. 2.: Test score improvement of students who used the Blepharoplasty Teacher compared to students who used textbook pages. *p < 0.05.These findings demonstrate that students who used the Blepharoplasty Teacher achieved significant improvement in their test scores, developed an increased comfort level with eyelid anatomy, and would recommend use of this device to a colleague. Thus, not only was the Blepharoplasty Teacher helpful in knowledge retention, but it was also an enjoyable resource that students believed would benefit the education of their peers. Further research should focus on widespread testing of the Blepharoplasty Teacher among medical students and plastic surgery trainees, as well as on creating similar models for teaching the anatomy of other complex structures.