National Board Of Medical Examiners Research Articles

Introducing AnatomyGPT: A customized artificial intelligence application for anatomical sciences education.

Artificial intelligence (AI) technologies are poised to become an increasingly important part of education in the anatomical sciences. OpenAI has also introduced generative pretrained transformers (GPTs), which are customizable versions of the standard ChatGPT application. There is little research that has explored the potential of GPTs to serve as intelligent tutoring systems for learning the anatomical sciences. The objective of this study was to describe the design and explore the performance of AnatomyGPT, a customized artificial intelligence application intended for anatomical sciences education. The AnatomyGPT application was configured with GPT Builder by uploading open-source textbooks as knowledge sources and by providing pedagogical instructions for how to interact with users. The performance of AnatomyGPT was compared with ChatGPT by evaluating the responses of both applications to prompts of the National Board of Medical Examiners (NBME) sample items with respect to accuracy, rationales, and citations. AnatomyGPT achieved high scores on the NBME sample items for Gross Anatomy, Embryology, Histology, and Neuroscience and scored comparably to ChatGPT. In addition, AnatomyGPT provided several citations in the responses that it generated, while ChatGPT provided none. Both GPTs provided rationales for all sample items. The customized AnatomyGPT application demonstrated preliminary potential as an intelligent tutoring system by generating responses with increased citations as compared with the standard ChatGPT application. The findings of this study suggest that instructors and students may wish to create their own custom GPTs for teaching and learning anatomy. Future research is needed to further develop and characterize the potential of GPTs for anatomy education.

Performance of ChatGPT vs. HuggingChat on OB-GYN Topics.

Background While large language models show potential as beneficial tools in medicine, their reliability, especially in the realm of obstetrics and gynecology (OB-GYN), is not fully comprehended. This study seeks to measure and contrast the performance of ChatGPT and HuggingChat in addressing OB-GYN-related medical examination questions, offering insights into their effectiveness in this specialized field. Methods ChatGPT and HuggingChat were subjected to two standardized multiple-choice question banks: Test 1, developed by the National Board of Medical Examiners (NBME), and Test 2, gathered from the Association of Professors of Gynecology & Obstetrics (APGO) Web-Based Interactive Self-Evaluation (uWISE). Responses were analyzed and compared for correctness. Results The two-proportion z-test revealed no statistically significant difference in performance between ChatGPT and HuggingChat on both medical examinations. For Test 1, ChatGPT scored 90%, while HuggingChat scored 85% (p = 0.6). For Test 2, ChatGPT correctly answered 70% of questions, while HuggingChat correctly answered 62% of questions (p = 0.4). Conclusion Awareness of the strengths and weaknesses of artificial intelligence allows for the proper and effective use of its knowledge. Our findings indicate that there is no statistically significant difference in performance between ChatGPT and HuggingChat in addressing medical inquiries. Nonetheless, both platforms demonstrate considerable promise for applications within the medical domain.

Comparing the Performance of Popular Large Language Models on the National Board of Medical Examiners Sample Questions.

Large language models (LLMs) have transformed various domains in medicine, aiding in complex tasks and clinical decision-making, with OpenAI's GPT-4, GPT-3.5, Google's Bard, and Anthropic's Claude among the most widely used. While GPT-4 has demonstrated superior performance in some studies, comprehensive comparisons among these models remain limited. Recognizing the significance of the National Board of Medical Examiners (NBME) exams in assessing the clinical knowledge of medical students, this study aims to compare the accuracy of popular LLMs on NBME clinical subject exam sample questions. The questions used in this study were multiple-choice questions obtained from the official NBME website and are publicly available. Questions from the NBME subject exams in medicine, pediatrics, obstetrics and gynecology, clinical neurology, ambulatory care, family medicine, psychiatry, and surgery were used to query each LLM. The responses from GPT-4, GPT-3.5, Claude, and Bard were collected in October 2023. The response by each LLM was compared to the answer provided by the NBME and checked for accuracy. Statistical analysis was performed using one-way analysis of variance (ANOVA). A total of 163 questions were queried by each LLM. GPT-4 scored 163/163(100%), GPT-3.5 scored 134/163 (82.2%), Bard scored 123/163 (75.5%), and Claude scored 138/163 (84.7%). The total performance of GPT-4 was statistically superior to that of GPT-3.5, Claude, and Bard by 17.8%, 15.3%, and 24.5%, respectively. The total performance of GPT-3.5, Claude, and Bard was not significantly different. GPT-4 significantly outperformed Bard in specific subjects, including medicine, pediatrics, family medicine, and ambulatory care, and GPT-3.5 in ambulatory care and family medicine. Across all LLMs, the surgery exam had the highest average score (18.25/20), while the family medicine exam had the lowest average score (3.75/5). Conclusion: GPT-4's superior performance on NBME clinical subject exam sample questions underscores its potential in medical education and practice. While LLMs exhibit promise, discernment in their application is crucial, considering occasional inaccuracies. As technological advancements continue, regular reassessments and refinements are imperative to maintain their reliability and relevance in medicine.

Transition From Normative to Criterion-Based Grading in the Obstetrics and Gynecology Clerkship.

To compare grades, National Board of Medical Examiners (NBME) Shelf Exam scores, and student satisfaction with the Obstetrics and Gynecology (OB/GYN) clerkship after transitioning from normative to criterion-based grading. Between July 2021 and July 2022, the Icahn School of Medicine at Mount Sinai (ISMMS) adhered to a normative grading scheme in which ∼60% of students achieved a grade of Honors, 30% achieved a grade of High Pass and 10% achieved a grade of Pass for the OB/GYN clerkship. In July 2022, ISMMS transitioned to a criterion-based scheme. In this scheme, 6 competencies were created. Criteria were determined for each competency, delineating achieving a score of "Pass" versus "Honors" for the specific objective. Students needed to meet the criteria for Honors for 4 out of 6 of the competencies in order to ultimately receive a grade of Honors for the clerkship. The number of students achieving Honors, NBME shelf exam scores, and student clerkship satisfaction ratings between the normative and criterion-based schemes were compared. The number of students studying in academic year (AY) 2021-2022 and AY 2022-2023 were 134 and 137, respectively. A significantly lower percentage of students received Honors in AY 2021-2022 than in AY 2022-2023 (66% vs. 96%, P < .01). Mean exam scores were significantly higher for those receiving Honors in AY 2021-2022 than in AY 2022-2023 (P < .05); scores for AY 2021-2022 and AY 2022-2023 were 78.9, 95% CI [77.6, 80.1] and 76.7, 95% CI [75.6, 77.8], respectively. Mean exam scores for all students were not significantly different between the 2 academic cohorts (77.8 vs. 76.2, P = .06). Clerkship satisfaction rating was significantly higher in AY 2022-2023 than in AY 2021-2022 (4.1 vs. 3.7, P < .05). These findings support a paradigm that compares learner performance to predefined measures as opposed to peer performance.

Pediatric Intersession: An Upfront Flipped-Classroom Curriculum to Promote Pediatric Clerkship Readiness.

Best practices for content selection, mode of delivery, and timing of pediatric clerkship readiness curricula for medical students have, by and large, not been established. Capitalizing on changes in structure of the clinical clerkships during the COVID-19 pandemic, we created an upfront clerkship readiness curriculum, termed Pediatric Intersession (PI), to replace the existing weekly lecture-based clerkship didactics. Our goal was to develop an interactive curriculum with innovative instructional design methodology intended to promote broad foundational pediatric knowledge and clerkship preparedness using case-based learning. We first conducted a needs assessment and crafted curriculum content using guiding principles from the 2019 Council on Medical Student Education in Pediatrics (COMSEP) curriculum. We then organized material into four daily modules prior to the start of the clerkship and employed flipped classroom (FC) methodology. Sixty-six percent of students completed course evaluations, and >90% of the 100 respondents reported that the PI enhanced their clinical learning. Pre-/post-testing demonstrated some knowledge gain following the small-group sessions and there was no change on the National Board of Medical Examiners (NBME) Subject Exam mean scores compared to prior cohorts of students. The global pandemic provided an opportunity to re-envision our pediatric clerkship didactics content, while also incorporating instructional design methodology preferred by students. Our curriculum promotes a small group-based, interactive approach to clerkship readiness that fosters learning in the clinical environment that can be adapted for various settings. Our evaluation suggests that the transition to a FC readiness curriculum can be done successfully while effectively preparing students for their pediatric clerkship.

Certified Nurse-Midwives as Teachers: Expanding Interprofessional Collaboration Learning Opportunities for Medical Students on the Obstetrics and Gynecology Clerkship.

Interprofessional experiences provide critical exposure to collaborative health care teams, yet medical students often lack this experience during clinical clerkships. We created a labor and delivery triage rotation exclusively staffed by certified nurse-midwives in the obstetrics and gynecology clerkship to address this gap. We sought to evaluate the clinical learning experiences of medical students rotating on this midwife-supervised collaborative team. Between 2018 and 2020, we electronically sent all medical students an evaluation after each required clerkship. Our primary outcome was the quality of teaching score, as defined by level of agreement on a 5-point Likert scale with a statement regarding teaching quality. A secondary outcome evaluated scores from the National Board of Medical Examiners (NBME) Obstetrics and Gynecology subject examination taken by all students at the end of the rotation. Evaluations and assessments were compared between students rotating on labor and delivery triage and those who did not. Evaluation comments from students rotating on labor and delivery triage were collated to identify common themes. Of 374 students, 370 (98.9%) completed the end-of-clerkship course evaluation and 312 (83.4%) completed teaching evaluations. Sixty-seven students (17.9%) rotated in labor and delivery triage; of these, 52 (77.6%) completed both the course and triage rotation evaluations. There were no differences in the mean quality of teaching scores (3.9 ± 1.2 versus 3.8 ± 1.2, p = 0.54) or in NBME Obstetrics and Gynecology subject examination scores between students rotating in labor and delivery triage compared to all other rotations (79.9% ± 7.2% vs 80.2% ± 7.8%, P = 0.436). Comments from teaching evaluations highlighted student exposure to the midwifery philosophy of care model. This work demonstrates the feasibility and benefits of this midwife-led authentic interprofessional collaborative experience for medical students. This model can serve as an example of how to implement and evaluate interprofessional collaboration experiences in the clinical setting.

Impact of USMLE Step 1 Transition to Pass/Fail Scoring System on Medical Students' Anxiety, Sleep Quality, and Burnout

BACKGROUND: Originally designed to evaluate the application of foundational scientific knowledge in clinical contexts, the United States Medical License Exam Step 1 evolved into a comparative tool for assessing candidates' educational foundations. This transition heightened the pressure on medical students to excel in the exam. In response, collaborative efforts involving the National Board of Medical Examiners prompted a significant change from reporting scores to a pass/fail system. However, the true impact of this shift remains insufficiently explored. This study aims to assess the emotional toll - encompassing burnout, anxiety, depression, and sleep quality - experienced prior to taking the Step 1 exam. Additionally, it aims to uncover potential gender-based disparities in perceived stress, anxiety, and depression. METHODS: The study encompasses the entirety of third-year medical students at Drexel University College of Medicine, who were invited to participate in a comprehensive survey. Drawing from retrospection, the survey relies on self-reported data regarding anxiety, depression, sleep quality index, and burnout levels. Data compilation was anonymized and executed through the Qualtrics platform. RESULTS: A total of 102 medical students completed the survey, with a 97% pass rate for the USMLE Step 1. Despite their excellent performance, 75% of students reported inadequate sleep quality, and 68% exhibited mild to moderate anxiety levels. Among them, a higher percentage of females (83%) experienced mild to moderate anxiety compared to their male counterparts (50%). Furthermore, 66% of students felt that their commitment to education exceeded what was reasonable for their well-being. CONCLUSION: The transition from traditional scoring to a pass/fail system was ostensibly intended to enhance the mental well-being of medical students. Nevertheless, our findings underscore that students continue to grapple with heightened levels of stress, anxiety, and burnout during the pivotal month leading up to the Step 1 examination.

Abstract 16722: Performance Evaluation of ChatGPT 4.0 on Cardiovascular Clinical Cases From the USMLE Step 2CK and Step 3 of the National Board of Medical Examiners

Introduction: As the prominence of online chat-based artificial intelligence (AI) platforms increases, medical students and residents in training may be inclined to utilize these tools as supplementary resources while preparing for their United States Medical Licensing Board (USMLE) examinations. Hypothesis: The aim of this study was to assess the performance of ChatGPT 4.0 in responding to cardiovascular (CV) clinical vignettes from USMLE sample questions provided by the National Board of Medical Examiners (NBME). Methods: We extracted the 31 CV clinical vignettes from the official NBME USMLE Step 2 Clinical Knowledge and Step 3 self-assessments, and submitted them to ChatGPT 4.0. The clinical cases encompassed a range of topics such as preventive medicine, congenital heart disease, valvular heart disease, electrophysiology, heart failure, and pericardial disease. Each of the 31 cases had a MCQ: 29/31 had at least 5 answer choices, while the remaining 2 had 4 answer choices each. The performance of the AI model was assessed using the NBME-provided answers. Subsequently, two experienced cardiologists independently reviewed the explanations provided by the model for each case and classified them as “accurate” - if they aligned with appropriate guidelines, and as “inaccurate” if they conveyed incorrect or inappropriate information. Results: The AI model attained a grade of 90.3% (28/31). The initial agreement between the 2 reviewers was near perfect with a Cohen’s Kappa of 0.85. Any identified discrepancies were subsequently addressed and resolved. Of the correctly answered questions, accurate explanations were provided by the AI model in 89.2% of cases (25/28). Conclusions: The high performance of ChatGPT 4.0, suggests it could be an effective additional CV study resource for residents and medical students preparing for the USMLE Step 2 CK and Step 3 exams.

Disparities in Gender and Diversity Representation Among Surgical Subspecialties: Are we Losing Momentum?

IntroductionRecruitment efforts have demonstrated small increases in female and under-represented applicants in recent years; however, the majority of surgical programs remain predominantly Caucasian and male. With increased national emphasis on Diversity, Equity, and Inclusion initiatives and mentoring programs, applicants to surgical specialties have continued to increase. While strategies to improve gender and racial diversity are now openly discussed, it is unclear if we have seen significant improvement. We sought to analyze the gender and diversity trends between surgical specialties. MethodsPublicly available data from the Association of American Medical Colleges, National Board of Medical Examiners, and Accreditation Council for Graduate Medical Education were extracted to determine total number, gender, and diversity of surgery applicants and active residents from the years 2018 to 2021. Surgical specialties within the main match were compared through an analysis completed through Microsoft Excel. ResultsBetween the years from 2018 to 2021, there was a rise in diversity representation among all surgical residents except for orthopedics which remained less than 30%. Orthopedics, Neurosurgery, and Thoracic Integrated training programs have the lowest rates of female representation among current residents at 16.72%, 20.37%, and 30.05%, respectively. General surgery demonstrates the greatest increase of female residents with a positive 6% change over this recent four-year time period. ConclusionsThere has been minimal positive progress in gender and diversity representation in surgical subspecialties overall in recent years. Continued advocacy through mentorship and scholarship programs is recommended to achieve greater gender and diversity representation in general surgery and surgical subspecialties.

The Impact of Tele-Education on Family Medicine Clerkship Students' Learning Outcomes.

The COVID-19 pandemicnecessitated rapidchanges to medical educationfor student and patient protection. A dearth of published US studies examineresultingclinical education outcomesdue topandemic-induced curricula changes.We describe adaptations made toafamily medicine clerkship to move itfrom traditional in-person delivery to virtualonly,and then from virtualto hybrid;and compareeducationaloutcomes of students across delivery types. We stratified 386 medical students in their third year completing their 8-week family medicine clerkship bytype of content delivery,including in person, virtual only,and hybridinstruction.We examined the impact of these changes on three clerkship learning outcomes: the midblock assessment score,the National Board of Medical Examiners (NBME)exam score,and the final numeric score(FNS). In our sample, 164 (42.5%) received content in person, 36 (9.3%) received virtual only, and 186 (48.2%) received hybridcontent.Studentsreceivingvirtual only (M=76.4, SD=9.1) had significantly higher midblock assessment scores (F=8.06,df=2,P=.0004) than studentsreceivinghybrid (M=71.7, SD=8.8) and in-person training (M=74.5, SD=7.2).Nosignificant differences existed in students' NBME exam scores or FNSs across deliverytypes. Students receiving virtual-onlyor hybrid contentperformed at least as well on threeclerkship-relatededucationaloutcomesas their pre-COVID peers participating in person.Further research is needed to understand how changes to medical education affected student learning and skill development.

Disadvantaged Students Utilize School Campus and Its Resources More Than Non-disadvantaged Students.

Introduction Disadvantaged populations were disproportionately affected by the COVID-19 pandemic, both in the medical and educational settings. Lower-income families often do not have a laptop/desktop computer, adequate internet connection, or a dedicated study space. This unfortunately contributed to poorer academic performance during distance learning. To combat this, the Kirk Kerkorian School of Medicine (KKSOM) did not close down campus during the pandemic. This study analyzes the utilization of campus and live Zoom lectures by KKSOM students and its impact on educational outcomes. Methods We sent an Institutional Review Board (IRB)-approved survey to KKSOM students and asked about study locations, Zoom lecture attendance, and relationship quality during the pandemic. The class of 2024 had a unique experience as they were first-year students during the start of the COVID-19 pandemic and the transition to online learning. However, they always had access to campus and technological resources. We compared the survey scores from a Qualtrics electronic survey and the National Board of Medical Examiners (NBME) scores of students who self-indicated disadvantaged status, first-generation, underrepresented minority, and lower socioeconomic status to those who did not meet these criteria.Data analysis was done using SPSS software version 28.0.1.1 (IBM Corp., Armonk, NY). Results First-generation students studied on campus more frequently than their counterparts (31% versus 20%, p < 0.05) and less at home in general (55.4% versus 67.5%, p < 0.05). Lower socioeconomic status (SES) students attended live Zoom lectures more often as well (56.6% versus 43.1%, p < 0.05). Lastly, no significant differences were found between disadvantaged and non-disadvantaged groups for the class of 2024 in the NBME exam scores or relationship quality scores. Conclusion Our results suggest that students from disadvantaged backgrounds spend more time studying on campus than at home. Additionally, during the COVID-19 pandemic, they attended live Zoom lectures more often than their non-disadvantaged counterparts. Access to campus was not restricted for KKSOM studentsduring the pandemic. This may be one explanation for the lack of disparity between disadvantaged and non-disadvantaged students with regard to academic performance and relationship quality. This makes a strong argument for the importance of campus accessibility for the success of students, especially those from disadvantaged backgrounds.

A Survey of Otolaryngology Residency Programs on Adapting to the United States Medical Licensing Examination (USMLE) Step 1 Transitioning to Pass/Fail.

Objectives In February 2020, the National Board of Medical Examiners (NBME) announced that the United States Medical Licensing Examination (USMLE) Step 1 licensing examination would changefrom a numerical score to Pass/Fail (P/F). After implementation, many believe that USMLE-Step 2-Clinical Knowledge (CK) will become an important metric for students applying to otolaryngology (ENT). The purpose of this study is to determine factors important to resident selection after these changes. Methods A survey containing 15 questions related to resident selection practices and how changing USMLE Step 1 to P/F would impact future resident selection was designed. It was distributed to all ENT residency programs accredited by the Accreditation Council for Graduate Medical Education (ACGME). Results Forty percent of programs responded; 66% (95% confidence interval (CI): 51.1%-78.4%) felt that changing Step 1 scoring would not lead to students being more prepared for clinical rotations; 55% believe class rank will increase in significance (95% CI: 35.7%-64.3%). There was also an increase in the importance of Step 2 CK, which had a mean ranking of 10.67 prior to changes in Step 1 scoring and increased to 7.80 after P/F. Conclusions The changes in Step 1 scoring will likely lead to increasing importance of other objective measures like class rank or Step 2 CK. This may defeat the intended purpose put forth by the NBME. Therefore, further guidance on measures correlated with student performance as a resident will be integral to the selection process.

Impact of Obstetrics and Gynecology Clerkship Order on NBME Exam Scores After a Curriculum Change

Our Institution instituted curriculum reform in 2013. We sought to examine the impact of rotation order on the end of clerkship National Board of Medical Examiners (NBME) Obstetrics and Gynecology (Ob/Gyn) Subject Exam scores after curriculum restructuring. This is a retrospective analysis of Ob/Gyn NBME scores over 2 years after curriculum reform. At our university, a 15-week block is divided into 5-week intervals of General Surgery, Ob/Gyn, and surgical subspecialties, in any order. During the 16 weeks, students take the NBME Subject Examinations for Ob/Gyn and Surgery. We defined rotation groups by proximity to the shelf exam. Group 1 completed Ob/Gyn first, furthest away from the exam, Group 2 completed Ob/Gyn second, and Group 3 completed Ob/Gyn last, closest to the test. We compared average shelf exam scores between Groups. Uniformed Services University of Health Sciences during the clerkship year. Medical students at the Uniformed Services University. We obtained data from 331 students (118 students in Group 1, 106 in Group 2, and 107 in Group 3). Scores ranged from 55 to 99 (mean 72.5, SD 7.3). Mean (SD) NBME score was 71.9 (6.9) in Group 1, 73.2 (7.2) in Group 2, and 72.6 (7.7) in Group 3 (p = 0.415). Rotation order does not affect NBME Ob/Gyn Subject exam scores in a fifteen-week integrated clerkship block.

Effectiveness of Near-Peer-Taught Case Reviews on Students' Confidence in National Board of Medical Examiners (NBME) Exams.

Introduction A key element to a first-year medical student's (MS1) education is guidance and practice in applying anatomy, pathophysiology, diagnosis, and treatment concepts to clinical vignettes. One potential solution to providing effective clinical reasoning training is the involvement of second-year medical students (MS2s) in small group sessions as teachers to provide more personalized instruction via case-based learning sessions. Near-peer teaching has been shown to benefit students' confidence in learning and improve test scores. Similarly, case-based learning is heavily associated with an improved understanding of complex topics. As such, this study assessed the efficacy of near-peer teaching with concomitant case-based style presentations on improving the comfort of MS1s with their understanding of the curriculum content and their comfort with applying their knowledge to clinical scenarios. Methods This randomized controlled crossover trial included several small-group study sessions, each consisting of five MS1s and led by an MS2 who reviewed selected clinical cases in a standardized slide decks. The control arm was provided the same slide decks but did not participate in the MS2-led sessions. During the first course, students were assigned to either the control or intervention group and then crossed over to the opposite group (control to intervention and vice versa) in every subsequent course. Comfort with the curriculum material was then assessed through pre and post surveys, with the post surveys administered after the MS1s took their final NBME examination for that course. Results The study was cut short due to COVID-19 precautions limiting in-person sessions. Nevertheless, the post survey demonstrated an increased understanding of pathological concepts for the intervention group compared to the control group. Conclusions Future work on near-peer group study sessions should enroll a larger sample size with measures to improve the response rate to better test whether near-peer-led case reviews had a significant effect on students' understanding of anatomical concepts and confidence during NBME examinations.

Consider implications of ruling that deemed gifted student eligible for testing modifications

A  New York federal court granted a preliminary injunction   to a gifted student with learning disabilities and attention    deficit hyperactivity disorder, holding that the National Board of Medical Examiners had to provide the student with double time for the Step 1 U.S. Medical Licensing Examination. The court rejected the NBME's argument that the student wasn’t an individual with a disability due to his academic success. (Sampson v. Nat’l Bd. of Med. Exam’rs, 22‐CV‐05120 (E.D.N.Y. 2022).

Medical Student Duty Hours and Shelf Performance - Is there a Correlation?

To date, there is limited information about medical student duty hours, shelf scores, and overall clerkship performance in Obstetrics and Gynecology. As a result, we were curious to know whether spending more time in the clinical environment translated to an improved learning experience or, in contrast, translated to decreased study time and worse overall clerkship performance. A retrospective cohort analysis was performed at a single academic medical center of all medical students on the Obstetrics and Gynecology clerkship from August 2018 to June 2019. Recorded student duty hours were tabulated per day and per week, by student. National Board of Medical Examiners (NBME) Subject Exam (Shelf) equated percentile scores for the quarter of year were used. Our statistical analysis showed that working long hours did not translate to higher or lower shelf score, or higher overall clerkship grade. However, working longer hours in the last two weeks of the clerkship was associated with high shelf score. Longer medical student duty hours did not correlate to higher shelf scores or overall clerkship grades. Future multi-center studies are necessary to evaluate the importance of medical student duty hours and continue optimizing the educational experience of the OB/GYN clerkship.

The importance of USMLE step 2 on the screening and selection of applicants for general surgery residency positions

BackgroundAs announced by the Federation of State Medical Boards (FSMB) and National Board of Medical Examiners (NBME), the United States Medical Licensing Examination (USMLE) Step 1 score reporting has transitioned to pass/fail outcomes instead of the traditional numeric score after January 26, 2022. USMLE Step 1 scores have been used widely as a crucial tool in screening and selecting applicants for residency programs. This study aims to determine the role of USMLE Step 2 in the selection of applicants for general surgery residency. MethodsA retrospective study was conducted over six recruiting cycles from 2016 to 2021. The data from 334 interviewed applicants from one general surgery residency program were assessed. Data analyzed included USMLE Step 1 and Step 2 scores, applicant gender, Alpha Omega Alpha (AOA) status, letters of recommendation (LOR), and research/publications (RS). ResultsOf the 334 interviewed applicants, 209 (62.6%) were male. The mean [SD] USMLE Step 1 and USMLE Step 2 C K (Clinical Knowledge) scores were 239.6 [±10.4] and 249.2 [±11.4], respectively. The mean (SD) LOR and RS scores were 4.24 [±0.4] and 3.9 [±0.7], respectively. A positive correlation was observed between USMLE Step 1 and USMLE Step 2 C K (Clinical Knowledge) scores (r = 0.60, p < .001), LOR scores (r = 0.24, p = .008), and AOA status (r = 0.19, p = .038). There was a negligible correlation between USMLE scores and applicant gender. ConclusionTransitioning USMLE Step 1 to pass/fail will make the initial screening and selection process of applications challenging for residency programs. In the short term, USMLE Step 2 scores, LOR, and AOA status are important as screening assessments. Valid measures to ensure appropriate, equitable, and fair assessments are needed.

Medical Students' Socioeconomic Status and Academic Performance in Medical School.

Background Students from lower socioeconomic groups tend to underestimate their chances of acceptance to medical school and their likelihood of success once admitted. Objective The objective of this study is to determine if socioeconomic disadvantage status is linked to lower medical college admission test (MCAT) scores and academic performance in medical school. Methods Using the Association of American Medical Colleges (AAMC) education/occupation (EO) indicator, we compared economically disadvantaged students to students with no financial disadvantage on the MCAT, Phase 1 National Board of Medical Examiners (NBME), United States Medical Licensing Examination (USMLE) Step 1, Phase 2 NBME, and USMLE Step 2 test scores. Results Medical students in the disadvantaged group scored significantly lower on the MCAT than students with no financial disadvantage. The disadvantaged group showed a non-significant lower trend in performance until USMLE Step 2. Conclusions Applicants from lower socioeconomic backgrounds may perform lower on their MCAT and early medical school benchmark exams, but they appear to catch up with and even surpass their peers on their USMLE Step 2 examination.

Exploring Radiation Oncology Representation on the National Board of Medical Examiners (NBME) Official Practice Material for the Undergraduate United States National Standardized Medical Board Examinations

Background The portrayal of oncology-related topics on the National Board of Medical Examiners (NBME) published practice material for undergraduate medical education (UGME) standardized examinations including the United States Medical Licensing Exams (USMLE) Step 1, Step 2 Clinical Knowledge (CK), and NBME Clinical Science Subject Examinations ('Shelf') is largely unknown. The goal of this study is to explore the presence and characterization of radiation oncology topics, including radiation therapy (RT), on these practice examinations. Methods All current and commercially accessible NBME published self-assessments and sample questions for the aforementioned examinations will be queried for content pertaining to oncology and associated treatment modalities of RT, surgery, and systemic therapy. The total volume of oncology-related questions and its distribution across all 3 major practice examination types will be recorded. The questions will be cataloged by selected attributes including context, treatment modality appearing as the correct answer for "next best step" (NBS) prompt, and answers appearing as the incorrect answer for NBS prompt. Results 1320 USMLE Step 1 practice questions are available for analysis. In preliminary analysis, minimal questions about RT were noted; all appearing to involve concepts of associated toxicity. 920 USMLE Step 2CK practice questions are available for analysis. RT-related questions appeared with higher frequency on the clinical science preparation material (n=10) compared to basic science (n=4), while also notably appearing as a listed answer choice for NBS prompt against other treatment modalities. 1325 practice questions from relevant NBME shelf examinations were available. RT appeared sparingly on the practice NBME shelf examinations of Surgery (n=5), Medicine (n=4), Ob/Gyn (n=2), and Clinical Neurology (n=2). RT questions were absent from Pediatrics, Emergency Medicine, and Family Medicine Shelf practice examination material. In the final analysis of these practice questions from all 3 major examination types, we seek to conduct a comparative analysis to other treatment modalities. Discussion This work represents the first attempt, to our knowledge, to characterize RT topic appearance within the NBME published practice questions and their accompanying explanations for the USMLE Step 1, Step 2CK, and select NBME shelf examinations. Our analysis is important in assessing the presence of RT topics on medical student board preparation material, a potential avenue of RT content exposure in UGME. Careful review and comparative analyses of all oncology-related content can provide important insight for UGME-related RT education. Our anticipated analyses and future discussion may identify areas for improvements in the important education of oncology-related topics in UGME. The portrayal of oncology-related topics on the National Board of Medical Examiners (NBME) published practice material for undergraduate medical education (UGME) standardized examinations including the United States Medical Licensing Exams (USMLE) Step 1, Step 2 Clinical Knowledge (CK), and NBME Clinical Science Subject Examinations ('Shelf') is largely unknown. The goal of this study is to explore the presence and characterization of radiation oncology topics, including radiation therapy (RT), on these practice examinations. All current and commercially accessible NBME published self-assessments and sample questions for the aforementioned examinations will be queried for content pertaining to oncology and associated treatment modalities of RT, surgery, and systemic therapy. The total volume of oncology-related questions and its distribution across all 3 major practice examination types will be recorded. The questions will be cataloged by selected attributes including context, treatment modality appearing as the correct answer for "next best step" (NBS) prompt, and answers appearing as the incorrect answer for NBS prompt. 1320 USMLE Step 1 practice questions are available for analysis. In preliminary analysis, minimal questions about RT were noted; all appearing to involve concepts of associated toxicity. 920 USMLE Step 2CK practice questions are available for analysis. RT-related questions appeared with higher frequency on the clinical science preparation material (n=10) compared to basic science (n=4), while also notably appearing as a listed answer choice for NBS prompt against other treatment modalities. 1325 practice questions from relevant NBME shelf examinations were available. RT appeared sparingly on the practice NBME shelf examinations of Surgery (n=5), Medicine (n=4), Ob/Gyn (n=2), and Clinical Neurology (n=2). RT questions were absent from Pediatrics, Emergency Medicine, and Family Medicine Shelf practice examination material. In the final analysis of these practice questions from all 3 major examination types, we seek to conduct a comparative analysis to other treatment modalities. This work represents the first attempt, to our knowledge, to characterize RT topic appearance within the NBME published practice questions and their accompanying explanations for the USMLE Step 1, Step 2CK, and select NBME shelf examinations. Our analysis is important in assessing the presence of RT topics on medical student board preparation material, a potential avenue of RT content exposure in UGME. Careful review and comparative analyses of all oncology-related content can provide important insight for UGME-related RT education. Our anticipated analyses and future discussion may identify areas for improvements in the important education of oncology-related topics in UGME.

Prior Enlisted Medical Students at the Uniformed Services University: Outcomes During Four Years of Medical School

The Uniformed Services University (USU) implemented the Enlisted to Medical Degree Preparatory Program (EMDP2) with the goal of enhancing the diversity of the military physician corps. Programs like EMDP2 can assist students in making the social and intellectual transition from undergraduate studies to medical school and beyond. These types of programs are also opportunities to reduce health disparities and prepare students to work in multicultural settings. The purpose of this study was to evaluate whether there was any significant difference in performance between USU medical students who had attended the EMDP2 and those who had not.We compared the results of National Board of Medical Examiners (NBME) Clinical Science Subjects, United States Medical Licensing Examination (USMLE) Step 1, and USMLE Step 2 Clinical Knowledge exams of EMDP2 learners from the School of Medicine classes of 2020 to 2023 to those of four similarly sized cohorts of their peers who varied by age and prior military service.We found that the performance of EMDP2 graduates was comparable to their peers who followed more traditional and other alternative paths to medical school. For example, regression models showed that EMDP2 status was not a statistically significant predictor of average clerkship NBME exam score, nor of USMLE Step 1 failure.EMDP2 graduates performed on a par with their medical school peers, and EMDP2 status does not appear to influence NBME or USMLE performance. EMDP2 provides a focused curriculum and addresses the mandate to make medical education opportunities available to a more diverse population.