United States Medical Licensing Exam Research Articles

Predictors of Plastic Surgery Applicant Success: An Analysis of the Texas STAR Database.

Integrated plastic surgery residency is one of the most competitive medical specialties. Although previous studies have surveyed integrated plastic surgery residency program directors regarding desired applicant characteristics, there is a paucity of literature assessing detailed application characteristics and reported match outcomes from applicants in recent application cycles. This study examines application characteristics associated with matching into integrated plastic surgery residency from 2017 to 2023. The authors accessed the Texas Seeking Transparency in Application to Residency database, which contains survey information from graduating medical students nationwide regarding residency application characteristics and specialties/programs to which applications were submitted. Characteristics of matched versus unmatched applicants between 2017 and 2023 were compared using χ2 and 2-sided, independent t tests. Univariate logistic regression models were used to assess predictors of a successful match. A total of 381 integrated plastic surgery residency applicants responded to the Texas Seeking Transparency in Application to Residency survey from 2017 to 2023. Mean United States Medical Licensing Exam Step 2 CK scores; the number of away rotations, interview offers, and honored clerkships; and Alpha Omega Alpha membership rate were significantly associated with and predictive of matching. Preference signaling of programs was associated with an increased interview offer rate. Higher board examination scores, increasing numbers of honored clerkships, away rotations, and Alpha Omega Alpha membership were identified as statistically significant predictors of matching into integrated plastic surgery residency. Prospective applicants should use this information to help guide their efforts in these areas that appear to be associated with a successful transition to residency.

Assessing ChatGPT 4.0’s test performance and clinical diagnostic accuracy on USMLE STEP 2 CK and clinical case reports

While there is data assessing the test performance of artificial intelligence (AI) chatbots, including the Generative Pre-trained Transformer 4.0 (GPT 4) chatbot (ChatGPT 4.0), there is scarce data on its diagnostic accuracy of clinical cases. We assessed the large language model (LLM), ChatGPT 4.0, on its ability to answer questions from the United States Medical Licensing Exam (USMLE) Step 2, as well as its ability to generate a differential diagnosis based on corresponding clinical vignettes from published case reports. A total of 109 Step 2 Clinical Knowledge (CK) practice questions were inputted into both ChatGPT 3.5 and ChatGPT 4.0, asking ChatGPT to pick the correct answer. Compared to its previous version, ChatGPT 3.5, we found improved accuracy of ChatGPT 4.0 when answering these questions, from 47.7 to 87.2% (p = 0.035) respectively. Utilizing the topics tested on Step 2 CK questions, we additionally found 63 corresponding published case report vignettes and asked ChatGPT 4.0 to come up with its top three differential diagnosis. ChatGPT 4.0 accurately created a shortlist of differential diagnoses in 74.6% of the 63 case reports (74.6%). We analyzed ChatGPT 4.0’s confidence in its diagnosis by asking it to rank its top three differentials from most to least likely. Out of the 47 correct diagnoses, 33 were the first (70.2%) on the differential diagnosis list, 11 were second (23.4%), and three were third (6.4%). Our study shows the continued iterative improvement in ChatGPT’s ability to answer standardized USMLE questions accurately and provides insights into ChatGPT’s clinical diagnostic accuracy.

Medical students' perspective on the United States Medical Licensing Examination (USMLE) Step 1 transition to Pass/Fail

Background The transition of the United States Medical Licensing Exam: Step 1 to Pass/Fail (P/F), from scored, caused uncertainty about students’ preparedness and wellbeing related to the exam. Comparison of study behavior and results, before and after the P/F transition can provide insights for the medical curriculum and student support. Methods Data from four cohorts of second-year medical students (Class of 2022–25, N = 204) were collected from their dedicated Step 1 self-study block. Student study regiments, aggregate practice test results, Step 1 pass rates and post-block self-reported surveys were analyzed. Results Analysis of practice exam averages across the four student cohorts demonstrates a potentially slower and less rigorous start to Step 1 self-studying during the dedicated preparation block for the cohorts that took the P/F Step 1 exam format as compared to the previous cohorts that prepared for a scored Step 1 exam. Similarly, self-reported study regiments decreased in the median number of hours/day and number of weeks of study for the cohorts with P/F Step 1 exam. There was also a slight shift in the type of study resources used, between the two groups, with the scored group using more traditional board preparation resources. Conclusions The P/F transition of the Step 1 exam may lead to reduced student preparedness and may require adjustments in the resources and support provided by institutions.

Disparities in Medical School Clerkship Grades Associated with Sex, Race, and Ethnicity: A Person-Centered Approach.

To determine whether students' self-reported race/ethnicity and sex were associated with grades earned in 7 core clerkships. A person-centered approach was used to group students based on observed clerkship grade patterns. Predictors of group membership and predictive bias by race/ethnicity and sex were investigated. Using data from 6 medical student cohorts at Johns Hopkins University School of Medicine (JHUSOM), latent class analysis was used to classify students based on clerkship grades. Multinomial logistic regression was employed to investigate if preclerkship measures and student demographic characteristics predicted clerkship performance-level groups. Marginal effects for United States Medical Licensing Exam (USMLE) Step 1 scores were obtained to assess the predictive validity of the test on group membership by race/ethnicity and sex. Predictive bias was examined by comparing multinomial logistic regression prediction errors across racial/ethnic groups. Three clerkship performance-level groups emerged from the data: low, middle, and high. Significant predictors of group membership were race/ethnicity, sex, and USMLE Step 1 scores. Black or African American students were more likely (odds ratio [OR] = 4.26) to be low performers than White students. Black or African American (OR = 0.08) and Asian students (OR = 0.41) were less likely to be high performers than White students. Female students (OR = 2.51) were more likely to be high performers than male students. Patterns of prediction errors observed across racial/ethnic groups showed predictive bias when using USMLE Step 1 scores to predict clerkship performance-level groups. Disparities in clerkship grades associated with race/ethnicity were found among JHUSOM students, which persisted after controlling for USMLE Step 1 scores, sex, and other preclerkship performance measures. Differential predictive validity of USMLE Step 1 exam scores and systematic error predictions by race/ethnicity show predictive bias when using USMLE Step 1 scores to predict clerkship performance across racial/ethnic groups.

Trends in the Main Residency Match From 2007 to 2020.

Background The United States Medical Licensing Exam (USMLE)Step 1 was recently changed from a numerically scored grading system to a pass/fail grading system. Until late 2024, there will be no formal studies about the impact that the grading change will have on the match process. To thoroughly assess the impact that this change will have on the overall match process, it is important to look at what the trends in applicants' objective measures have been in the years before the change. We aim to systematically evaluate the rates of change and mean trend of objective metrics found in residency applications in the main residency match. Methods Objective medical student data of matched and unmatched applicants were queried from the National Matching Program's Charting Outcomes in the Match Reports for the 2007 to 2020 application cycles. Data were used to create linear regression analyses and statistical tests were performed to evaluate trends over time. Results For matched applicants, there were statistically significant positive trends for the mean number of contiguous ranks (m=0.33, p<0.01), having another non-doctoral graduate degree (m=0.67, p<0.01), membership to Alpha Omega Alpha(AOA) honor society (m=0.22, p<0.01), mean USMLE Step 1 score (m=1.01, p<0.01), mean USMLE Step 2 score (m=1.68, p<0.01), mean number of research experiences (m=0.12, p<0.01), and mean number of abstracts, presentations, and publications(m=0.34, p<0.01). Additionally, there was a statistically significant negative trend for the percentage who graduated from a top 40 National Institutes of Health-funded medical school (m=-0.41, p<0.01). For unmatched applicants, there were statistically significant positive trends for having another non-doctoral graduate degree (m=0.83, p<0.01), mean USMLE Step 1 score (m=1.26, p<0.01), mean USMLE Step 2 score (m=2.27, p<0.01), mean number of research experiences (m=0.13, p<0.01), and mean number of abstracts, presentations, and publications (m=0.33, p<0.01). Conclusion Our study shows that there have been statistically significant increases in almost all objective measures in the residency application. Recent changes to the abstracts, presentations, and publications on the Step 1 scoring system will force almost all residency programs to overhaul their application process and potentially increase reliance on Step 2, research, and other nonobjective factors. For students early in their medical education, emphasis on Step 2 and research will yield increased chances of matching into residency in the future.

ChatGPT-4 Performance on USMLE Step 1 Style Questions and Its Implications for Medical Education: A Comparative Study Across Systems and Disciplines.

We assessed the performance of OpenAI's ChatGPT-4 on United States Medical Licensing Exam STEP 1 style questions across the systems and disciplines appearing on the examination. ChatGPT-4 answered 86% of the 1300 questions accurately, exceeding the estimated passing score of 60% with no significant differences in performance across clinical domains. Findings demonstrated an improvement over earlier models as well as consistent performance in topics ranging from complex biological processes to ethical considerations in patient care. Its proficiency provides support for the use of artificial intelligence (AI) as an interactive learning tool and furthermore raises questions about how the technology can be used to educate students in the preclinical component of their medical education. The authors provide an example and discuss how students can leverage AI to receive real-time analogies and explanations tailored to their desired level of education. An appropriate application of this technology potentially enables enhancement of learning outcomes for medical students in the preclinical component of their education.

Application of artificial intelligence chatbots, including ChatGPT, in education, scholarly work, programming, and content generation and its prospects: a narrative review

This study aims to explore ChatGPT’s (GPT-3.5 version) functionalities, including reinforcement learning, diverse applications, and limitations. ChatGPT is an artificial intelligence (AI) chatbot powered by OpenAI’s Generative Pre-trained Transformer (GPT) model. The chatbot’s applications span education, programming, content generation, and more, demonstrating its versatility. ChatGPT can improve education by creating assignments and offering personalized feedback, as shown by its notable performance in medical exams and the United States Medical Licensing Exam. However, concerns include plagiarism, reliability, and educational disparities. It aids in various research tasks, from design to writing, and has shown proficiency in summarizing and suggesting titles. Its use in scientific writing and language translation is promising, but professional oversight is needed for accuracy and originality. It assists in programming tasks like writing code, debugging, and guiding installation and updates. It offers diverse applications, from cheering up individuals to generating creative content like essays, news articles, and business plans. Unlike search engines, ChatGPT provides interactive, generative responses and understands context, making it more akin to human conversation, in contrast to conventional search engines’ keyword-based, non-interactive nature. ChatGPT has limitations, such as potential bias, dependence on outdated data, and revenue generation challenges. Nonetheless, ChatGPT is considered to be a transformative AI tool poised to redefine the future of generative technology. In conclusion, advancements in AI, such as ChatGPT, are altering how knowledge is acquired and applied, marking a shift from search engines to creativity engines. This transformation highlights the increasing importance of AI literacy and the ability to effectively utilize AI in various domains of life.

Pediatric residency milestone performance is not predicted by the United States Medical Licensing Examination Step 2 Clinical Knowledge

Objectives: This study aims to show whether correlation exists between pediatric residency applicants’ quantitative scores on the United States Medical Licensing Exam Step 2 Clinical Knowledge examination and their subsequent performance in residency training based on the Accreditation Council for Graduate Medical Education Milestones, which are competency-based assessments that aim to determine residents’ ability to work unsupervised after postgraduate training. No previous literature has correlated Step 2 Clinical Knowledge scores with pediatric residency performance assessed by Milestones. Methods: In this retrospective cohort study, the United States Medical Licensing Exam Step 2 Clinical Knowledge Scores and Milestones data were collected from all 188 residents enrolled in a single categorical pediatric residency program from 2012 - 2017. Pearson correlation coefficients were calculated amongst available test and milestone data points to determine correlation between test scores and clinical performance. Results: Using Pearson correlation coefficients, no significant correlation was found between quantitative scores on the Step 2 Clinical Knowledge exam and average Milestones ratings (r = -0.1 for post-graduate year 1 residents and r = 0.25 for post-graduate year 3 residents). Conclusions: These results demonstrate that Step 2 scores have no correlation to success in residency training as measured by progression along competency-based Milestones. This information should limit the importance residency programs place on quantitative Step 2 scores in their ranking of residency applicants. Future studies should include multiple residency programs across multiple specialties to help make these findings more generalizable.

Child Neurology and Neurodevelopmental Disabilities Program Directors’ Opinions on Preference Signaling in the 2023-2024 NRMP Match: A Survey

Changes in residency recruitment have significantly altered how programs and applicants evaluate each other including virtual interviews, discontinuation of the United States Medical Licensing Exam Step 2 Clinical Skills exam, and transition of United States Medical Licensing Exam Step 1 to pass-fail scoring. To improve program-applicant fit, the Electronic Residency Application Service introduced supplemental application features including geographic preference, program signaling, and the opportunity to highlight impactful and meaningful experiences. We sought to evaluate child neurology (CN) and neurodevelopmental disabilities (NDD) program director's (PD) opinions regarding these changes. A 10-question anonymous survey was sent to CN (n=75) and NDD (n=8) PDs. The questions centered on PDs' opinions regarding components of the supplemental application, having a standard application review period and in-person recruitment activities. Answer choices to the questions were all close-ended. Respondents could select questions to complete. Thirty-eight CN residency PDs (49%) and 4 NDD residency PDs (50%) responded to the survey. Among CN PDs, there was strong support for use of the supplemental application questions and for the use of 3 program signals per applicant. Most PDs supported a standardized application review period prior to programs sending interview offers; however, there was no consensus on the appropriate length of time. Nearly half agreed with virtual-only interviews, and 62% agreed with the option of in-person second-look visits. CN PDs generally support many of the recent or proposed changes to residency recruitment. The impact of these changes on recruitment will be a topic of future investigation.

Sociodemographics Impact Competitiveness in the Neurosurgical Match: Survey Results

BackgroundThe Neurosurgical Match involves selecting future neurosurgeons who will comprise the future national workforce, based on a competitive ranking process against other applicants. We aimed to identify which sociodemographic and academic factors influence competitiveness and rank position in The Match. MethodsA Council of State Neurosurgical Societies survey was distributed to current United States neurosurgical residents. The primary outcome measure was self-reported rank position of matched program. Variables queried included sociodemographic and academic metrics. Fisher’s exact, logistic regression, and t-tests were performed. ResultsFrom the 72 responses, there was a median United States Medical Licensing Exam (USMLE) Step 1 score of 248, 34.7% were Alpha Omega Alpha inductees, 77.8% completed one-to-three sub-internship rotations, median number of publications was 5, and 13.9% of respondents had PhDs. Sociodemographic analysis demonstrated that 69.4% were male and 30.6% were female. Applicants with a home-neurosurgery program or of female gender had statistically significantly higher odds of matching into a top three program on their rank list (OR 9 and OR 6, respectively). Females exhibited similar mean, but less variance, compared to male respondents for USMLE Step 1 scores and number of publications. Of those with a top three program match, respondents were more likely to agree that the home program supported his/her pursuance of neurosurgery. ConclusionsTwo sociodemographic factors were independently associated with high match rank: presence of home-neurosurgery program and female gender (OR 9 and 6, respectively). Female respondents reported consistently strong academic metrics (similar mean to males, but less variance compared to males).

Impact of Applicants' Characteristics and Geographic Connections to Residency Programs on Preference Signaling Outcomes in the Match.

To assess the impact of applicant and residency program characteristics on preference signaling outcomes in the Match during the first 2 years of implementation across 6 specialties. Data were obtained from the Texas Seeking Transparency in Application to Residency survey for applicants applying into otolaryngology during the 2020-2021 and 2021-2022 application cycles and into dermatology, internal medicine (categorical and preliminary year), general surgery, and urology during the 2021-2022 application cycle. The primary outcome was signal yield, defined as the number of interviews at signaled programs divided by the total number of signals sent. Associations with applicant-reported characteristics and geographic connections to residency programs were assessed using Wilcoxon-Rank Sum Testing, Spearman's rank correlation testing, and ordinary least squares regression. A total of 1,749 applicants with preference signaling data were included from internal medicine (n = 884), general surgery (n = 291), otolaryngology (n = 217), dermatology (n = 147), urology (n = 124), and internal medicine preliminary year (n = 86). On average 60.9% (standard deviation 32.3%) of signals resulted in an interview (signal yield). There was a stepwise increase in signal yield with the percentage of signals sent to programs with a geographic connection (57.3% for no signals vs. 68.9% for 5 signals, P < .01). Signal yield was positively associated with applicant characteristics, such as United States Medical Licensing Exam Step 1 and 2 scores, honors society membership, and number of publications (P < .01). Applicants reporting a lower class rank quartile were significantly more likely to have a higher percentage of their interviews come from signaled programs (P < .01). Signal yield is significantly associated with geographic connections to residency programs and applicant competitiveness based on traditional metrics. These findings can inform applicants, programs, and specialties as preference signaling grows.

Benchmarking ChatGPT-4 on a radiation oncology in-training exam and Red Journal Gray Zone cases: potentials and challenges for ai-assisted medical education and decision making in radiation oncology.

The potential of large language models in medicine for education and decision-making purposes has been demonstrated as they have achieved decent scores on medical exams such as the United States Medical Licensing Exam (USMLE) and the MedQA exam. This work aims to evaluate the performance of ChatGPT-4 in the specialized field of radiation oncology. The 38th American College of Radiology (ACR) radiation oncology in-training (TXIT) exam and the 2022 Red Journal Gray Zone cases are used to benchmark the performance of ChatGPT-4. The TXIT exam contains 300 questions covering various topics of radiation oncology. The 2022 Gray Zone collection contains 15 complex clinical cases. For the TXIT exam, ChatGPT-3.5 and ChatGPT-4 have achieved the scores of 62.05% and 78.77%, respectively, highlighting the advantage of the latest ChatGPT-4 model. Based on the TXIT exam, ChatGPT-4's strong and weak areas in radiation oncology are identified to some extent. Specifically, ChatGPT-4 demonstrates better knowledge of statistics, CNS & eye, pediatrics, biology, and physics than knowledge of bone & soft tissue and gynecology, as per the ACR knowledge domain. Regarding clinical care paths, ChatGPT-4 performs better in diagnosis, prognosis, and toxicity than brachytherapy and dosimetry. It lacks proficiency in in-depth details of clinical trials. For the Gray Zone cases, ChatGPT-4 is able to suggest a personalized treatment approach to each case with high correctness and comprehensiveness. Importantly, it provides novel treatment aspects for many cases, which are not suggested by any human experts. Both evaluations demonstrate the potential of ChatGPT-4 in medical education for the general public and cancer patients, as well as the potential to aid clinical decision-making, while acknowledging its limitations in certain domains. Owing to the risk of hallucinations, it is essential to verify the content generated by models such as ChatGPT for accuracy.

Implementation of a Holistic Review Process of US Allopathic Medical Students Eliminates Non-Comparable Metrics and Bias in General Surgery Residency Interview Invitations

Use of traditional scoring metrics for residency recruitment creates racial and gender bias. In addition, widespread use of pass/fail grading has led to noncomparable data. To adjust to these challenges, we developed a holistic review (HR) rubric for scoring residency applicants for interview selection. Single-center observational study comparing the proportion of underrepresented in medicine (URM) students and their United States Medical Licensing Exam (USMLE) scores who were invited for interview before (2015-2020) and after (2022) implementation of a holistic review process. General surgery residency program at a tertiary academic center. US allopathic medical students applying for general surgery residency. After initial screening, a total of 1514 allopathic applicants were narrowed down to 586 (38.7%) for HR. A total of 52% were female and 17% identified as URM. Based on HR score, 20% (118/586) of applicants were invited for an interview. The median HR score was 11 (range 4-19). There was a fourfold higher coefficient of variation of HR scores (22.3; 95% CI 21.0-23.7) compared to USMLE scores (5.1; 95% Cl 4.8-5.3), resulting in greater spread and distinction among applicants. There were no significant differences in HR scores between genders (p = 0.60) or URM vs non-URM (p = 0.08). There were no significant differences in Step 1 (p = 0.60) and 2CK (p = 0.30) scores between those who were invited to interview or not. On multivariable analysis, USMLE scores (OR 1.01; 95% CI 0.98-1.03), URM status (OR 1.71 95% CI 0.98-2.92), and gender (OR 0.94, 95% CI 0.60-1.45) did not predict interview selection (all p > 0.05). There was a meaningful increase in the percentage of URM interviewed after HR implementation (12.9% vs 23.1%, p = 0.016). The holistic review process is feasible and eliminates the use of noncomparable metrics for surgical applicant interview invitations and increases the percentage of URM applicants invited to interview.

Exploring Radiation Oncology Representation on the National Board of Medical Examiners (NBME) Official Practice Material for the Undergraduate United States National Standardized Medical Board Examinations

Background The portrayal of oncology-related topics on the National Board of Medical Examiners (NBME) published practice material for undergraduate medical education (UGME) standardized examinations including the United States Medical Licensing Exams (USMLE) Step 1, Step 2 Clinical Knowledge (CK), and NBME Clinical Science Subject Examinations ('Shelf') is largely unknown. The goal of this study is to explore the presence and characterization of radiation oncology topics, including radiation therapy (RT), on these practice examinations. Methods All current and commercially accessible NBME published self-assessments and sample questions for the aforementioned examinations will be queried for content pertaining to oncology and associated treatment modalities of RT, surgery, and systemic therapy. The total volume of oncology-related questions and its distribution across all 3 major practice examination types will be recorded. The questions will be cataloged by selected attributes including context, treatment modality appearing as the correct answer for "next best step" (NBS) prompt, and answers appearing as the incorrect answer for NBS prompt. Results 1320 USMLE Step 1 practice questions are available for analysis. In preliminary analysis, minimal questions about RT were noted; all appearing to involve concepts of associated toxicity. 920 USMLE Step 2CK practice questions are available for analysis. RT-related questions appeared with higher frequency on the clinical science preparation material (n=10) compared to basic science (n=4), while also notably appearing as a listed answer choice for NBS prompt against other treatment modalities. 1325 practice questions from relevant NBME shelf examinations were available. RT appeared sparingly on the practice NBME shelf examinations of Surgery (n=5), Medicine (n=4), Ob/Gyn (n=2), and Clinical Neurology (n=2). RT questions were absent from Pediatrics, Emergency Medicine, and Family Medicine Shelf practice examination material. In the final analysis of these practice questions from all 3 major examination types, we seek to conduct a comparative analysis to other treatment modalities. Discussion This work represents the first attempt, to our knowledge, to characterize RT topic appearance within the NBME published practice questions and their accompanying explanations for the USMLE Step 1, Step 2CK, and select NBME shelf examinations. Our analysis is important in assessing the presence of RT topics on medical student board preparation material, a potential avenue of RT content exposure in UGME. Careful review and comparative analyses of all oncology-related content can provide important insight for UGME-related RT education. Our anticipated analyses and future discussion may identify areas for improvements in the important education of oncology-related topics in UGME. The portrayal of oncology-related topics on the National Board of Medical Examiners (NBME) published practice material for undergraduate medical education (UGME) standardized examinations including the United States Medical Licensing Exams (USMLE) Step 1, Step 2 Clinical Knowledge (CK), and NBME Clinical Science Subject Examinations ('Shelf') is largely unknown. The goal of this study is to explore the presence and characterization of radiation oncology topics, including radiation therapy (RT), on these practice examinations. All current and commercially accessible NBME published self-assessments and sample questions for the aforementioned examinations will be queried for content pertaining to oncology and associated treatment modalities of RT, surgery, and systemic therapy. The total volume of oncology-related questions and its distribution across all 3 major practice examination types will be recorded. The questions will be cataloged by selected attributes including context, treatment modality appearing as the correct answer for "next best step" (NBS) prompt, and answers appearing as the incorrect answer for NBS prompt. 1320 USMLE Step 1 practice questions are available for analysis. In preliminary analysis, minimal questions about RT were noted; all appearing to involve concepts of associated toxicity. 920 USMLE Step 2CK practice questions are available for analysis. RT-related questions appeared with higher frequency on the clinical science preparation material (n=10) compared to basic science (n=4), while also notably appearing as a listed answer choice for NBS prompt against other treatment modalities. 1325 practice questions from relevant NBME shelf examinations were available. RT appeared sparingly on the practice NBME shelf examinations of Surgery (n=5), Medicine (n=4), Ob/Gyn (n=2), and Clinical Neurology (n=2). RT questions were absent from Pediatrics, Emergency Medicine, and Family Medicine Shelf practice examination material. In the final analysis of these practice questions from all 3 major examination types, we seek to conduct a comparative analysis to other treatment modalities. This work represents the first attempt, to our knowledge, to characterize RT topic appearance within the NBME published practice questions and their accompanying explanations for the USMLE Step 1, Step 2CK, and select NBME shelf examinations. Our analysis is important in assessing the presence of RT topics on medical student board preparation material, a potential avenue of RT content exposure in UGME. Careful review and comparative analyses of all oncology-related content can provide important insight for UGME-related RT education. Our anticipated analyses and future discussion may identify areas for improvements in the important education of oncology-related topics in UGME.

Problems Faced by International Medical Graduates during the United States Medical Licensing Exam and Match Pathway

Most students aspire to pursue higher education andimmigrate to countries that offer opportunities to growtheir character and succeed in their fields. Theseimmigrants face a lot of issues and challenges in theirinitial years of hard work to settle in a completelydifferent culture away from home. This opinion articleaims to bring forward some of the issues thatInternational Medical Graduates (IMGs) face during thewhole United State Medical Licensing Exam (USMLE)and Match Pathway

Diagnostic Radiology Fellowship Applicant Selection Criteria: A Survey of Program Directors

Introduction In this study, we aimed to investigate the importance of various characteristics used by diagnostic radiology subspecialty fellowship programs when selecting candidates for their fellowship programs. Methods An online survey was sent to program directors from multiple radiology fellowship programs in the United States. Multiple questions were asked, including applicant gender, personal statement, interview performance, research experience, United States Medical Licensing Exam (USMLE) performance, and residency program attended. Survey recipients were asked to rank these characteristics from least to most important. Results A total of 119 responses were collected (response rate of 26.2%), with a relatively balanced representation of the different subspecialties. An aggregate analysis of all the subspecialties showed interview performance, letters of recommendation, and residency program attended as the three most important factors considered by the program directors when selecting a candidate for their fellowship program. In contrast, the three least important factors were gender, prior subspecialty training, and minority status. The applicant's visa status, personal statement, and USMLE Step 3 performance were the only statistically significant (p<0.05) differences between the subspecialties. The women's and body imaging subspecialties rated the personal statement more important than musculoskeletal imaging and neuroradiology. Cardiothoracic and musculoskeletal imaging rated the applicant's visa status more important than neuroradiology. Women's and cardiothoracic imaging rated USMLE Step 3 performance as more important than musculoskeletal imaging. Conclusion Selecting the ideal candidate for a fellowship in radiology can be difficult. Our findings outline which applicant characteristics program directors believe are the most and least important when selecting an ideal candidate. For instance, personal characteristics such as interview performance and letters of recommendation are more valued than extracurricular activities such as research experience. Identifying these characteristics provides a better understanding of the fellowship application process and may guide future applicants.

414 Competitive Advantage: An Analysis of Matched and Unmatched United States Neurological Surgery Residency Applicants From 2009-2021

INTRODUCTION: National Resident Matching Program releases a detailed report of residency match statistics for each specialty. Securing a training position in a United States neurological surgery residency program is notoriously competitive, and the subspecialty has long maintained a match rate much lower than other specialties. There remains a paucity of recently studied longitudinal trends in academic and extracurricular characteristics of successfully matched neurological surgery residency applicants. METHODS: Residency applicant self-reported data, obtained from the American Association of Medical Colleges Electronic Residency Application Service, were collected from yearly “Results and Data: Main Residency Match” (2009-2021) and biennial “Charting Outcomes in the Match” (2009-2020) National Residency Match Program summary reports. RESULTS: From 2009-2021, the number of neurological surgery residency programs and positions increased from 97 to 115 and 191 to 234, respectively. Matched applicants’ average United States Medical Licensing Exam (USMLE) Step 1 scores significantly increased (239 to 248; p = 0.01332), while average USMLE Step 2 Clinical Knowledge scores rose (237 to 252; p = 0.05682) during the study period. Average number of distinct research experiences among matched applicants increased (3.3 to 6.1; p = 0.5398), while the average number of abstracts, presentations, and publications increased from 7.8 to 23.4 (p = 0.0746). Lastly, Alpha Omega Alpha (AOA) membership (p = 0.00077) and percentage graduating from a top National Institutes of Health-funded medical school (p = 0.00396) significantly differed between matched and unmatched neurological surgery residency applicants. CONCLUSIONS: Matched neurological surgery residency applicants had significantly fewer average number of distinct specialties ranked, higher USMLE Step 1 scores, greater AOA membership, and more frequently graduated from top 40 NIH-funded medical schools than their unmatched counterparts. Additional research considering applicant-level intersectional data is warranted to further understand the match process.

Response to: Comment on The AI and I: A Collaboration on Competence.

We thank the authors for their thoughtful comments on the present and future use of artificial intelligence in surgery.1 As they creatively demonstrated in their letter written by Open AI GPT-3 chatbot (ChatGPT), some AI technology performance has become very difficult to distinguish from human performance. It has even been reported that ChatGPT could likely pass the United States Medical Licensing Exam and the Bar Exam.2 The rapid rise of ChatGPT use has led to debates and uncertainty in numerous fields. For example, some educators are concerned that it will facilitate cheating. Others have embraced it. In an interview with National Public Radio, Dr. Ethan Mollick, Associate Professor of Management at the Wharton School of the University of Pennsylvania, stated that he not only allowed, but even required his students to use ChatGPT to facilitate ideas for class projects. He noted that use of AI is analogous to using a calculator to do math.3 “I think everybody is cheating…So what I’m asking students to do is just be honest with me.” Although the direct applications of ChatGPT to surgical care may be limited in the sense that it cannot perform operations, the impact that ChatGPT and similar technology will have on academic publishing is unknown. Will journals and publishers prohibit ChatGPT from being used? Will they use technology, some of which has already been developed, to identify and “root out” ChatGPT text?4 Conversely, will they take Dr. Mollick’s approach and welcome ChatGPT use and require disclosure? Cheating and plagiarism have always been challenges in academia and publishing. ChatGPT raises new questions about the process and traditional approach to applied intellect. With new technology that can generate high level, authentic appearing writing, publishers and journal editors will need to develop new strategies and standards for what is considered original or novel content. Of note, ChatGPT was not used to write this.

Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models.

We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.

Practical Applications of ChatGPT in Undergraduate Medical Education.

ChatGPT is a chatbot developed by OpenAI that has garnered significant attention for achieving at or near a passing standard on the United States Medical Licensing Exam (USMLE). Currently, researchers and users are exploring ChatGPT's broad range of potential applications in academia, business, programming, and beyond. We attempt outline how ChatGPT may be applied to support undergraduate medical education during the preclinical and clinical years, and highlight possible concerns regarding its use which necessitates the creation of formal policies and training by medical schools.