Residency Application Process Research Articles

Assessment of Gender Differences in Letters of Recommendation for Physical Therapy Residency Applications.

Letters of recommendation (LOR) are an integral component of physical therapy residency applications. Identifying the influence of applicant and writer gender in LOR will help identify whether potential implicit gender bias exists in physical therapy residency application processes. Several medical and surgical residency education programs have reported positive, neutral, or negative LOR female gender bias among applicants and writers. Little research exists on gender differences in LOR to physical therapy education programs or physical therapy residency programs. Seven hundred sixty-eight LOR were analyzed from 256 applications to 3 physical therapy residency programs (neurologic, orthopaedic, sports) at one institution from 2014 to 2020. Thematic categories were developed to identify themes in a sample of LOR. Associations between writer and applicant gender were analyzed using summary statistics, word counts, thematic and psycholinguistic extraction, and rule-based and deep learning Natural Language Processing . No significant difference in LOR word counts were found based on writer or applicant gender. Increased word counts were seen in sports residency LOR compared with the orthopaedic residency. Thematic analysis showed LOR gender differences with male applicants receiving more positive generalized recommendations and female applicants receiving more comments regarding interpersonal relationship skills. No thematic or psycholinguistic gender differences were seen by LOR writer. Male applicants were 1.9 times more likely to select all male LOR writers, whereas female applicants were 2.1 times more likely to choose all female LOR writers. Gender differences in LORs for physical therapy residencies were found using a comprehensive Natural Language Processing approach that identified both a positive recommendation male applicant gender bias and a positive interpersonal relationship skill female applicant gender bias. Applicants were not harmed nor helped by selecting LOR writers of the opposite gender. Admissions committees and LOR writers should be mindful of potential implicit gender biases in LOR submitted to physical therapy residency programs.

Use of Signaling in Family Medicine Residency Interviewing.

Although signals have been used in the residency application process by other specialties, family medicine residency directors have not previously participated. With applicant signal information available for the first time in the 2023-2024 application cycle, the current study describes family medicine residency program directors' intended use of signals and provides benchmarking descriptive data that may help inform best practices and future studies. A total of 691 of the 745 family medicine program directors in US family medicine residency programs accredited by the Accreditation Council for Graduate Medical Education were surveyed. We used ꭓ2 and Pearson correlation analyses to examine how program directors of family medicine residency programs intended to use signaling and their perceived impact of signaling on the residency interviewing process. Most program directors indicated that applicant signals would assist them in deciding who to invite for an interview and would be a positive factor in a holistic review process. However, program directors also noted that rotation experience or geographic ties would be more powerful inducements to interview or rank a specific candidate. Program directors did not indicate a belief that signals would decrease interview season stress or workload. Signals may play an important role in the residency application process for family medicine in 2023-2024. While signals are not anticipated to decrease application workload or stress, a signal may be an important mechanism for a specific applicant to distinguish themselves with a program.

Machine Learning for Predictive Analysis of Otolaryngology Residency Letters of Recommendation.

Letters of recommendation (LORs) are a highly influential yet subjective and often enigmatic aspect of the residency application process. This study hypothesizes that LORs do contain valuable insights into applicants and can be used to predict outcomes. This pilot study utilizes natural language processing and machine learning (ML) models using LOR text to predict interview invitations for otolaryngology residency applicants. A total of 1642 LORs from the 2022-2023 application cycle were retrospectively retrieved from a single institution. LORs were preprocessed and vectorized using three different techniques to represent the text in a way that an ML model can understand written prose: CountVectorizer (CV), Term Frequency-Inverse Document Frequency (TF-IDF), and Word2Vec (WV). Then, the LORs were trained and tested on five ML models: Logistic Regression (LR), Naive Bayes (NB), Decision Tree (DT), Random Forest (RF), and Support Vector Machine (SVM). Of the 337 applicants, 67 were interviewed and 270 were not interviewed. In total, 1642 LORs (26.7% interviewed) were analyzed. The two best-performing ML models in predicting interview invitations were the TF-IDF vectorized DT and CV vectorized DT models. This preliminary study revealed that ML models and vectorization combinations can provide better-than-chance predictions for interview invitations for otolaryngology residency applicants. The high-performing ML models were able to classify meaningful information from the LORs to predict applicant interview invitation. The potential of an automated process to help predict an applicant's likelihood of obtaining an interview invitation could be a valuable tool for training programs in the future. N/A Laryngoscope, 2024.

Implementing a Neurology Mentorship Program to Ease Residency Application Process (P1-7.008)

Implementing a Neurology Mentorship Program to Ease Residency Application Process (P1-7.008)

Changing Perceptions About the Dermatology Residency Application Process Among Applicants and Program Directors: 2020-2022.

Importance Over the last two years, dermatology has undergone significant reforms in the residency application process in efforts to reduce applicant stress, increase equity, and due to the COVID-19 pandemic. Objective We aimed to determine applicant and program director (PD) perspectives in implementing these changes over the last two application cycles. Design, setting, and participants Anonymous online surveys were administered by theAssociation of American Medical Colleges (AAMC) to PDs and applicants from the 2021-2022 dermatology residency application cycle. These results were compared with similar online surveys distributed after the 2020-2021 cycle. Results Coordinated interview release was introduced in the 2020-2021 dermatology application cycle. At that time, 57% of PDs and 84% of applicants wished that more programs participated in the release, compared to 53% and 84%, respectively, in the 2021-2022 cycle. In 2021, 28% of PDs reported matching applicants from their home institution higher on their list compared to 14% in 2022. In 2021 and 2022, 94% of PDs reported that diversity was an explicit goal in their application process. However, in 2021, 33% of PDs reported that they matched no UIMs (underrepresented in medicine) in their cohort, which grew to 39% in 2022. Conclusions This study identifies key trends in applicant and PD perspectives associated with changes in the application process such as coordinated interview release, virtual interviews, and emphasis on diversity. Additional data is needed from subsequent cycles in order to determine the efficacy of these reforms.

Considerations for Artificial Intelligence (AI) Use in the Dermatology Residency Application Process.

Artificial intelligence (AI) has gained traction in the field of dermatology, but the role of AI in the dermatology residency application process has been minimally explored. We encourage dermatology residency programs to proactively set guidelines to applicants and letter writers on the use of AI tools. Further, programs should consider implementing natural language processing models in reviewing residency applications to promote principles of diversifying the dermatology workforce.

A Study on Pre-clinical Anaesthesia Elective in Shaping Medical Students' Knowledge and Career Perspectives

Anesthesiology stands out as one of the rapidly growing and highly competitive medical specialties, necessitating students to cultivate an interest in anesthesia early and prepare extensively for clinical electives. Pre-clinical electives that introduce students to anesthesia and ready them for clinical settings in the early stages of their academic journey may confer benefits and enhance their competitiveness as residency applicants. This study implemented an eight-week pre-clinical elective titled "The History of Anesthesia" for first- and second-year students at Baylor College of Medicine. Eighty-eight students participated in the elective, with thirteen completing both pre-course and post-course surveys. The survey, consisting of thirteen questions related to anesthesia interest and relevant clinical knowledge, was analyzed, and statistical significance was calculated. The survey results demonstrated significantly elevated levels of self-assessed knowledge in the post-course survey compared to the pre-course survey (p < .05). Notably, topics considered more clinically relevant, such as modern anesthetic techniques, regional anesthesia, and obstetric anesthesia, exhibited significantly higher average survey results in the post-course compared to the pre-course results. The implementation of pre-clinical electives focusing on success in clinical settings appears beneficial for students aspiring to apply to an anesthesiology residency. Institutions should consider exposing students to anesthetic topics earlier in their academic journey to adequately prepare them for the competitive residency application process.

An Innovative Approach to E-mentorship for Black Applicants to Graduate Medical Education (GME) Programs

The path to medical education is complex and challenging. This is particularly true for Black applicants seeking to pursue graduate medical education. These individuals encounter a multitude of challenges due to disparities in access and resources. These hurdles include difficulties in navigating the residency application process and lower matriculation rates. To address these issues, medical schools need to support diverse and innovative mentorship initiatives, including E-mentoring that is specifically tailored to Black applicants. This article examines the benefits of E-mentoring from the perspectives of our innovative E-mentorship platforms.

Racial and ethnic bias in letters of recommendation in academic medicine: a systematic review.

Letters of recommendations (LORs) are key components of academic medicine applications. Given that bias against students and trainees underrepresented in medicine (UIM) has been demonstrated across assessment, achievement, and advancement domains, the authors reviewed studies on LORs to assess racial, ethnic, and UIM differences in LORs. Standardized LORs (SLORs), an increasingly common form of LORs, were also assessed for racial and ethnic differences. A systematic review was conducted for English-language studies that assessed racial or ethnic differences in LORs in academic medicine published from database inception to July 16, 2023. Studies evaluating SLORs underwent data abstraction to evaluate their impact on the given race or ethnicity comparison and outcome variables. Twenty-three studies describing 19,012 applicants and 41,925 LORs were included. Nineteen studies (82.6%) assessed LORs for residency, 4 (17.4%) assessed LORs for fellowship, and none evaluated employment or promotion. Fifteen of 17 studies (88.2%) assessing linguistic differences reported a significant difference in a particular race or ethnicity comparison. Of the 7 studies assessing agentic language (e.g., "strong," "confident"), 1 study found fewer agentic terms used for Black and Latinx applicants, and 1 study reported higher agency scores for Asian applicants and applicants of races other than White. There were mixed results for the use of communal and grindstone language in UIM and non-UIM comparisons. Among 6 studies, 4 (66.7%) reported that standout language (e.g., "exceptional," "outstanding") was less likely to be ascribed to UIM applicants. Doubt-raising language was more frequently used for UIM trainees. When SLORs and unstructured LORs were compared, fewer linguistic differences were found in SLORs. There is a moderate bias against UIM candidates in the domains of linguistic differences, doubt-raising language, and topics discussed in LORs, which has implications for perceptions of competence and ability in the high-stakes residency and fellowship application process.

Urology Residency Applicant Selection: Program Directors’ New Criteria

ObjectiveTo evaluate the impact of the recent changes to the urology residency application process on the criteria utilized by residency program directors (PDs) for interview invitations and their perspectives concerning these changes. MethodsOne hundred thirty-seven urology residency PDs were invited to participate in an anonymous survey to explore interview selection criteria and the impact of the increase in preference signals (PS) per applicant. ResultsFifty-eight PDs (42.8%) completed the survey. The highest-ranked criteria were letters of recommendation (LoR) and successful sub-internship (sub-I) at the PD’s institution, without statistically significant differences between these two. Gender, ethnicity and medical school prestige were the lowest rated criteria, without significant differences between these three. Compared to before the increase in the number of PS per applicant, 80.7% of PDs reported that not receiving a PS from an applicant this cycle would more negatively impact the chances of offering an interview to that applicant. Moreover, 12.2% stated they would not interview any applicants who did not send a PS. Finally, 62.1% of PDs believed recent changes worsened the process. ConclusionRecent changes impacted PDs applicant evaluation, with the highest ranked criteria being LoRs and sub-I. Paradoxically, the increase in the number of PS per applicant has increased their importance as applicants are much less likely to receive interview offers from programs they have not signaled. Lastly, most PDs believe changes have worsened the evaluation process.

Perceptions and Expectations Regarding the Step 1 Score Change Among Surgical and Non-Surgical Program Directors: A Systematic Review and Meta-Analysis.

There are currently no studies examining differences in perceptions and expected impact of the Step 1 score change to pass/fail between surgical and non-surgical program directors (PDs). We conducted a systematic review in May 2023 of PubMed, Scopus, Web of Science, and PSYCInfo to evaluate studies examining PDs' perspectives regarding the Step 1 score change. We performed random-effects meta-analyses to determine differences in perspectives among surgical and non-surgical PDs. Surgical PDs (76.8% [95% CI, 72.1%-82.0%], I2 = 52%) reported significantly greater rates of disagreement with the score change compared to non-surgical (65.1% [95% CI, 57.9%-73.1%], I2 = 69.7%) (P = .01). Surgical PDs also reported significantly greater rates of agreement that the score change will increase the difficulty in objectively comparing applicants (88.1% [95% CI, 84.6%-91.7%], I2 = 16.4%), compared to non-surgical (81.0% [95% CI, 75.6%-86.8%], I2 = 72.6%) (P = .04). There was less heterogeneity among non-surgical PDs (88.7% [95% CI, 86.2%-91.2%], I2 = 0%), compared to surgical (84.7% [95% CI, 79.0%-90.8%], I2 = 67.3%), regarding expected increases in emphasis on Step 2, although the difference in rates of agreement was not statistically significant. Overall, there is significant heterogeneity in the literature regarding expected changes in the residency application review process. Most PDs reported significant disagreement with the score change, greater expected difficulty in objectively evaluating applicants, and greater emphasis on Step 2, with surgical PDs reporting greater rates of disagreement, greater expected difficulty, and heterogeneity regarding expected increases in emphasis on Step 2, compared to non-surgical. Additionally, there is significant heterogeneity in the overall literature regarding expected changes in the residency application review process. Further research is needed to establish evidence-based guidelines that improve the overall residency application process for all stakeholders.

Measuring and Predicting Faculty Consensus Rankings of Standardized Letters of Evaluation.

Background Standardized letters of evaluation (SLOE) are becoming more widely incorporated into the residency application process to make the letter of recommendation, an already critical component in a residency application packet, more objective. However, it is not currently known if the reviewers of these letters share consensus regarding the strength of an applicant determined by their SLOE. Objective We measured the level of faculty agreement regarding applicant competitiveness as determined by SLOEs and the ability of 2 algorithms to predict faculty consensus rankings. Methods Using data from the 2021-2022 Match cycle from the Council of Residency Directors in Emergency Medicine SLOE Database as a blueprint, authors created 50 fictional SLOEs representative of the national data. Seven faculty then rated these SLOEs in order of applicant competitiveness, defined as suggested rank position. Consensus was evaluated using cutoffs established a priori, and 2 prediction models, a point-based system and a linear regression model, were tested to determine their ability to predict consensus rankings. Results There was strong faculty consensus regarding the interpretation of SLOEs. Within narrow windows of agreement, faculty demonstrated similar ranking patterns with 83% and 93% agreement for "close" and "loose" agreement, respectively. Predictive models yielded a strong correlation with the consensus ranking (point-based system r=0.97, linear regression r=0.97). Conclusions Faculty displayed strong consensus regarding the competitiveness of applicants via SLOEs, adding further support to the use of SLOEs for selection and advising. Two models predicted consensus competitiveness rankings with a high degree of accuracy.

Bridge to M4 (B2M4): An innovative residency application preparation curriculum.

The medical residency application process is daunting, time-consuming, and variable, making mentorship in preparation for this process important. In light of changes to the residency application process due to the COVID-19 pandemic, medical students identified a need to create a formal residency application preparation curriculum. The curriculum focused on reflection, while also furthering professional development and preparing for the nuances of the residency application cycle through structured advising. Bridge to M4 (B2M4) included reflective activities to help guide curriculum vitae (CV) refinement, personal and professional values clarification, residency program exploration, elevator pitch formation, personal statement development, mentor identification, guidance on requesting letters of recommendation, and virtual residency interview preparation. The B2M4 elective first took place in June 2020 with 36 rising fourth year students enrolled. Perceived effectiveness of the curriculum was assessed by having student cohorts evaluate the importance of the CV, personal statement, general preparation, and residency program selection before and after the course. Student feedback was reviewed with content analysis, which highlighted the usefulness of a structured residency preparation course that was rooted in reflection and incorporated faculty mentorship and feedback. This student-driven, innovative curricular design provided students the opportunity to prepare for the residency application process through reflective activities including values clarification, strengths identification, and goal setting. B2M4 demonstrates a process that can be used not only during the Residency Match process, but also throughout graduate medical education and future career decisions.

Best practices for PGY1 letters of intent: Recommendations from a modified Delphi process.

In an effort to expedite the publication of articles, AJHP is posting manuscripts online as soon as possible after acceptance. Accepted manuscripts have been peer-reviewed and copyedited, but are posted online before technical formatting and author proofing. These manuscripts are not the final version of record and will be replaced with the final article (formatted per AJHP style and proofed by the authors) at a later time. The aim of this study was to provide consensus recommendations from residency program leaders on letters of intent (LOIs) written by postgraduate year 1 (PGY1) pharmacy residency candidates. A 3-round modified Delphi process was used to determine consensus among PGY1 residency program leaders across the country. A screening and demographic survey was utilized to ensure representation of panelists. The initial items for round 1 of the study were developed using existing published literature, with piloting by 2 residency program directors. For each round, respondents rated items on a 7-point Likert scale, with opportunities to provide qualitative feedback and modifications for lower-rated items. For future rounds, items were adjusted based on respondent feedback. Only items meeting predefined consensus were included in the final recommendations. A total of 254 pharmacists were invited to participate in the panel, with 41 completing the demographic and study consent survey. There were 35 participants in round 1, 34 of whom remained for rounds 2 and 3. The panel created 18 LOI recommendations for PGY1 residency candidates. Most recommendations were focused on the content of the LOI, while others were related to formatting. The recommendations from this study can be employed by PGY1 pharmacy residency candidates to enhance their likelihood of success in the residency application process.

Practical Tips for Undergraduate Medical Education Advisors in Residency Application Signaling.

The residency application process has become increasingly complex for medical students and advisors to navigate. Program signaling was piloted to improve applicants' abilities to obtain interview offers at programs they were strongly interested in. The initial positive results led to expansion of signaling to additional specialties over the next two application cycles. Despite the benefits of program signaling, the variation in signaling practices among specialties has presented challenges for both advisors and students when determining how to best allocate signals. The aim of this study is to identify students' perceptions of the signaling process, how this may impact outcomes, and to guide future educational programming. This is an exploratory original survey study of students in a US allopathic medical school applying in ERAS for the 2023 residency cycle. The survey was developed to determine students' understanding of how programs would use signals in the application process and assess strategies students used to allocate signals. We compared program signals to student interview offers and match outcomes using descriptive statistics. 57 of 96 eligible students completed the survey. 51% signaled a range of programs based on their perceived competitiveness for the program while 40% signaled programs of interest regardless of perceived competitiveness. 53% of students thought sending a signal would increase their chance of an interview, while 42% were unsure how the signal would be used by residency programs. Students received interviews at 49% of the programs signaled, which increased to 56.5% when specialties offering more than 7 signals were excluded. 35% of students matched at a signaled program. Students' perceptions and strategies related to the signaling process are varied and may impact interview offers. Advisors should monitor and review internal institutional trends to help inform future educational programming to optimize signal allocation for their students.

Navigating the couples match: Key challenges and lessons learned from two general surgery applicants

The Couples Match presents unique obstacles and considerations to medical students who are already participating in a rigorous residency application process. We aim to describe the distinct challenges presented by the Couples Match and present advice from our own experience successfully matching into general surgery residency as a couple.

Understanding the Match: The Effect of Signaling, Demographics, and Applicant Characteristics on Match Success in the Orthopaedic Residency Application Process.

As of the 2022 to 2023 match cycle, orthopaedic residency programs began offering applicants 30 signals as part of a preference signaling program. Many have assumed that signals would become powerful tools in the match process, yet no objective data currently exist analyzing their effect. This study aims to provide such analysis while also offering comparisons with other factors affecting match success. Self-reported survey data on applicants and applications from 2017 to 2023 from the Texas Seeking Transparency in Application to Residency database were queried. Variables associated with receiving interviews and match success were analyzed using two-sided Student t -tests, chi-squared tests, variance ratio testing, and receiver operating characteristic analysis. Compared with 2017 to 2022, 2023 applicants submitted fewer applications (61.8 versus 78; P < 0.001), received fewer interview offers (11.6 versus 13.8; P < 0.001), and interview offers were spread more evenly among applicants (SD, 6.82 versus 9.10; P < 0.001). For 2023 applications, odds of securing an interview were increased most by away rotations (odds ratios [OR] 61.8; P < 0.001), use of a signal (OR, 9.61; P < 0.001), and geographic connection (OR, 4.70; P < 0.001). Female applicants received more interview offers from signaled programs than their male counterparts (11.2 versus 8.94; P = 0.003). Applicant variables most predictive of match success in 2023 were interview offers (area under the receiver operating characteristic curve [AUC] = 0.802), step 2 CK score (AUC = 0.673), and step 1 score (AUC = 0.648). The preference signaling program seems to be accomplishing its goals of reducing applications and more evenly distributing interviews. Away rotations, signals, and geographic connections represent the strongest predictors of applications resulting in a successful match. Applicants must use their signals carefully to maximize their chance of success. Level III.

Improving Transparency in the Residency Application Process: Survey Study.

Increasing numbers of residency applications create challenges for applicants and residency programs to assess if they are a good fit during the residency application and match process. Applicants face limited or conflicting information as they assess programs, leading to overapplying. A holistic review of residency applications is considered a gold standard for programs, but the current volumes and associated time constraints leave programs relying on numerical filters, which do not predict success in residency. Applicants could benefit from increased transparency in the residency application process. This study aims to determine the information applicants find most beneficial from residency programs when deciding where to apply, by type of medical school education background. Match 2023 applicants voluntarily completed an anonymous survey through the Twitter and Instagram social media platforms. We asked the respondents to select 3 top factors from a multiple-choice list of what information they would like from residency programs to help determine if the characteristics of their application align with program values. We examined differences in helpful factors selected by medical school backgrounds using ANOVA. There were 4649 survey respondents. When responses were analyzed by United States-allopathic (US-MD), doctor of osteopathic medicine (DO), and international medical graduate (IMG) educational backgrounds, respondents chose different factors as most helpful: minimum United States Medical Licensing Examination (USMLE) or Comprehensive Osteopathic Medical Licensing Examination (COMLEX) Step 2 scores (565/3042, 18.57% US-MD; 485/3042, 15.9% DO; and 1992/3042, 65.48% IMG; P<.001), resident hometown region (281/1132, 24.82% US-MD; 189/1132, 16.7% DO; and 662/1132, 58.48% IMG; P=.02), resident medical school region (476/2179, 22% US-MD; 250/2179, 11.5% DO; and 1453/2179, 66.7% IMG; P=.002), and percent of residents or attendings underrepresented in medicine (417/1815, 22.98% US-MD; 158/1815, 8.71% DO; and 1240/1815, 68.32% IMG; P<.001). When applying to residency programs, this study found that the factors that respondents consider most helpful from programs in deciding where to apply differ by educational background. Across all educational groups, respondents want transparency around standardized exam scores, geography, and the racial or ethnic backgrounds of residents and attendings.

Navigating obstacles: Perspectives of US pharmacy residency directors on non‐US candidates for PGY‐1 residencies: A survey‐based research

AbstractIntroductionPharmacy education and postgraduate training in the United States (US) are esteemed, drawing international students to American Council for Pharmacy Education (ACPE)‐accredited schools and pursuing residency training after graduation. Despite this, limited literature addresses the obstacles international students face in pursuing American Society of Health‐System Pharmacists (ASHP) accredited residency programs.ObjectivesTo summarize potential eligibility barriers for international graduates of ACPE programs during the residency application process from residency program directors' (RPDs) perspectives.MethodsA 16‐question electronic survey was developed and distributed to all RPDs of US‐based ASHP‐accredited postgraduate year‐1 (PGY‐1) programs in July 2021. Questions addressed whether non‐US citizens were eligible to apply, barriers to recruiting non‐US citizen candidates, and barriers to offer visa sponsorships. Descriptive statistics were used to summarize the data.ResultsA total of 196 programs responded to the survey (response rate, 13.5%). Eighty‐nine (45.4%) programs reported that candidates with F‐1/optional practical training (OPT) status were eligible to apply for their program, while 86 programs (43.9%) reported candidates were not eligible. Barriers for recruiting candidates with F‐1/OPT status included lack of resources to support application process (n = 42), citizenship requirements (n = 41), institution policy (n = 33), department preference (n = 12), and unfamiliar with procedures in selecting these candidates (n = 18). Over 70% (n = 146) would not offer visa sponsorship for pharmacy resident positions. Barriers to offering sponsorship were lack of support resources (n = 62), cost (n = 60), time preparing applications (n = 46), citizenship requirements (n = 45), and department preference (n = 23).ConclusionThis study investigates the obstacles encountered by non‐US citizens in the residency application process, as perceived by program directors. The primary obstacle in recruiting candidates with F‐1/OPT status and providing visa sponsorships revolves around the inadequate support and resources available for the application process.

Rethinking the Internal Medicine Residency Application Process to Prioritize the Public Good: A Consensus Statement of the Alliance for Academic Internal Medicine

Rethinking the Internal Medicine Residency Application Process to Prioritize the Public Good: A Consensus Statement of the Alliance for Academic Internal Medicine