Underrepresented Minority Status Research Articles

Diversity on the American Academy of Orthopaedic Surgeons National Meeting Podium: Changes Over Two Decades.

There has been an increase in diversity initiatives regarding selecting speakers for the American Academy of Orthopaedic Surgeons (AAOS) annual meeting and courses. The purpose of this study was to determine the percentage of female or underrepresented minority (URM) speakers for instructional course lectures (ICLs) and AAOS courses over the past 2 decades including a surrogate for expertise. For 2002, 2012, and 2022, the academic and demographic information of speakers and the number of publications at the time of their speaking role were obtained and compared by sex and URM status. Owing to the unequal sample sizes between male versus female cohorts and URM versus non-URM cohorts, the Welch t -test was used. The percentage of ICL and AAOS course speakers who were female increased over time (ICL, AAOS courses): 2002 (2.6%, 3.3%), 2012 (3.9%, 6.3%), and 2022 (11.8%, 15.5%) ( P < 0.001, P < 0.001). The percentage of female AAOS fellows in these years was 2.9%, 4.7%, and 7.4%, respectively. For ICLs and AAOS course speakers, female presenters had fewer publications than male counterparts (ICL, AAOS courses): 2002 ( P < 0.001, P = 0.048), 2012 ( P = 0.003, P < 0.001), and 2022 ( P < 0.001, P < 0.001). For ICLs in 2022, URM speakers had a similar number of publications compared with non-URM speakers. In 2022, URMs comprised 6.9% of ICL speakers and 4% of AAOS fellows. For 2022 ICLs, there were no significant differences in academic institution, position, or region when compared by sex or URM status. For AAOS courses, the percentage of URM speakers increased over time: 2002 (1.1%), 2012 (4.5%), and 2022 (8.6%). For AAOS courses, URM presenters had similar publications compared with non-URM presenters in 2002 and 2022 but less in 2012 ( P = 0.027). The percentage of women and URMs presenting ICLs and AAOS courses has increased over the past 2 decades and exceeded the percentage they represent in the AAOS by over 50%. The female cohort has fewer publications, on average, than the male cohort for all years evaluated, indicating no institutional bias against female speakers.

Associations between Goal Orientation and Self-Regulated Learning Strategies are Stable across Course Types, Underrepresented Minority Status, and Gender

Associations between Goal Orientation and Self-Regulated Learning Strategies are Stable across Course Types, Underrepresented Minority Status, and Gender

Time to Degree, Identity, and Grant Writing: Lessons Learned From a Mixed-Methods Longitudinal Case Study of Biosciences PhD Students

Time to degree completion is an important metric of academic progress and success for doctoral students. It is also a common way for educational stakeholders to compare programs even if the content of the degree programs varies. But what types of behaviors and experiences are associated with faster times to degree? In this study, we examine the relationship between receiving competitive research awards (e.g., grant writing) and time to degree for PhD students. We organize our analyses by student identities, specifically gender and underrepresented racial minority (URM) status, to examine differences in time to degree based on student demographics. Our dataset included students that graduated between academic years 2008 to 2009 through 2018 to 2019. We also interviewed students currently enrolled in these same programs. We find that underrepresented minority women who won competitive research awards graduate faster than all other students who also won awards but also report lower levels of advisor support. Underrepresented minority women and all underrepresented minority students who won research awards tended to graduate faster compared to other groups of students who did not win awards. Students who reported feeling supported by their advisors, most of which reflected hands-on guidance through the writing process, in the interviews were more likely to apply for grants. Combined, these results highlight that writing grants and specific types of advisor support may influence faster times to degree for biosciences PhD students. This study suggests similar introspective analyses at other institutions and databases are needed.

Group peer mentoring is effective for different demographic groups of biomedical research faculty: A controlled trial.

Improved mentoring of midcareer researchers in medical schools has been identified as an important potential avenue for addressing low vitality and high burnout rates in faculty, and the scarcity of both underrepresented minority (URM) faculty and women in biomedical research. To address the need for widescale effective mentoring, we sought to determine whether a group peer mentoring intervention (C-Change Mentoring and Leadership Institute) for early midcareer research faculty was effective for different demographic groups in a controlled trial. Thirty-five diverse early midcareer faculty and 70 propensity-matched (PM) control subjects matched to intervention subjects on a) study inclusion criteria; b) gender, race, and ethnicity, degree, rank, years of experience, publications, grants; and c) pretest survey outcome variables, participated in the intervention. The C-Change Participant Survey assessed vitality, self-efficacy in career advancement, research success, mentoring others, valuing diversity, cognitive empathy, and anti-sexism/anti-racism skills at pretest and intervention completion. Analysis using multiple regression models included outcome pretest values and indicator variables for intervention, gender, URM status, and MD vs. PhD. Hypotheses regarding differential effectiveness of the intervention by demographic group were tested by including cross-product terms between the demographic indicator variables and the intervention indicator. Missing data were addressed using chained equations to create 100 data sets. The intervention participants had significantly higher (favorable) scores than PM controls for: self-assessed change in vitality; self-efficacy for career advancement, research, and mentoring others; cognitive empathy; and anti-sexism/racism skills. The benefits of the intervention were nearly identical across: gender, URM vs non-URM faculty, and degree MD/PhD, except vitality significantly increased for non-URM subjects, and not for URM faculty. Self-assessed change in vitality increased for URM and non-URM. The intervention worked successfully for enhancing vitality, self-efficacy and cross-cultural engagement across different demographic groups of biomedical research faculty.

Potential Impact of Race and Ethnicity on Pharmacy Residency Match Rate and Match Participation

ObjectiveThe purpose of this study was to investigate if a correlation exists between a student’s self-identified race or ethnicity and their rates of participating in the match or matching with a pharmacy residency. MethodsThis retrospective cohort study included pharmacy graduates from 2016 through 2021 at two schools/colleges of pharmacy, a Historically Black College or University and a non-Historically Black College or University. The primary outcome was to compare the percentage of underrepresented minority (URM) students and non-URM students who successfully matched with a postgraduate year 1 pharmacy residency. The secondary outcomes compared rates of participation in the match and the effects of student-related factors that influenced match success and participation rates between URM students and non-URM students. ResultsOf the 900 included students, 273 participated in the match. The match rate among students participating in the match was 53.5% (146 of 273) with 50.3% (84 of 167) in the URM group compared to 58.5% (62 of 106) in the non-URM group (P = .186) successfully matching. Rates of match participation were lower for students identified as URM (26% vs 39.3%, P < .001). When adjusting for previously published predictors of match success, URM status did not significantly affect match rates or match participation. ConclusionAt two schools of pharmacy, URM students were less likely to participate in the match but just as likely to match to an American Society of Health-System Pharmacists residency in the univariable analysis. However, when adjusting for other characteristics, there was no difference in match participation or match success in students who self-identified as URM.

Academic seminars as tool for clinical and translational science education and dissemination: Perceptions and priorities in the COVID-19 era.

Academic seminars are an important venue through which investigators in health services research (HSR) and other clinical and translational science disciplines can share their work, gather feedback, and connect with colleagues. However, the format and focus of these seminars shifted abruptly when the coronavirus disease 2019 (COVID-19) pandemic necessitated social distancing and underscored the salience of health equity. Little is known about how contemporary academic seminars are meeting the evolving needs of the HSR community. We surveyed 2021-2022 participants in a virtual HSR seminar series to understand experiences of and priorities for the seminars. We also compared results stratified by self-reported under-represented minority (URM) status. Of 45 respondents (including 26 faculty members, 8 trainees, 9 staff members, and 2 community partners), 38% self-identified as URM. Participants reported high satisfaction with seminar quality, diversity of topics, and audience participation (median ≥4 for all outcomes in Likert-scaled items where 5 = very satisfied). Knowledge acquisition, understanding of research methods, and collaboration were commonly cited as benefits of seminar attendance. Implementation science content and external collaboration were most often endorsed as priorities for future seminars. URM participants were over three times more likely than non-URM participants to cite learning about engaging community stakeholders and historically excluded populations as a benefit of seminar participation. Virtual academic seminars can be an effective modality for knowledge sharing and collaboration worth continuing after COVID-19 restrictions are lifted. Emphasizing equity, diversity, and inclusion (EDI) values in these seminars may hold potential for advancing academic departments' own EDI goals.

Factors associated with perceived stress and depressive symptoms among college students during the COVID-19 pandemic

The COVID-19 pandemic had detrimental effects on college students’ psychosocial well-being due to campus closures starting in Spring 2020. This study examined changes in perceived stress and depressive symptoms before and during the pandemic using longitudinal data collected from students over the 2019-2020 academic year. We also compared data from this sample in Fall 2019 (N = 282) to data collected from another sample in Fall 2020 (N = 240). Latent change modeling showed increases in stress during Spring 2020. Additionally, the Fall 2020 sample had higher levels of stress than the Fall 2019 sample. Differences in stress and depressive symptoms did not depend on underrepresented minority (URM) status, income, gender, or membership in Greek life or other organizations. However, third- and fourth-year students and student-athletes experienced more stress during the pandemic. Findings suggest an adverse and differential impact of the pandemic on students’ psychosocial well-being.

What Predicts Undergraduate Students' Decision to Pursue a Career in Biomedical/Behavioral Research within an Upper-Division Research Training Program? A Study of Trainees' Science Identity and Educational Outcomes.

This study examined how science identity and positive educational outcomes relate to student trainees' decision to pursue a research career in health-related sciences, within the context of a two-year research training program that prepares diverse undergraduate students for a research career. In analyses using the evaluation data, science identity and one of the positive educational outcomes significantly predicted trainees' decision to pursue a research career in biomedical and behavioral sciences. In general, students with stronger science identity and interest in pursuing research in academia exhibited a firmer decision to pursue a research career in sciences. In a separate analysis that examined the associations between the trainees' decision to pursue a health-related research career and their underrepresented minority identities, gender, and disciplinary track, results revealed that: (1) the interaction between trainees' disciplinary track (i.e., biomedical vs. behavioral sciences) and their total number of underrepresented minority status was associated with their decision to pursue a research career, but (2) the gender x disciplinary track interaction was not. Emphasizing the need for supporting diverse undergraduate trainees to solidify their science identity and prepare them for a research career in academia, we discuss implications of our findings for research training programs with similar aims.

Workplace Belonging of Women Healthcare Professionals Relates to Likelihood of Leaving.

There is a high rate of attrition of professionals from healthcare institutions, which threatens the economic viability of these institutions and the quality of care they provide to patients. Women professionals face particular challenges that may lower their sense of belonging in the healthcare workplace. We sought to test the hypothesis that workplace belonging of women healthcare professionals relates to the likelihood that they expect to leave their institution. Participants of a continuing education course on women's leadership skills in health care completed a survey about their experiences of belonging in workplace and their likelihood of leaving that institution within the next 2 years. An association between workplace belonging (measured by the cumulative number of belonging factors experienced, scale 0-10) and likelihood of leaving (measured on a 5-point Likert scale) was evaluated using ordinal logistic regression. The relative importance of workplace belonging factors in predicting the likelihood of leaving was assessed using dominance analysis. Ninety-nine percent of survey participants were women, and 63% were clinicians. Sixty-one percent of participants reported at least a slight likelihood of leaving their healthcare institution within the next 2 years. Greater workplace belonging was found to be associated with a significant reduction in the reported likelihood of leaving their institution after accounting for the number of years having worked in their current institution, underrepresented minority status, and the interaction between the latter two covariates. The workplace belonging factor found to be most important in predicting the likelihood of leaving was the belief that there was an opportunity to thrive professionally in the institution. Belonging factors involving feeling able to freely share thoughts and opinions were also found to be of relatively high importance in predicting the likelihood of leaving. Greater workplace belonging was found to relate significantly to a reduced likelihood of leaving their institution within the next 2 years. Our findings suggest that leaders of healthcare organizations might reduce attrition of women by fostering workplace belonging with particular attention to empowering professional thriving and creating a culture that values open communication.

Disadvantaged Students Utilize School Campus and Its Resources More Than Non-disadvantaged Students.

Introduction Disadvantaged populations were disproportionately affected by the COVID-19 pandemic, both in the medical and educational settings. Lower-income families often do not have a laptop/desktop computer, adequate internet connection, or a dedicated study space. This unfortunately contributed to poorer academic performance during distance learning. To combat this, the Kirk Kerkorian School of Medicine (KKSOM) did not close down campus during the pandemic. This study analyzes the utilization of campus and live Zoom lectures by KKSOM students and its impact on educational outcomes. Methods We sent an Institutional Review Board (IRB)-approved survey to KKSOM students and asked about study locations, Zoom lecture attendance, and relationship quality during the pandemic. The class of 2024 had a unique experience as they were first-year students during the start of the COVID-19 pandemic and the transition to online learning. However, they always had access to campus and technological resources. We compared the survey scores from a Qualtrics electronic survey and the National Board of Medical Examiners (NBME) scores of students who self-indicated disadvantaged status, first-generation, underrepresented minority, and lower socioeconomic status to those who did not meet these criteria.Data analysis was done using SPSS software version 28.0.1.1 (IBM Corp., Armonk, NY). Results First-generation students studied on campus more frequently than their counterparts (31% versus 20%, p < 0.05) and less at home in general (55.4% versus 67.5%, p < 0.05). Lower socioeconomic status (SES) students attended live Zoom lectures more often as well (56.6% versus 43.1%, p < 0.05). Lastly, no significant differences were found between disadvantaged and non-disadvantaged groups for the class of 2024 in the NBME exam scores or relationship quality scores. Conclusion Our results suggest that students from disadvantaged backgrounds spend more time studying on campus than at home. Additionally, during the COVID-19 pandemic, they attended live Zoom lectures more often than their non-disadvantaged counterparts. Access to campus was not restricted for KKSOM studentsduring the pandemic. This may be one explanation for the lack of disparity between disadvantaged and non-disadvantaged students with regard to academic performance and relationship quality. This makes a strong argument for the importance of campus accessibility for the success of students, especially those from disadvantaged backgrounds.

Hearing Impairment and School Engagement Outcomes in US Children

Ensuring appropriate school engagement for deaf or hard of hearing (DHH) children in the US is important for improving their long-term outcomes as they grow into adults. To examine the associations between hearing loss (HL), its sequelae (speech and/or language disorders, behavioral and/or conduct problems, and neuropsychiatric and/or learning disorders), and various school engagement measures among school-aged children in the US. This cross-sectional study combined data from the nationally representative 2016-2021 National Survey of Children's Health. A total of 155 178 randomly selected children (weighted, approximately 49 340 700 children) aged 6 to 17 years with a parent or caregiver who responded to an address-based survey by mail or online were included. All analyses were weighted to account for the probability of selection and nonresponse and to reflect population-based estimates representative of all noninstitutionalized school-aged US children and adolescents residing in housing units. Diverse school engagement measures, including extracurricular participation in sports, clubs, paid work, volunteer work, and organized lessons, as well as educational performance variables, including missed school days, not caring about doing well in school, not doing required homework, grade repetition, and parent or guardian contacted by school. Of the estimated 49 340 700 children aged 6 to 17 years (41.1% aged 6-10 years; 51.1% male; 54.7% without underrepresented minority status), an estimated 1.4% (95% CI, 1.2%-1.5%) were reported by their parent or guardian to have deafness or hearing problems. Having DHH status was associated with significantly worse outcomes on 8 of 10 school engagement measures (eg, participation in sports: adjusted odds ratio [AOR], 0.75 [95% CI, 0.60-0.93]; missed school days: AOR, 2.98 [95% CI, 2.21-4.00]), even after adjustment for age, sex, underrepresented minority status, highest educational level of parent or guardian, federal poverty level of the household, and primary language in the household. Moreover, although subgroup analyses of DHH children with and without HL sequelae revealed significant differences (speech and/or language disorder: AOR, 5.83 [95% CI, 4.31-7.89]; behavioral and/or conduct problem: AOR, 2.75 [95% CI, 2.10-3.60]; neuropsychiatric and/or learning disorder: AOR, 3.06 [95% CI, 2.39-3.91]), HL sequelae only partially mediated the associations between these disparities. In this cross-sectional study, DHH status itself may have been the primary factor directly associated with school engagement disparities. These findings suggest the need for greater emphasis on educational accommodations and support for hearing status itself, independent of the presence or absence of HL sequelae.

Physician assistant education program director attrition and consideration to leave

BackgroundThis study explores factors related to physician assistant (PA) education program directors’ (PD) consideration to leave their leadership role. This is important to better understand, with the need for additional PA education PDs as the number of PA programs grows in addition to current PA program leaders considering leaving their PD role.MethodsData from the 2019 Physician Assistant Education Association (PAEA) Faculty and Directors Survey were used to analyze factors related to consideration for leaving the PD position. Multiple logistic regression analyses were utilized to identify predictors of PD consideration for leaving their position. Multiple regression analyses were also used to explore factors related to burnout.ResultsThe study found burnout was a modest predictor for consideration of leaving the PD position, while underrepresented minority status was not. Additional job stress, job satisfaction, and job experience variables were found to have a modest relationship with consideration of leaving, with odds ratios between 0.28 (lack of faculty respecting each other) and 5.29 (stress from lack of personal time) for those with statistically significant relationships.ConclusionsPD consideration of leaving is a complex phenomenon with many variables and confounding factors likely at play, including, as demonstrated by this study, level of burnout. Study implications include a further understanding of how effective strategies might be designed and implemented to address the drivers of PA PD attrition. Further exploration of burnout as a possible mediating variable as well as more specific data collection directed at better understanding predictors of PD attrition would be valuable future research directions.

Match Rates Among Underrepresented Minority and Female Applicants to Orthopaedic Surgery Residency Programs from 2011 to 2021: How Are We Doing?

Orthopaedic surgery residency programs have traditionally had less representation of underrepresented minority (URM) and female trainees compared with other medical specialties. Widespread efforts have been implemented to increase the diversity of orthopaedic surgery residency programs; however, it is not known whether URM and female applicants are increasingly likely to match as a result. Thus, we aimed to study the independent association between URM and female applicants and matching into orthopaedic surgery over the past decade. Applicant-level data from the Electronic Residency Application Service were reviewed from 2011 to 2021 with variables including demographic variables, URM status, and matriculation to an orthopaedic surgery residency program. Multivariate logistic regression was used to identify the likelihood of matriculating into orthopaedic surgery when controlling for number of applications, top 40 medical school status, AOA status, and MD/other degree. Twelve thousand one hundred eleven applicants were identified from 2011 to 2021 with a match rate of 70% overall. Two thousand fifty-six applicants (17%) were female and 1,926 (16%) classified as URM. The total number of applications increased from 1,074 in 2011 to 1,229 in 2021. The adjusted odds ratio (OR) associated with matching among all applicants decreased from 0.75 in 2011 to 0.64 in 2021, p < 0.001, and the OR of non-URM male and female applicants also decreased (female: 0.79-0.69, p < 0.001; male: 0.78-0.65, p < 0.001). The OR of URM male applicants did not change significantly (0.57-0.55, p = 0.60). The OR for URM female applicants, however, increased significantly from 0.46 to 0.61, p < 0.001. Over the entire time frame, the odds of matching were significantly lower for URM applicants compared with non-URM applicants (both male and female). Overall, the adjusted odds ratio of matching into orthopaedic surgery among female URM applicants has increased over the past decade, indicating successful efforts to improve the diversity of orthopaedic surgery training programs. The odds of URM male applicants have remained relatively constant, and the odds of URM male and female applicants were significantly lower than all non-URM applicants. III.

Quantifying grade benefits of study: Practice makes better

Problem Roulette (PR), an online study service at the University of Michigan, offers points-free formative practice to students preparing for examinations in introductory science, technology, engineering, and mathematics courses. Using four years of PR data involving millions of problem attempts by thousands of students, we quantify the benefits of increased practice study volume in introductory physics. After conditioning the mean final grade on standardized (ACT and SAT) math test score, we analyze deviations based on student study volume. We find a strong effect; mean course grade rises quadratically with the logarithm of the total number of PR questions encountered over the term (NQ,tot), with an overall gain of 0.77±0.12 grade points between 1<NQ,tot<1000. The gains are persistent across the range of math test scores represented in our sample. While NQ,tot surely correlates with other study habits, the benefits of increased study in general still hold. A model for final grade using test score and study volume largely accounts for demographic stratification, including by sex, parental education level, number of parents at home, nationality or underrepresented minority status, and regional income level, with two significant exceptions: students whose parents did not earn a college degree, who earn −0.27±0.04 grade points (6.1σ) below expectations, and underrepresented minority students at −0.14±0.04 points (3.6σ). Residual scatter in final grade remains comparable to the maximal study gains, implying that the model is far from deterministic: individual variation trumps mean trends. Our findings can help motivate students to study more and help teachers to identify which types of students may especially need such encouragement.1 MoreReceived 20 December 2022Accepted 25 April 2023DOI:https://doi.org/10.1103/PhysRevPhysEducRes.19.010139Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasInstructional materials developmentLearning environmentLearning theoryScientific reasoning & problem solvingStudent preparationTechnologyPhysics Education Research

Factors associated with emergency physician income.

Income fairness is important, but there are limited data that describe income equity among emergency physicians. Understanding the magnitude of and factors associated with income differences may be helpful in eliminating disparities. This study analyzed the associations of demographic factors, training, practice setting, and board certification with emergency physician income. We distributed a survey to professional members of the American College of Emergency Physicians. The survey included questions on annual income, educational background, practice characteristics, gender, age, race, ethnicity, international medical graduate status, type of medical degree (MD vs DO), completion of a subspecialty fellowship, job characteristics, and board certification. Respondents also reported annual income. We used linear regression to determine the respondent characteristics associated with reported annual income. From 45,961 members we received 3407 responses (7.4%); 2350 contained complete data for regression analysis. The mean reported annual income was $315,306 (95% confidence interval [CI], $310,649 to $319,964). The mean age of the respondents was 47.4 years, 37.4% were women, 3.2% were races underrepresented in medicine (Black, American Indian, or Alaskan Native), and 4.8% were Hispanic or Latino. On linear regression, female gender was associated with lower reported annual income; difference -$43,565, 95% CI, -$52,217 to -$34,913. Physician age, degree (MD vs DO), underrepresented racial minority status, and underrepresented ethnic minority status were not associated with annual income. Fellowship training was associated with lower income; Accreditation Council for Graduate Medical Education (ACGME) program difference -$30,048; 95% CI, -$48,183 to -$11,912, non-ACGME-program difference -$27,640, 95% CI, -$40,970 to -$14,257. Working at a for-profit institution was associated with higher income; difference $12,290, 95% CI, $3693 to $20,888. Board certification was associated with higher income; difference, $43,267, 95% CI, $30,767 to $55,767. This study identified income disparities associated with gender, practice setting, fellowship completion, and American Board of Emergency Medicine or American Osteopathic Board of Emergency Medicine certification.

Influence of participant and reviewer characteristics in application scores for a hematology research training program

AbstractThe American Society of Hematology Clinical Research Training Institute (CRTI) is a clinical research training program with a competitive application process. The objectives were to compare application scores based on applicant and reviewer sex and underrepresented minority (URM) status. We included applications to CRTI from 2003 to 2019. The application scores were transformed into a scale from 0 to 100 (100 was the strongest). The factors considered were applicant and reviewer sex and URM status. We evaluated whether there was an interaction between the characteristics and time related to application scores. In total, 713 applicants and 2106 reviews were included. There was no significant difference in scores according to applicant sex. URM applicants had significantly worse scores than non-URM applicants (mean [standard error] 67.9 [1.56] vs 71.4 [0.63]; P = .0355). There were significant interactions between reviewer sex and time (P = .0030) and reviewer URM status and time (P = .0424); thus, results were stratified by time. For the 2 earlier time periods, male reviewers gave significantly worse scores than did female reviewers; this difference did not persist for the most recent time period. The URM reviewers did not give significantly different scores across time periods. URM applicants received significantly lower scores than non-URM applicants. The impact of reviewer sex and URM status changed over time. Although male reviewers gave lower scores in the early periods, this effect did not persist in the late period. Efforts are required to mitigate the impact of applicant URM status on application scores.

Year two of virtual interviews: longitudinal changes and diverse perspectives

BackgroundThe coronavirus disease (COVID-19) pandemic brought the virtual interview (VI) format to graduate medical education (GME) and the trainee recruitment process. It is unclear if applicants’ VI experience is consistent across all demographic groups. Our group collected 2 years of survey data to assess longitudinal changes in applicants’ attitudes towards the VI format. In addition, demographic data were collected, and analyses were performed to identify if between-group differences were present amongst a diverse applicant population.MethodsWe distributed an anonymous electronic survey to applicants to the pulmonary disease and critical care medicine fellowship programs at Case Western Reserve University/University Hospitals Cleveland Medical Center and MetroHealth Medical Center for the 2021 and 2022 appointment years.ResultsWe received 112 responses (20% response rate) for our surveys. Although there was an overall stability of responses between the first 2 years, there were significant gender differences with applicants identifying as female more likely to recommend VI as a future model. Similarly, there were a significant difference in factor importance based on underrepresented minority (URM) status with applicants identifying as URM placing more emphasis on programs’ social media presence.ConclusionsThere were no significant change in the responses of applicants between the first 2 years of VI. However, subset analyses revealed multiple significant findings. These differences have implications for future iterations of the VI format.

Analyses of Orthopaedic Surgery Residency Interviews.

We retrospectively reviewed the 2020-2021 and 2021-2022 residency interview cycles, both conducted virtually due to the COVID-19 pandemic. Residency application (i.e., applicant demographic and academic backgrounds) and interview data (i.e., faculty interviewer scores) were recorded. Interobserver reliability among faculty interviewers was calculated. Statistical analysis was performed to determine factors associated with ranking of applicants. There were 195 included applicants from the 2020 and 2021 interview cycles. There was no true agreement of interviewers' scoring of shared applicants (kappa intraclass coefficient range 0-0.2). Applicant factors associated with being ranked include applying to the match for the first time, USMLE Step 1 and 2 scores, educational break (vs. consecutive completion of college and medical school in 4 years each), higher class rank, and greater interviewer scores. Factors associated with better rank included additional degrees (i.e., PhD or MBA), couples match, AOA designation, educational break, underrepresented minority status, and notable attributes (i.e., collegiate athletics or Eagle Scout participation). Factors associated with worse rank included male sex, international medical graduate, prior match history, science major, extended research (i.e., >1 year spent in a research role), and home medical school students. There was significant variability and no reliability at our institution among faculty interviewers' applicant ratings. Being ranked was based more on academic record and interview performance while final rank number seemed based on applicant qualities. The removal of merit-based objective applicant measurements offers challenges to optimal residency applicant and program match. III (retrospective cohort study).

Evaluation of Bias in Medical Student Clinical Clerkship Evaluations Using Natural Language Processing.

Purpose: There is increasing interest in understanding potential bias in medical education. Prior studies have demonstrated potential disparities in the words used to describe medical students in their evaluations, such as variations by gender/race. 1,2 For example, natural language processing (NLP) has been used to determine narrative differences between medical clerkship evaluations on the basis of gender and under-represented minority status. 2 Females were more commonly described using words for personal attributes, whereas males were more likely to be described using words related to professional competencies. 1 In this study, we used NLP to evaluate potential bias among third-year clinical clerkship evaluations. Method: This study was conducted at the University of California San Diego School of Medicine. Data was extracted from medical evaluation and administrative databases for medical students enrolled in third-year clinical clerkship rotations across 2 academic years (2019–2020 and 2020–2021). For each evaluation, we collected information regarding demographics of both the student and the faculty evaluator. These were used to determine gender and racial concordance (i.e., whether the student and faculty identified in the same group). We extracted the full narrative text of each evaluation, as well as numerical evaluation scores. Narrative text was processed using a Python NLP package, which assigned a sentiment score for each evaluation. Word clouds were generated for various demographic groups to exhibit the most commonly used words. We analyzed the distributions of sentiment scores and clerkship grades to determine any differences by gender or race/ethnicity. We used multinomial logistic regression to model final clerkship grades, using predictors such as numerical evaluation scores, gender and racial concordance between faculty and students, and sentiment scores. Statistical analyses were performed using R. Statistical significance was defined as P < .05. Results: We analyzed 963 evaluations from 198 students, with 109 (55%) females. Ninety-two (47%) identified as Caucasian, 77 (39%) Asian, 13 (7%) African American, and 16 (9%) other/unspecified or declined to state race. Ten (5%) identified as Hispanic. Females (190/534, 34%) were more likely to receive honors than males (113/420, 26%) (P = .02). Significantly more students received honors or near-honors grades in the 2020–2021 school year in comparison with the 2019–2020 school year (P < .00001). Sentiment scores for evaluations did not vary significantly by student gender, race, or ethnicity (P = .88, .64, and .06, respectively). Word choices were similar across faculty and student demographic groups. Similarly, in the multinomial logistic regression model of final clerkship grades, the narrative evaluation sentiment score was not predictive of an honors grade (odds ratio [OR] 1.23, P = .58). However, the numerical evaluation average (OR 1.45, P < .001) and gender concordance between faculty and student (OR 1.32, P = .049) were significant predictors of receiving honors. Discussion: There was no clear evidence of bias in medical student evaluations. No differences were found in sentiment scores or in word choices by gender or race/ethnicity. Of note, sentiment scores from narrative evaluations were not significantly associated with final grades, while numerical evaluation scores were. Narrative feedback tended to be positive for all students regardless of final grade, which may not be constructive to students for understanding how to improve. Significance: The lack of disparities in our study contrasts prior findings from other institutions. Ongoing efforts include comparative analyses with other institutions to understand what institutional factors (e.g., geographic location) may contribute to bias. NLP enables a systematic approach for investigating bias. The insights gained from the lack of association between word choices, sentiment scores, and final grades show potential opportunities to improve feedback processes for students.

A novel algorithm to reduce bias and improve the quality and diversity of residency interviewees.

ObjectiveImprove the quality and diversity of candidates invited for the Otolaryngology‐Head and Neck Surgery residency match by reducing geographical and inter‐rater bias with a novel geographic distribution algorithm.MethodsInterview applicants were divided into geographic regions and assigned to reviewers. Each reviewer selected by force‐ranking a pre‐determined number of applicants to invite for interviews based on the percentage of applications received for each region. Our novel geographic distribution algorithm was then applied to maintain the geographic representation and underrepresented minority status of invited applicants to match the applicant pool.ResultsAnalysis of previous interview selection methods demonstrated a statistically significant overrepresentation of local applicants invited for interviews. In 2022, 324 domestic applications were received for the otolaryngology match, which were divided into six geographic regions. There was no significant difference in USMLE scores between regions. The implementation of our distribution algorithm during applicant selection eliminated local overrepresentation in the invited pool of applicants and maintained the representation of underrepresented minority applicants. Following the match, reviewers indicated that implementation of the geographic distribution algorithm was simple and improved the quality and diversity of the group of interviewed applicants.ConclusionTraditional methods of scoring and inviting otolaryngology residency applicants can be confounded by regional and inter‐rater biases. Employing a geographic distribution algorithm improves the quality and diversity of invited applicants, eliminates bias, and maintains the representation of underrepresented minority applicants.