Clerkship Grades Research Articles

Validity and reliability of Brier scoring for assessment of probabilistic diagnostic reasoning.

Educators need tools for the assessment of clinical reasoning that reflect the ambiguity of real-world practice and measure learners' ability to determine diagnostic likelihood. In this study, the authors describe the use of the Brier score to assess and provide feedback on the quality of probabilistic diagnostic reasoning. The authors describe a novel format called Diagnostic Forecasting (DxF), in which participants read a brief clinical case and assign a probability to each item on a differential diagnosis, order tests and select a final diagnosis. DxF was piloted in a cohort of senior medical students. DxF evaluated students' answers with Brier scores, which compare probabilistic forecasts with case outcomes. The validity of Brier scores in DxF was assessed by comparison to subsequent decision-making in the game environment of DxF, as well as external criteria including medical knowledge tests and performance on clinical rotations. Brier scores were statistically significantly correlated with diagnostic accuracy (95 % CI-4.4 to-0.44) and with mean scores on the National Board of Medical Examiners (NBME) shelf exams (95 % CI-474.6 to-225.1). Brier scores did not correlate with clerkship grades or performance on a structured clinical skills exam. Reliability as measured by within-student correlation was low. Brier scoring showed evidence for validity as a measurement of medical knowledge and predictor of clinical decision-making. Further work must evaluated the ability of Brier scores to predict clinical and workplace-based outcomes, and develop reliable approaches to measuring probabilistic reasoning.

Evaluating the 2023 Standardized Letter of Recommendation Form in Applications to Orthopaedic Surgery Residency.

Successfully matching into orthopaedic surgery residency is a challenging endeavor due to the limited number of available positions. Implementation of the Standard Letter of Recommendation (SLOR) in 2017 was intended to allow better comparisons between applicants; however, the form suffered from notable rank inflation ultimately requiring introduction of an updated SLOR for the 2023 to 2024 application cycle. The purpose of this study was to evaluate whether the 2023 SLOR improves on deficiencies of the 2017 SLOR. Applications submitted to our institution during the 2023 to 2024 orthopaedic surgery residency application cycle were analyzed. Applicant objective metrics were collected, including US Medical Licensing Exam scores, clerkship grades, AΩA status, and research productivity. Scores for each domain of the 2023 SLOR forms were recorded. Mean scores were calculated for each question, and cumulative scores per SLOR were determined. Intraclass correlation coefficients were determined per applicant and evaluator. The Spearman rank correlation coefficient was calculated for comparison of SLOR cumulative scores and objective metrics. Three hundred sixty-eight applications were reviewed, and 743 2023 SLORs were submitted in conjunction with these applications. Most of the 2023 SLORs (n = 399, 53.70%) had cumulative scores placing applicants in at least the 90th percentile. Furthermore, 267 evaluators (72.36%) consistently gave scores of 99s and 10s for each question. Intraclass correlation coefficients were rated as fair to moderate agreement between evaluators for each applicant. Objective applicant metrics did not meaningfully correlate with SLOR cumulative scores. The 2023 SLOR demonstrated notable rank inflation with only 22.7% of applicants receiving a score of less than 7 on any domain. By contrast, 69.9% of applicants received a score of "rare, perfect gem" on at least one domain of the form. We encourage the American Orthopaedic Association Council of Orthopaedic Residency Director to continue to improve the form and provide guidance to residency program leadership on proper use of the grading system.

Imprecise Clinical Assessments and Inaccurate Grades: Family Medicine Clerkship Director Perspectives.

As application to residency programs becomes increasingly competitive, educational leaders face growing student concern about imprecise clinical assessments and clerkship grades. As part of a large annual survey of family medicine clerkship directors (FMCDs), 10 questions were disseminated in May 2023 about perceived levels of imprecise assessments by faculty. We aimed to determine to what extent respondents felt their institution's evaluation system propagated inaccurate grading. A total of 52% of 169 FMCDs responded to the survey. Of these, 7% of respondents were completely confident that their preceptors would give two students of identical competence the same clinical evaluation rating. FMCDs estimated that an average of 38% of their preceptors inaccurately rate student performance. Most clerkships use an Honors/High Pass/Pass/Fail grading system. We found that 51% of FMCDs prefer to use a different grading paradigm than they currently use. We asked FMCDs to estimate the percentage of students that expressed concern over inaccurate preceptor ratings. Grading systems with more tiers were associated with a higher percentage of concerned students. Clerkship grades are widely used by residency program directors to classify and differentiate student applicants. We identified a significant concern from FMCDs that clinical evaluation ratings can vary greatly. Given the high stakes and perceived inaccuracy of clerkship grading, we recommend continued investigation into the appropriate weighing and usage of clinical evaluations. Continued exploration is recommended to develop grading paradigms centered on criterion-based assessment.

Snapshot of Examination Usage in Emergency Medicine Clerkships.

Objective Emergency Medicine (EM) clerkships often use a written exam to assess the knowledge gained over the course of an EM rotation in medical school. Clerkship Directors (CDs) may choose the National Board of Medical Examiners (NBME) EM Advanced Clinical Science Subject Exam (ACE), the Society for Academic Emergency Medicine (SAEM) M4 exam, which has two versions, the SAEM M3 exam, or departmental exams. There are currently no published guidelines or consensus regarding their utility. This survey-based study was designed to collect data regarding current practices of EM clerkship exam usage to analyze trends and variability in what exams are used and how. Methods The authors designed a cross-sectional observational survey to collect data from EM CDs on exam utilization in clerkships. The survey population consisted of clerkship directors, assistant clerkship directors, or faculty familiar assessments in their EM clerkship. Initial dissemination was by electronic distribution to subscribers of the Clerkship Directors in Emergency Medicine (CDEM) list-serve on the SAEM website. Subsequently, contact information of CD's from institutions that had not responded was obtained by manual search of the Emergency Medicine Residents' Association (EMRA) Match website and individual correspondence was sent at regular intervals. Data obtained include clerkship characteristics, exam used, weight of the exam relative to the overall grade, and alternatives if the preferred exam was previously taken. Results Eighty-seven programs (42% response rate) completed the survey between August 2019 and February 2021. Of the 87 responses, 71 (82%) were completed by a CD. Forty-six (53%) institutions required an EM rotation. Students were tested in 34 (74%) required EM clerkships and 48 (69%) out of 70 EM electives. In required rotations that used an exam, 20 (59%) used the NBME EM ACE, while 28 of 46 (61%) of EM electives that reported an exam used the SAEM M4 Exam. Five (15%) of the required clerkships used a departmental exam. Of clerkships requiring an exam, 46 (57%) weighed the score at 11-30% of the final grade. Data for extramural rotations mirrored that of EM electives. One-third of respondents indicated they do not inquire about previously taken exams. Conclusion This survey demonstrates significant variability in the type of exam, the weighting of the score, and alternatives if the preferred exam was previously taken. The lack of a consistent approach in how these exams are used in determining students' final EM grades diminishes the reliability of the EM clerkship grade as a factor used by residency directors in choosing future residents. Further research on optimal usage of these exams is needed.

Structures and Processes of Grading Committees in Internal Medicine Clerkships: Results of a National Survey.

Clerkship grades are important in the residency selection process but can be influenced by individual bias and grading tendencies. Although clinical competency committees are standard in graduate medical education, in undergraduate medical education, they have not gained widespread traction. This study describes structures and processes of grading committees in internal medicine (IM) clerkships and strategies used to mitigate grading bias. From September to December 2022, the Clerkship Directors in Internal Medicine conducted its annual survey of IM core clerkship directors at 140 U.S. and U.S. territory-based medical schools. This study was based on 23 questions about grading committees in IM clerkships. The survey response rate was 80% (n = 112/140). Forty-seven respondents (42%) reported using grading committees in their IM clerkship (median committee size, 7; range, 3-20) (primarily clerkship leadership and faculty). Responsibilities included determining grades for all students (31 [66%]) and students at borderline of failing (30 [64%]), adjudicating on students with professionalism concerns (25 [53%]), and reconciling discordant clinical evaluations (24 [51%]). To mitigate deliberation bias, committees most frequently used multisource assessments (38 [81%]) and adoption of a shared mental model (36 [77%]). Approximately one-third of grading committees "rarely" discussed gender (14 [30%]) and race or ethnicity (15 [32%]), and 7 committees (15%) "never" discussed gender and race or ethnicity. Clerkship directors perceived developing a shared mental model (60 [92%]), promoting consistency (59 [91%]) and transparency (57 [88%]) in the process, mitigating assessment bias (58 [89%]), improving student satisfaction (54 [83%]), and sharing grading responsibility (44 [68%]) as potential benefits. This study found that grading committees in IM clerkships are not widely used and that existing committees vary in structure and process. These findings highlight an opportunity for medical schools to consider using grading committees to improve grade assignment and address grading inconsistencies.

Innovations report: A grading committee for a required fourth-year emergency medicine clerkship.

Assessment of medical students in the clinical learning environment is fraught with challenges. Seemingly small variations in clinical clerkship evaluation can significantly impact a student's future. As such, the integrity of the grade selection process must be heavily scrutinized. Group decision making in the form of a clerkship grading committee may be part of a solution to address this complex problem. We conducted a retrospective observational study to describe grading committee decisions for a required fourth-year EM clerkship from August 2021 to April 2022. Literature on best practices for group decision making and assessment were reviewed. This informed the development and implementation of the committee process. Each committee meeting was video recorded and coded for discussion time per student, times the committee grade differed from historical-grade cutoffs with reasoning, and the frequency a committee member voiced a first-hand account of student performance. Data from nine meetings was reviewed (86 students). The mean discussion time per student was 2 min 13 s (range 11 s to 9 min 22 s). The final committee decision differed from historical-grade cutoffs for nine students (10%), six were adjusted above and three below. In 64% (55/86) of student reviews a committee member voiced a first-hand experience of working with the student. Positive grade adjustments were made due to outlier evaluations and negative adjustments were made for professionalism concerns. Grading committees are a means to conduct a comprehensive review of student performance and offer shared ownership of the grade decision among committee members. More study is needed to directly determine their potential benefit and role in clerkship grading.

How Prominent Are Gender Bias, Racial Bias, and Score Inflation in Orthopaedic Surgery Residency Recommendation Letters? A Systematic Review.

Letters of recommendation are considered an important factor in the holistic ranking of orthopaedic residency applications. The standardized letter of recommendation (SLOR) was introduced in 2017 in response to the inherent bias and limited comparative value of traditional LORs. It includes standardized questions about an applicant's orthopaedic qualifications, designed to enable fair comparisons among candidates. However, disparate and inconsistent findings have made it difficult to draw meaningful conclusions from individual studies on the SLOR and narrative letter of recommendation. In this systematic review we asked: (1) What is the distribution of applicant SLOR rating among nine domains and summative scores? (2) Are there applicant characteristics associated with SLOR ratings? (3) Is there gender and racial bias in the SLOR and narrative letter of recommendation? PubMed, EBSCO, and Google Scholar electronic databases were queried on March 20, 2023, to identify all studies evaluating SLOR and narrative letter of recommendations between January 1, 2017, and March 20, 2023. Articles that evaluated orthopaedic SLOR or narrative LORs were included. Systematic reviews, case reports, duplicate studies between databases, and grey literature such as abstracts and articles on preprint servers were excluded. Eight studies including 6179 applicants and 18,987 letters of recommendation were included in the final analysis. The applicant classes ranged from years 2014 to 2020. Two reviewers independently evaluated the quality of the included studies using the Joanna Briggs Institute (JBI) tool for cross-sectional studies. The mean JBI score of included studies was 7.4 out of a maximum of 8, with higher scores indicating better quality. The primary outcome was to determine the distribution of applicant SLOR rankings in their summative score. Summative scores were separated into ranked to match, top one-third, middle one-third, lower one-third, and not a fit. Four studies reported the summative SLOR scores of applicants. Our secondary study goal was to assess applicant characteristics associated with SLOR summative scores and assess any bias present in the SLOR or narrative recommendation. Five studies compared SLOR ratings across applicant characteristics including gender, race, USMLE Step 1 score, USMLE Step 2 score, Alpha Omega Alpha (AΩA) membership, clerkship grades, and publications. Gender and racial bias were also assessed across five studies. Four studies utilized a linguistic analysis software to search letters of recommendation for differences in word category use by gender and race. Studies consistently found that a higher percentage of candidates were identified in the top percentile blocks than is mathematically possible. For example, the two studies that tallied the proportion of candidates ranking in the top 10% of applicants found that 36% and 42% were rated as being in the top 10%. Similarly, articles found more than 87% of applicants scored in the top one-third. Studies had mixed results, but in general, they suggested that AΩA status, higher Step 1 scores, and more research publications have a slightly positive association with increased SLOR scores. We found no evidence of gender bias against women, and in fact, most studies evaluating word choices found word choices for women candidates generally were stronger. Similarly, no consistent disparities were identified in word categories utilized in SLORs based on applicant race. Most notably, a mathematically impossible proportion of applicants were counted in the top percentiles in letters of recommendation. This compromises readers' abilities to differentiate candidates. Factors like AOA status and research publications displayed a modest positive correlation with SLOR scores. Gender bias against women or candidates from racial minority groups was not evident. We suggest that group SLOR authorship, with a consensus assessment by clerkship or residency directors, should be considered. We also recommend the incorporation of mean and SD scores for each letter writer (or group) alongside their letters. Furthermore, studies indicate that submitting all four SLOR letters can substantially improve an applicant's chances of success. We advise students to choose strategically and submit the maximum allowable number of SLORs.

R.I.M.E. and reason: multi-station OSCE enhancement to neutralize grade inflation

ABSTRACT To offset grade inflation, many clerkships combine faculty evaluations with objective assessments including the Medical Examiners Subject Examination (NBME-SE) or Objective Structured Clinical Examination (OSCE), however, standardized methods are not established. Following a curriculum transition removing faculty clinical evaluations from summative grading, final clerkship designations of fail (F), pass (P), and pass-with-distinction (PD) were determined by combined NBME-SE and OSCE performance, with overall PD for the clerkship requiring meeting this threshold in both. At the time, 90% of students achieved PD on the Internal Medicine (IM) OSCE resulting in overall clerkship grades primarily determined by the NBME-SE. The clerkship sought to enhance the OSCE to provide a more thorough objective clinical skills assessment, offset grade inflation, and reduce the NBME-SE primary determination of the final clerkship grade. The single-station 43-point OSCE was enhanced to a three-station 75-point OSCE using the Reporter-Interpreter-Manager-Educator (RIME) framework to align patient encounters with targeted assessments of progressive skills and competencies related to the clerkship rotation. Student performances were evaluated pre- and post-OSCE enhancement. Student surveys provided feedback about the clinical realism of the OSCE and the difficulty. Pre-intervention OSCE scores were more tightly clustered (SD = 5.65%) around a high average performance with scores being highly negatively skewed. Post-intervention OSCE scores were more dispersed (SD = 6.88%) around a lower average with scores being far less skewed resulting in an approximately normal distribution. This lowered the total number of students achieving PD on the OSCE and PD in the clerkship, thus reducing the relative weight of the NMBE-SE in the overall clerkship grade. Student response was positive, indicating the examination was fair and reflective of their clinical experiences. Through structured development, OSCE assessment can provide a realistic and objective measurement of clinical performance as part of the summative evaluation of students.

Analysis of variability and trends in medical school clerkship grades

BackgroundMedical school clerkship grades are used to evaluate orthopedic surgery residency applicants, however, high interinstitutional variability in grade distribution calls into question the utility of clerkship grades when evaluating applicants from different medical schools. This study aims to evaluate the variability in grade distribution among medical schools and look for trends in grade distribution over recent years. MethodsApplications submitted to Rush University's orthopedic surgery residency program from 2015, 2019, and 2022 were collected from the Electronic Residency Application Service. Applications from the top 100 schools according to the 2023–2024 U.S. News and World Report Research Rankings were reviewed. The percentage of “honors” grades awarded by medical schools for the surgery and internal medicine clerkships were extracted from applicants' Medical Student Performance Evaluation letters. ResultsThe median percentage of honors given in 2022 was 36.0 % (range 10.0–82.0) for the surgery clerkship and 33.0 % (range 6.7–80.0) for the internal medicine clerkship. Honors were given 6.6 % more in the surgery clerkship in 2022 compared to 2015. There was a negative correlation between a higher (worse) U.S. News and World Report research ranking and the percentage of honors awarded in 2022 for the surgery and internal medicine clerkships. ConclusionThere is substantial interinstitutional variability in the rate that medical schools award an “honors” grade with evidence of grade inflation in the surgery clerkship. Residency programs using clerkship grades to compare applicants should do so cautiously provided the variability demonstrated in this study.

Program Directors' Perspectives on Residency Applications in the Post‒United States Medical Licensing Examination Step 1 Era: A Case for Standardized Letters of Recommendation?

The United States Medical Licensing Examination (USMLE) Step 1 test evolved into a key metric utilized by program directors (PDs) in assessing candidates for residency. The transition to a USMLE Step 1 binary pass/fail scoring system has resulted in a loss of an important objective assessment. With national movements toward pass/fail systems for clerkship grading and trends toward abandonment of class ranking, assessing residency applications has become increasingly challenging. The Society of Academic Urologists convened a task force to, in part, assess the perspectives of urology PDs regarding the importance of various aspects of a residency application for predicting clinical performance. An anonymous survey was disseminated to all urology PDs in the US. Perspectives on 11 potential application predictors of clinical performance and demographics were recorded. Descriptive statistics characterized PD responses. Friedman test and pairwise Wilcoxon tests were used to evaluate the relative ranks assigned to application elements by PDs. There was a 60.5% response rate (89/147). Letters of recommendation (LORs) were ranked as the most important predictor, with a mean rank of 2.39, median of 2 (IQR 1-3). Clerkship grades and USMLE Step 1 were comparable and ranked second. Medical school reputation ranked the lowest. There was significant subjective heterogeneity among categories; however, this was less so for LORs, which predominated as the most important factor among application elements (P < .001). To our knowledge, this is the largest sample size assessing PD perspectives on application factors that predict clinical performance. The second (clerkship grades) and third (USLME Step 1) most important factors moving toward binary pass/fail systems create an opportunity for actionable change to improve assessment objectivity. Our data demonstrate LORs to be the most important factor of residency applications, making a compelling argument for moving toward a standardized LOR to maximize this tool, mitigate bias, and improve interreviewer reliability.

Evaluation of bias and gender/racial concordance based on sentiment analysis of narrative evaluations of clinical clerkships using natural language processing

There is increasing interest in understanding potential bias in medical education. We used natural language processing (NLP) to evaluate potential bias in clinical clerkship evaluations. Data from medical evaluations and administrative databases for medical students enrolled in third-year clinical clerkship rotations across two academic years. We collected demographic information of students and faculty evaluators to determine gender/racial concordance (i.e., whether the student and faculty identified with the same demographic). We used a multinomial log-linear model for final clerkship grades, using predictors such as numerical evaluation scores, gender/racial concordance, and sentiment scores of narrative evaluations using the SentimentIntensityAnalyzer tool in Python. 2037 evaluations from 198 students were analyzed. Statistical significance was defined as P < 0.05. Sentiment scores for evaluations did not vary significantly by student gender, race, or ethnicity (P = 0.88, 0.64, and 0.06, respectively). Word choices were similar across faculty and student demographic groups. Modeling showed narrative evaluation sentiment scores were not predictive of an honors grade (odds ratio [OR] 1.23, P = 0.58). Numerical evaluation average (OR 1.45, P < 0.001) and gender concordance between faculty and student (OR 1.32, P = 0.049) were significant predictors of receiving honors. The lack of disparities in narrative text in our study contrasts with prior findings from other institutions. Ongoing efforts include comparative analyses with other institutions to understand what institutional factors may contribute to bias. NLP enables a systematic approach for investigating bias. The insights gained from the lack of association between word choices, sentiment scores, and final grades show potential opportunities to improve feedback processes for students.

Disparities in Medical School Clerkship Grades Associated With Sex, Race, and Ethnicity: A Person-Centered Approach.

To determine whether students' self-reported race/ethnicity and sex were associated with grades earned in 7 core clerkships. A person-centered approach was used to group students based on observed clerkship grade patterns. Predictors of group membership and predictive bias by race/ethnicity and sex were investigated. Using data from 6 medical student cohorts at Johns Hopkins University School of Medicine (JHUSOM), latent class analysis was used to classify students based on clerkship grades. Multinomial logistic regression was employed to investigate if preclerkship measures and student demographic characteristics predicted clerkship performance-level groups. Marginal effects for United States Medical Licensing Exam (USMLE) Step 1 scores were obtained to assess the predictive validity of the test on group membership by race/ethnicity and sex. Predictive bias was examined by comparing multinomial logistic regression prediction errors across racial/ethnic groups. Three clerkship performance-level groups emerged from the data: low, middle, and high. Significant predictors of group membership were race/ethnicity, sex, and USMLE Step 1 scores. Black or African American students were more likely (odds ratio [OR] = 4.26) to be low performers than White students. Black or African American (OR = 0.08) and Asian students (OR = 0.41) were less likely to be high performers than White students. Female students (OR = 2.51) were more likely to be high performers than male students. Patterns of prediction errors observed across racial/ethnic groups showed predictive bias when using USMLE Step 1 scores to predict clerkship performance-level groups. Disparities in clerkship grades associated with race/ethnicity were found among JHUSOM students, which persisted after controlling for USMLE Step 1 scores, sex, and other preclerkship performance measures. Differential predictive validity of USMLE Step 1 exam scores and systematic error predictions by race/ethnicity show predictive bias when using USMLE Step 1 scores to predict clerkship performance across racial/ethnic groups.

Rapid increase in departmental scholarly activity independent of residents demonstrates reproducibility and success of intensive research initiative in neurosurgery department

Background: Neurological surgery remains one of the most competitive specialties with a match rate of <70%. Historically, medical student performance was gauged through the USMLE Step 1. However, with the recent exam score change, metrics such as recommendation letters, research, and clerkship grades carry increased importance. Research experiences vary greatly between institutions and medical students depend on faculty/resident mentorship in order to facilitate scholarly activity. We previously reported our 2-year intensive research initiative (IRI) in a neurosurgery program. Here we report successful implementation of the IRI in a disparate setting, a department devoid of residents, and demonstrate the IRI's reproducibility with non-resident learners.Materials & Methods: We compared retrospective data from 2007 to 2020 with the IRI's results during the 2-year study period (July 2020–July 2022).Results: The IRI resulted in a rapid exponential increase in publications, with medical student led peer-reviewed publications (PRPs) increasing 1000% and pre-residency fellow (PRF) PRPs increasing by 4900%. Learner involvement on PRPs pre-IRI was 31%, increasing to 72% post-IRI implementation.Conclusions: We present the IRI's success increasing academic productivity despite utilizing only non-resident learners. Students underrepresented in medicine and those at non-tier 1 institutions receive unequal research and clinical opportunities, therefore, prioritizing and providing sufficient opportunities/mentorship is crucial in their success in matching into competitive specialties. Our IRI allows for early faculty/resident student mentorship and gives students more flexibility as it allows medical students at varying stages to participate in research with no set time frame.

Evaluating Urology Program Directors' Perception on Resident Application Parameters Following the Transition of USMLE Step 1 to Pass/Fail

ObjectiveTo investigate how the shift of the United States Medical Licensing Examination (USMLE) Step 1 to a Pass/Fail (P/F) scoring system impacts the perceptions of Urology Program Directors (PDs) on evaluating urology residency applicants. Methods and MaterialsA cross-sectional survey was sent to 117 PDs, including questions about program characteristics, perceptions of shelf scores and medical school rank post-transition, beliefs about the predictive value of Step 1 and Step 2 Clinical Knowledge (CK) scores for board success and residency performance, and changes in applicant parameter ranking. Results45 PDs (38% response rate) participated. Notably, 49% favored releasing quantitative clerkship grades, and 71% valued medical school rank more. Opinions on Step 1 scores' correlation with board success were split (49% agreed), and 44% endorsed Step 2 CK scores' connection to board performance. As predictors of good residents, only 9% and 22% considered Step 1 and Step 2 CK scores, respectively, indicative. Clerkship grades and Urology rotation recommendation letters-maintained significance, while research experience gained importance. Step 2 CK scores' importance rose but didn’t match Step 1 scores' previous significance. ConclusionThe transition to Pass/Fail (P/F) for USMLE Step 1 adds intricacies to urology residency selection, exposing Program Directors' uncertainties regarding clerkship grades and the relevance of medical school rank. This research underscores the dynamic nature of urology residency admissions, emphasizing the increasing importance of research in evaluating applicants and a diminishing emphasis on volunteering and leadership.

Medical student perceptions of assessments of clinical reasoning in a general surgery clerkship

BackgroundComponents factoring into general surgery clerkship grades vary by institution, and while evaluators attempt to remain unbiased when evaluating medical student performance, subjectivity and implicit bias remain an issue. Our institution recently implemented a case-based structured oral examination to provide the general surgery clerkship director objective insight into students’ clinical reasoning skills. We hypothesized that medical students believe this exam, along with graded clinical documentation and the Observed Standardized Clinical Encounter (OSCE), are fair assessments and increase students’ awareness of their clinical reasoning skills.MethodsA survey was sent to third-year medical students in the classes of 2023 and 2024 at our institution who had completed their general surgery clerkship. Students rated five grading assessments (i.e., preceptor evaluations, the oral examination, clinical documentation, the OSCE, and the shelf exam) on fairness and the ability of the assessment to give them insight into their clinical reasoning on a five-point Likert scale 1–5 (with 1 = Strongly Agree, 5 = Strongly Disagree).ResultsOne hundred and ten of 162 (67.9%) students responded to the survey. The shelf examination was the most highly regarded assessment tool followed by the oral examination. Seventy-three percent agreed or strongly agreed that the oral exam was a fair assessment, and 80% agreed or strongly agreed that it gave them insight into their clinical reasoning skills. Alternatively, only 41.8% of students agreed or strongly agreed that preceptor evaluations were fair assessments and 42.7% agreed or strongly agreed that it gave them insight into their clinical reasoning.ConclusionsThird-year medical students on a general surgery clerkship favor the shelf examination and a case-based oral examination over other assessment tools regarding fairness and perception of their clinical reasoning. This type of examination can provide general surgery clerkship directors with additional objective data to assess medical students more fairly and improve students’ clinical reasoning.

Association between inpatient team continuity and clerkship student academic performance.

To determine the association between inpatient team continuity, defined as the maximum number of days the same student, resident, and attending worked together on the inpatient wards, and the academic performance of students in a pediatric block clerkship. We retrospectively identified students who rotated in the pediatric clerkship at a single institution from 2020 to 2022. We used multiple linear regression models to adjust for multiple confounders and used a one-way analysis of variance to compare adjusted outcomes across quartiles of inpatient team continuity. A total of 227 students were included in the analysis. Students' preceptor ratings increased by 0.04 on a scale of 0-4 (95% confidence interval [CI] 0.01-0.06; p = .001), and their final pediatric grade increased by 0.02 on a scale of 0-4 (95% CI 0.01-0.02; p < .001) with each 1-day increase in inpatient team continuity. There was no statistically significant association between team continuity and shelf exam scores or observed structured clinical examination scores. Preceptor ratings and final clerkship grades increased across quartiles of team continuity, with the greatest increase being between the second, 6-7 days of continuity, and third, 8-10 days of continuity, quartiles. Increased inpatient team continuity is associated with students receiving higher preceptor ratings and achieving a higher final pediatric clerkship grade. While the mechanisms driving these associations remain unknown, the results add to the literature base supporting the importance of preceptor continuity in undergraduate medical education.

Reimagining a pass/fail clinical core clerkship: a US residency program director survey and meta-analysis

Pass/fail (P/F) grading has emerged as an alternative to tiered clerkship grading. Systematically evaluating existing literature and surveying program directors (PD) perspectives on these consequential changes can guide educators in addressing inequalities in academia and students aiming to improve their residency applications.In our survey, a total of 1578 unique PD responses (63.1%) were obtained across 29 medical specialties. With the changes to United States Medical Licensure Examination (USMLE), responses showed increased importance of core clerkships with the implementation of Step 2CK cutoffs. PDs believed core clerkship performance was a reliable representation of an applicant’s preparedness for residency, particularly in Accreditation Council for Graduate Medical Education’s (ACGME)Medical Knowledge and Patient Care and Procedural Skills. PDs disagreed with P/F core clerkships because it more difficult to objectively compare applicants. No statistically significant differences in responses were found in PD preferential selection when comparing applicants from tiered and P/F core clerkship grading systems. If core clerkships adopted P/F scoring, PDs would further increase emphasis on narrative assessment, sub-internship evaluation, reference letters, academic awards, professional development and medical school prestige.In the meta-analysis, of 6 studies from 2,118 participants, adjusted scaled scores with mean difference from an equal variance model from PDs showed residents from tiered clerkship grading systems overall performance, learning ability, work habits, personal evaluations, residency selection and educational evaluation were not statistically significantly different than from residents from P/F systems.Overall, our dual study suggests that while PDs do not favor P/F core clerkships, PDs do not have a selection preference and do not report a difference in performance between applicants from P/F vs. tiered grading core clerkship systems, thus providing fertile grounds for institutions to examine the feasibility of adopting P/F grading for core clerkships.

Predictors of Excellent Urology Residents at the Time of the Urology Match

ObjectiveTo update our experience and report on features predictive of high quality urology residents at the time of the urology match, because data predicting which medical students will mature into excellent urology residents are sparse. MethodsWe reviewed our experience with 84 urology residents who graduated from 2006 to 2023. Residents were independently scored 1-10 based on overall quality by the current and former Program Director. Discrepant scoring by >2 was resolved by an independent review. Associations of features from the medical student application with an excellent score (defined as 8-10) were evaluated with logistic regression. ResultsDiscrepant scoring >2 was noted in only 5 (6%) residents. Among the 84 residents, the median overall score was 7 (range 1-10) and 36 (43%) residents had an excellent score of 8-10. Univariably, higher USMLE step II score (p=0.03), election to AOA (p=0.004), no negative interview comments (p=0.002), honors in OB/Gyn (p=0.048) and psychiatry clerkships (p=0.04), and honors in all core clinical clerkships (p<0.001) were significantly associated with an excellent score. In a multivariable model, no negative interview comments (p=0.003) and honors in all core clinical clerkships (p=0.001) were independently associated with an excellent score (c-index 0.76). There were several notable features (sex, letters of recommendation, USMLE step I, externship at our institution, surgery clerkship grade, and rank list) that were not significantly associated with excellent residents. ConclusionsWe demonstrate features associated with excellent urology residents, most notably no negative interview comments and an honors grade in all core clinical clerkships.

So You Found Inequities in Clerkship Grades? How to Talk With Students and Community Members About Your Findings.

So You Found Inequities in Clerkship Grades? How to Talk With Students and Community Members About Your Findings.

Why Are Students Appealing Clerkship Grades? A Multischool Root Cause Analysis.

Why Are Students Appealing Clerkship Grades? A Multischool Root Cause Analysis.