Residency Program Director Perceptions of Resident Performance Between Graduates of Medical Schools With Pass/Fail Versus Tiered Grading System for Clinical Clerkships: A Meta-Analysis.

Abstract

Purpose: The variability and imprecision of clerkship grading raise concerns for program directors (PDs) during applicant selection. 1 Although the majority of medical schools have adopted pass/fail (P/F) preclinical grading, only 12 schools formally use P/F grading for clerkships. 2 Additionally, per Liaison Committee on Medical Education recommendations, events such as Hurricane Katrina and the COVID-19 pandemic have led additional schools to adopt P/F clerkship grading temporarily. 3 Moreover, the rise in studies describing student- and institutional-level disparities in clerkship grading has tasked medical education personnel with weighing the fairness and risk of disparities as well as the potential benefits of tiered grading. 1,4 With the continued paucity of literature examining P/F grading in core clerkships, 5 we conducted a meta-analysis of studies comparing PDs’ perceptions of residency performance among residents from schools using P/F versus tiered clerkship grading systems. Approach/Methods: Embase, PubMed, and Scopus were searched since inception through October 2020, and hand searches were performed of the retrieved reference lists. No study or language restrictions were applied. Studies exploring P/F clerkship in the context of a cohort of PD assessments were included, and the CLARITY risk of bias was used. Reviewers assessed study characteristics, overall resident performance, learning ability, work habits, work products, educational assessments, and PD’s personal evaluation (worse: 0 to best: 100), and were assessed using Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines. To account for different survey and grading metrics, adjusted standard difference in means were used. Main outcomes and measures were program director ratings of residents from U.S. medical schools using tiered versus P/F clerkship grading. Results/Outcomes: From 4,931 studies screened, we identified 6 eligible studies (3 cohort, 3 surveys; 6 low risk of bias) with 2,118 participants. From 31 accredited medical specialties, 7 specialties were represented with a median response rate of 81.0% (95% CI, 49.0–100.0). Reported as means, there was no difference in PD preference for residents from P/F or tiered grading system throughout residency training (37.0% Tiered; 95% CI, 0–100, P > .05). Adjusted scaled scores utilizing mean difference from an equal variance model from PDs showed overall performance (5.5; 95% CI, –1.9 to 12.9), learning ability (2.7; 95% CI, 0–5.4), work habits (2.9; 95% CI, 0–5.8), personal evaluations (–1.6; 95% CI, –3.8 to 0.6) and educational evaluation (1.7; 95% CI, –0.8 to 4.3) of residents from tiered clerkship grading systems were not statistically significant (P > .05) from P/F residents. However, there was a difference in work products produced (6.8; 95% CI, 1.4–12.2, P < .0001). Meta-regression standard difference in means revealed that there is no difference in tiered applicant’s overall performance in residency compared with P/F applicants (0.0001 fixed, P = .98; –0.0047 random, P = .81) Discussion: Clerkship performance has often been used as a metric to assess medical student preparedness for residency. 1,2 While PDs reported that residents from medical schools using P/F clerkship gradings tended to perform slightly lower on average, these results were not statistically significant. In addition, PDs did not generally prefer applicants from tiered medical schools. Given the continued expansion of P/F grading in medical school curricula, our findings allow for early discussion regarding the implications of P/F clerkship grading and its anticipated reception by stakeholders. Significance: In our cohort, there appears to be no perceived difference in resident performance based on clerkship grading system. This study provides further impetus for discussion of wider adoption of P/F grading in clinical clerkships.