Assessing Residents’ Competency at Baseline: How Much Does the Medical School Matter?

Abstract

Purpose: Medical education is in the midst of a shift towards a more competency-based assessment of learners.1,2 Although there is now some agreement about the competencies needed to enter residency,3 what few data we have about the actual skills of graduating medical students suggest that many are not meeting expectations.4 Although there is some evidence of an association between residency education and practice patterns,5 little is known about the association between undergraduate medical education and performance during residency. The goal of this study was to explore the association between medical school of origin and clinical performance, as assessed by a multistation objective structured clinical examination for incoming residents at the University of Michigan. Method: All first-year residents at the University of Michigan participate in the Postgraduate Orientation Assessment (POA) prior to assuming clinical duties. The POA focuses on competencies that are needed during the first months of residency. Performance data for 1,795 residents were collected between 2002 and 2012. Residents were assessed in four core skills: data gathering, clinical assessment, team skills/procedure competencies, and communication skills. A mixed general linear model was conducted for each core skills score, using “medical school” as a random factor, to estimate the association of a particular school with each score and the amount of variance that school explained in the overall sample. Results: Medical school predicted 1.67% [95% CI 0.36%–2.93%] of the variance of data gathering scores; 4.93% [95% CI 1.84%–6.00%] of the variance of assessment scores; 0.80% [95% CI 0.00%–1.82%] of the variance of teamwork scores; 2.37% [95% CI 0.66%–3.83%] of the variance of communication scores; and 4.19% [95% CI 1.59%–5.35%] of the variance of the overall POA score. Conclusions: These results suggest that resident medical school of origin is associated with clinical competency. The practical significance of these factors needs further evaluation, but the results highlight the potential variability in resident preparation and the need for individualized learning plans and feedback.