Measuring medical education and curriculum during orthopedic surgical residency 1

Abstract

Evaluating residency programs requires objective assessment tools, but few are readily available. The purpose of this study was to measure education by correlating resident test scores with several measurements of educator performance. The study group included residents and educators from a single residency program. We performed a retrospective analysis of scores from the Orthopaedic In-Training Examination collected during a 6-year period. Resident examination scores were indexed by dividing program averages by national averages to determine yearly score trends and then were correlated with educator attendance and teaching hours. Subspecialty scores were ranked to gauge residency strengths and weaknesses. Teaching hours devoted to subspecialties were compared with test scores to measure curricular emphases and to appraise teaching efficiency. Yearly average examination scores were proportional to national averages (P < 0.001). However, of 3436 possible educator-score associations, only 15 scores correlated highly (r > 0.9) with educators, and only 26 were significant (P < 0.05). Trend analysis put subspecialty scores in yearly perspective. Ranking was inaccurate until scores were indexed to the national average. In 2002, the distribution of 238 teaching hours ranged from 4 to 48 h for subspecialties, and 9 of 12 subspecialties were emphasized disproportionately to the examination. Teaching efficiency varied more than 10-fold by subspecialty. The creation of a score index helped to identify and address imbalances between teaching hours devoted to subspecialties and resident needs as evidenced by low In-Training examination scores. The present study improved educator accountability by correlating measurements of teaching and learning.