Enhancing Formative Feedback in Orthopaedic Training: Development and Implementation of a Competency-Based Assessment Framework

Abstract

The purpose of this study was to develop, implement, and evaluate the effectiveness of an assessment framework aimed at improving formative feedback practices in a Canadian orthopaedic postgraduate training program. Tool development began in 2014 and took place in 4 phases, each building upon the previous and informing the next. The reliability, validity, and educational impact of the tools were assessed on an ongoing basis, and changes were made accordingly. One hundred eighty-two tools were completed and analyzed during the study period. Quantitative results suggested moderate to excellent agreement between raters (intraclass correlation coefficient = 0.54-0.93), and an ability of the tools to discriminate between learners at different stages of training (p's < 0.05). Qualitative data suggested that the tools improved both the quality and quantity of formative feedback given by assessors and had begun to foster a culture change around assessment in the program. The tool development, implementation, and evaluation processes detailed in this article can serve as a model for other training programs to consider as they move towards adopting competency-based approaches and refining current assessment practices.