Entrustment versus performance scale in high-stakes OSCEs: Rater insights and psychometric properties

Abstract

Background Although entrustment scales are increasingly applied in workplace-based assessments, their role in OSCEs remains unclear. We investigated raters’ perceptions using an entrustment scale and psychometric analyses. Method A mixed-methods design was used. OSCE raters’ (n = 162) perceptions were explored via questionnaire and four focus groups (n = 14). Psychometric OSCE properties were analyzed statistically. Results Raters (n = 53, response rate = 41%) considered the entrustment scale comprehensible (89%) and applicable (60%). A total of 43% preferred the entrustment scale, 21% preferred the global performance scale, and 36% were undecided. Raters’ written comments indicated that while they appreciated the authenticity of entrustment levels, they considered them subjective. The focus groups highlighted three main themes: (1) recollections of the clinical workplace as a cognitive reference triggered by entrustment scales; (2) factors influencing entrustment decisions; and (3) cognitive load is reduced at the perceived cost of objectivity. Psychometric analyses (n = 480 students) revealed improvements in some OSCE metrics when entrustment and global performance scales were combined. Conclusion Entrustment scales are beneficial for high-stakes OSCEs and have greater clinical relevance from the raters’ perspective. Our findings support the use of entrustment and global performance scales in combination.