Assessment of communication, professionalism, and surgical skills in an objective structured performance-related examination (OSPRE): a psychometric study

Abstract

Background The purpose of this study was to investigate the reliability and validity of a performance assessment of communication, professionalism, and surgical skills competencies for surgery residents. Methods Fourteen residents from the general surgery program of the University of Calgary were assessed in 7 surgical simulation stations that included communication and professionalism skills. Results The internal consistency reliability of the checklists and global rating scales combined was adequate for communication (α = .75–.92) and surgical skills (α = .86–.96), but not for professionalism (α = 0). There was evidence of validity as surgical skills performance improved as a function of postgraduate year level but not for the professionalism checklist. Surgical skills and communication correlated in the 2 stations assessed ( r = .55 and .57; P < .05). Conclusions There is evidence for both reliability and validity for simultaneously assessing surgical skills and communication skills. Further instrument development is required to assess professionalism in a structured examination context.