What Evidence is There for the Use of Workplace-Based Assessment in Surgical Training?

Abstract

Shalhoub and colleagues have conducted a pertinent review of the evidence for workplace-based assessments (WBAs) within surgical training. However, there are a number of further important points that need highlighting. The validity and reliability of the WBAs used within the intercollegiate surgical curriculum project in the UK have previously been questioned. 1,2 Determining the validity of these assessments is not easy. Although they would all appear to have face validity, it is less easy to determine their degree of criterion validity and in particular, their ability to predict future performance. WBAs cannot claim to be valid without also being reliable. We agree with the authors’ comments that the sheer number of assessments required by some deaneries is extreme. This is especially so given the lack of infrastructure within hospitals for facilitating electronic assessments and the additional time required by supervisors to complete assessments. Another important determinant of reliability is “triangulation” of assessments. This requires assessments to be performed on multiple occasions, by multiple trainers, and where possible, using more than one assessment tool, and it has not been addressed by the current system. The reliability of WBAs may further be compromised by their potential misuse by both trainers and trainees. Originally designed as formative assessment tools, Shalhoub and colleagues correctly state that they are often completed in a summative manner, defeating their original purpose. As an important reason for the use of WBAs is to identify struggling trainees, incorrect use of these assessments in this manner can make it very challenging to identify such trainees.