545 Review of the Validity of DOPS as an Assessment Tool for the Procedural Skills of Surgical Trainees

Abstract

Abstract Introduction Surgical trainees at all stages are mandated to use workplace-based assessments (WBAs) to gain feedback from their trainers. Direct observation of procedural skills (DOPS) is a tool to assess hands-on surgical skills. This review of the literature seeks to ascertain how valid DOPS are as an assessment tool for the procedural skills of surgical trainees according to the APA (American Psychology Association) validity framework. Method Relevant literature was identified through a structured search of Medline, CINAHL and web of science databases, with further papers included on citation review. Following this, papers meeting the inclusion and exclusion criteria were included in the final review specifying those investigating WBAs including DOPS, assessments in practice rather than simulation and specifically for post-graduate surgical trainees. Results 16 papers were included in the final analysis. Extracted data from the returned papers was assessed for evidence of validity within each of the five domains on the APA framework: validity based on consequence, response process, reliability, content, and relationship to other variables. Conclusions There are studies that show good parameters for reliability and validity for specific DOPS used in endoscopy and otolaryngology. However, there is confusion over the purpose of DOPS amongst trainers and trainees as to whether they should be used formatively or summatively. There is a wealth of evidence examining the perspectives of trainees and trainers into the summative nature of DOPS, however future research should be conducted into to validity of DOPS as a summative tool for career progression and selection processes.