MP14-03 VALIDATION OF AN INTRAOPERATIVE ASSESSMENT TOOL FOR EVALUATING TRAINEES AND PROVIDING FEEDBACK DURING ROBOT-ASSISTED RADICAL PROSTATECTOMY

Abstract

INTRODUCTION AND OBJECTIVES: As urology training shifts toward competency-based frameworks, valid and reliable methods for high-stakes assessment of trainees are crucial. Standardized evaluation metrics are lacking for robot-assisted radical prostatectomy (RARP). As RARP is becoming the gold standard for treatment of localized prostate cancer, the development and validation of a RARP assessment instrument is timely. We are currently validating a RARP assessment tool for use in the intraoperative setting, with a primary focus on providing feedback throughout the learning curve. METHODS: An initial inventory of 13 procedural steps and 60 sub-steps was generated using a modified Delphi technique at McMaster University. We then recruited 13 RARP surgeons from across North America to serve as our expert Delphi panel. Experts anonymously rated each RARP step and sub-step on a 5-point Likert scale of agreement for inclusion in the final assessment tool. Qualitative feedback was elicited to determine appropriate step placement, wording, and suggestions. Responses were compiled, the inventory was edited through three iterations, and 100% consensus was achieved. Intraoperative da Vinci videos of residents were then recorded as they independently performed the steps of ‘dropping the bladder’ and ‘ligation of the dorsal venous complex’. Five videos each from PGY 3-5 residents and 5 randomly-selected expert videos from our database of RARP cases were edited to include these two steps. Videos have been distributed to our panel of 8 expert RARP surgeons who are serving as raters throughout the validation process. Raters are blinded to level of training and will use the inventory as a guide to assess each step using the validated GEARS tool (Goh, et al.), a procedure-specific checklist, and an overall ‘pass/fail’ rating. Raters will assess 2-4 steps at a time until all steps and sub-steps have been evaluated. RESULTS: Through the Delphi study, steps were decreased by 13% and a skip pattern was incorporated, and there was no attrition until the last round (final N1⁄412). The result was 13 critical steps with 52 substeps. The assessment tool will be piloted in our urology program and revisions will be made as necessary. CONCLUSIONS: Our team has developed the first comprehensive inventory of RARP steps with excellent expert consensus. A novel, psychometrically sound intraoperative RARP assessment tool that is currently undergoing validation will be incorporated into urology curricula. This instrument has the potential to be used for future credentialing of RARP surgeons.