PD02-01 DEVELOPMENT AND VALIDATION OF AN OBJECTIVE SCORING TOOL TO EVALUATE SURGICAL DISSECTION: DISSECTION ASSESSMENT FOR ROBOTIC TECHNIQUE (DART)

Abstract

You have accessJournal of UrologyEducation Research I (PD02)1 Sep 2021PD02-01 DEVELOPMENT AND VALIDATION OF AN OBJECTIVE SCORING TOOL TO EVALUATE SURGICAL DISSECTION: DISSECTION ASSESSMENT FOR ROBOTIC TECHNIQUE (DART) Erik Vanstrum, Runzhuo Ma, Jacqueline Maya-Silva, Daniel I. Sanford, Jessica H. Nguyen, Alireza Ghoreifi, Xiaomeng Lei, Steven Cen, and Andrew J. Hung Erik VanstrumErik Vanstrum More articles by this author , Runzhuo MaRunzhuo Ma More articles by this author , Jacqueline Maya-SilvaJacqueline Maya-Silva More articles by this author , Daniel I. SanfordDaniel I. Sanford More articles by this author , Jessica H. NguyenJessica H. Nguyen More articles by this author , Alireza GhoreifiAlireza Ghoreifi More articles by this author , Xiaomeng LeiXiaomeng Lei More articles by this author , Steven CenSteven Cen More articles by this author , and Andrew J. HungAndrew J. Hung More articles by this author View All Author Informationhttps://doi.org/10.1097/JU.0000000000001966.01AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: There is a need for effective surgical competency assessment to ensure safe and efficient transfer of skills to training robotic surgeons. Additionally, competency assessment has important implications for credentialing and improving patient outcomes. This project aimed to develop an objective scoring tool that comprehensively evaluates surgical dissection quality. METHODS: The study was conducted in two phases. During phase 1, development, the Delphi method was used to validate content of Dissection Assessment for Robotic Technique (DART). A multi-institutional and multi-disciplinary (n=12 Urology, n=1 OBGYN, n=1 General Surgery) panel of expert surgeons systematically evaluated each element; those with content validity index (CVI) of ≥ 0.8 were included in the final product (Figure 1). During phase 2, validation, 10 blinded reviewers evaluated 46 de-identified videos of the pelvic lymph node and seminal vesicle dissection steps during robot-assisted radical prostatectomy. Inter-rater reliability was calculated using prevalence-adjusted and bias-adjusted kappa (PABAK). Expert and trainee surgeons were defined by prior robotic caseload >100 and ≤100, respectively. Mann-Whitney U test was used to compare between groups. RESULTS: In phase 1, Delphi method achieved content validity in 27/28 elements after 4 rounds. The element that did not reach consensus was use of a 3-point or 5-point scale (CVI=0.64). Most surgeons (93%) agreed that it is important to maximize agreement between raters. In phase 2, the 3-point scale showed greater inter-rater reliability for each domain as compared with the 5-point scale (PABAK≥0.52, ≥0.38, respectively). When rated on the 3-point scale, total scores between experts and trainees exhibited a significant difference (median 17.2 (IQR 16.9-17.4) vs. 15.7 (14.8-16.3), p<0.001). CONCLUSIONS: DART is an objective and reproducible 3-point surgical assessment that effectively differentiates between levels of surgical experience. Source of Funding: Research reported in this publication was supported in part by the National Institute Of Biomedical Imaging And Bioengineering of the National Institutes of Health under Award Number K23EB026493 © 2021 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 206Issue Supplement 3September 2021Page: e37-e37 Advertisement Copyright & Permissions© 2021 by American Urological Association Education and Research, Inc.MetricsAuthor Information Erik Vanstrum More articles by this author Runzhuo Ma More articles by this author Jacqueline Maya-Silva More articles by this author Daniel I. Sanford More articles by this author Jessica H. Nguyen More articles by this author Alireza Ghoreifi More articles by this author Xiaomeng Lei More articles by this author Steven Cen More articles by this author Andrew J. Hung More articles by this author Expand All Advertisement Loading ...