MP20-11 DEVELOPMENT OF A SMARTPHONE-FRIENDLY WEB APP FOR LIVE EVALUATION OF ROBOTIC SURGERY

Abstract

You have accessJournal of UrologySurgical Technology & Simulation: Training & Skills Assessment II1 Apr 2016MP20-11 DEVELOPMENT OF A SMARTPHONE-FRIENDLY WEB APP FOR LIVE EVALUATION OF ROBOTIC SURGERY Thomas Bottyan, Thomas G. Clifford, Zein K. Nakhoda, Sarfaraz Serang, and Andrew J. Hung Thomas BottyanThomas Bottyan More articles by this author , Thomas G. CliffordThomas G. Clifford More articles by this author , Zein K. NakhodaZein K. Nakhoda More articles by this author , Sarfaraz SerangSarfaraz Serang More articles by this author , and Andrew J. HungAndrew J. Hung More articles by this author View All Author Informationhttps://doi.org/10.1016/j.juro.2016.02.2780AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookTwitterLinked InEmail INTRODUCTION AND OBJECTIVES Herein we describe an initial experience with data-driven development and implementation of a novel web application to assess and track trainee performance within a structured robotics curriculum based on stepwise learning. Graduation of surgical steps requires demonstration of proficiency, as tracked by the application. METHODS A skills evaluation form was developed utilizing the enterprise survey solution Qualtrics (Provo, Utah) augmented with custom R and Javascript software coding. An initial root survey optimized for smartphones queries trainees on what steps of a robotic case they performed. Their responses trigger new evaluation forms to be automatically sent to the attending surgeon, fellow, and scrub technician on the case to evaluate the trainee's performance on those steps (Figure). Trainees were evaluated using the Global Evaluative Assessment of Robotic Surgery (GEARS) tool and a novel Proficiency Score. Survey compliance rates, time to complete survey, and number of clicks required were measured and compared for three distinct phases: 1. initial testing phase (2/17/15 - 3/31/15), 2. mature phase (4/01/15 - 4/30/15), and 3. maintenance phase (5/01/15 - 6/30/15). RESULTS Surveys were distributed to trainees on a total of 171 cases, of which 117 (68%) were deemed teaching cases. Overall compliance increased from 53.5% to 85.7% between phase 1 and 2, respectively (p<0.0001) due to maturation of the app and reporting of individual compliance, with a slight decrease to 71.5% during phase 3 (p<0.0004). Median number of clicks and time required to complete the survey were used to guide app development after phase 1 and decreased from 10 to 6 and 1:34 to 1:05 between phases 2 and 3, respectively (<0.0001). Users rated the app as “very user friendly” (median 4.75, range 4-5 of 5) and survey criteria had high clarity (4.88, 4-5 of 5). An average of 6.1 minutes (range: 2-12.2) of nightly maintenance is required to assign surveys and extract compliance and score reports. CONCLUSIONS During this pilot study, participant data was used to drive app development. High overall compliance suggests sustainability of this educational tool. Streamlining the existing app may enable other residency training programs to adopt the curriculum. © 2016FiguresReferencesRelatedDetails Volume 195Issue 4SApril 2016Page: e216-e217 Advertisement Copyright & Permissions© 2016MetricsAuthor Information Thomas Bottyan More articles by this author Thomas G. Clifford More articles by this author Zein K. Nakhoda More articles by this author Sarfaraz Serang More articles by this author Andrew J. Hung More articles by this author Expand All Advertisement Advertisement PDF downloadLoading ...