Correlation of Virtual Reality Simulation and Dry Lab Robotic Technical Skills

Abstract

Study ObjectiveTo examine whether a set of virtual reality (VR) surgical simulation drills have correlative validity when compared with the validated Robotic Objective Structured Assessment of Technical Skills (R-OSATS) dry lab drills. DesignA prospective methods comparison study (Canadian Task Force classification II-2). SettingA teaching hospital. ParticipantsThirty current residents, fellows, and faculty from the Departments of Obstetrics and Gynecology, Urology, and General Surgery. InterventionsParticipants completed 5 VR drills on the da Vinci Skills Simulator and 5 dry lab drills. Participants were randomized to the order of completion. Measurements and Main ResultsVR drills were scored automatically by the simulator. Dry lab drills were recorded, reviewed by 3 blinded experts, and scored using the R-OSATS assessment tool. Spearman correlation coefficients were calculated comparing simulator scores and R-OSATS scores for the same surgeon. The correlation for overall summary scores between VR and dry lab drills was strong (r = 0.83; p < .01). Each of the 5 VR drills was also found to have a statistically significant correlation to its corresponding dry lab drill, with correlation coefficients ranging from r = 0.49 to 0.73 (p < .01 for all). The performance on VR drills also confirmed construct validity. Faculty and fellows had consistently higher overall scores than residents (median VR scores: 458 for faculty, 425 for fellows, 339 for residents; p < .01). ConclusionWe selected a core set of VR drills that reliably correlate with validated dry lab R-OSATS drills. Because dry lab drills require significant time and effort on the part of the trainees and the evaluators, this set of VR drills could serve as an ancillary method of determining trainee competence.