Da Vinci Skills Simulator Construct Validation Study: Correlation of Prior Robotic Experience With Overall Score and Time Score Simulator Performance

Abstract

To assess the construct validity of the da Vinci Skills Simulator (Intuitive Surgical, Sunnyvale, CA). Ideally, a well-designed simulator should demonstrate construct validity, which is defined in this study as the correlation between robotic surgical experience and performance on the simulator. Thirty-nine surgeons (18 [46%] group I [0-20 robotic cases]; 8 [21%] group II [21-150 robotic cases]; and 13 [33%] group III [>150 robotic cases]) were enrolled from September 2010 to December 2010. Participants completed 24 virtual-reality exercises on the da Vinci Skills Simulator. Data on 12 performance metrics were collected by the software. Overall means for score and time across exercises were analyzed. Overall scores (64.7%/79.1%/87.4%) and time scores (39.1%/58.6%/87.3%) were significantly different among surgeons in groups I-III (P <.001) and demonstrated significant linear relationships (P <.001) for all 24 exercises. Comparisons between the 3 groups using a univariate general linear model (GLM) was used to compare groups I and II and II and III. Groups I and II differed using overall score for 15 exercises and time score for 11 exercises. Groups II and III differed using overall score for 6 exercises and time score for 15 exercises. Mean overall score for 1 exercise displayed significance between both groups I and II and II and III; while using time score, 5 exercises displayed significance between surgeons in groups I and II and II and III. Initial construct validity analysis revealed that both overall scores and time scores showed a significant linear relationship when comparing the surgeons in groups I, II, and III. Overall score seems to be a stronger indicator for differences between surgeons in groups I and II. Time score seems to be a stronger indicator for differences between surgeons in groups II and III.