The Arthroscopic Surgery Skill Evaluation Tool Global Rating Scale is a Valid and Reliable Adjunct Measure of Performance on a Virtual Reality Simulator for Hip Arthroscopy

Abstract

The purpose of this study is to further evaluate the construct validity and interobserver reliability of a hip arthroscopy virtual simulator using the Arthroscopic Surgery Skill Evaluation Tool (ASSET) global rating scale. Thirty participants (23 male/7 female) completed a diagnostic arthroscopy and a loose body retrieval simulation on the VirtaMed Arthros Hip Simulator (Zurich, Switzerland) twice at a minimum of 1 week apart. Subjects consisted of 12 novices (medical students, postgraduate year [PGY] 1-2), 5 intermediate trainees (PGY3-4), 9 senior trainees (PGY5 and fellows), and 4 attending faculty. Simulator metrics were recorded and then compiled to generate a total simulator score (TSS). The loose body retrieval was graded using the ASSET scoring tool. Inter-rater and intrarater reliability for the ASSET for 2 blinded raters and construct validity of the ASSET and the TSS were calculated. Correlation between the TSS, ASSET and individual simulator metrics was determined. Prior simulation experience (P ≤ 0.01) correlated with higher TSS and higher ASSET, while video game experience correlated with higher TSS on the diagnostic module only (P= 0.004). There was a significant difference in ASSET score among all experience groups (P < 0.04). Novices had the lowest mean ASSET whereas experts had the highest mean ASSET with a difference of 17.4 points. Overall performance on the surgical module significantly correlated with the ASSET score (r= 0.444, P= 0.016). There was a significant positive correlation among higher ASSET and number of loose bodies retrieved, operation time, camera path and grasper path length, and percentage of cartilage injury. ASSET demonstrated excellent intrarater reliability and showed substantial or better inter-reliability in 8 of 9 domains. The VirtaMed hip arthroscopy simulator demonstrated good construct validity and excellent reliability for simulator-based metrics and ASSET score. Use of both simulator metrics and ASSET offers a more comprehensive performance assessment on hip arthroscopy simulation than either measure alone. As virtual reality simulation for arthroscopy becomes more commonplace in orthopaedic training, evaluation of the most effective objective and subjective measures of performance is necessary to optimize simulation training.