Can Robotic Surgery VR-Simulation Metrics be Used to Devise a Universal Proficiency Index That Would Allow Comparisons for All Users?

Abstract

Study Objective Calculate a universal proficiency metric for Robotic Surgery VR simulation that will allow comparison of all users across any VR curriculum. Design Retrospective analysis of VR Simulation metrics. Setting Two training institutions. Patients or Participants Residents, Fellows and practicing surgeons. Interventions By analyzing a robotic surgery database of over 600,000 sessions using Mimic simulators, we calculated Mean scores for each exercise. Those Mean scores were then normalized to 100. Subject's scores were also averaged and normalized to 100. We called this Index score the MLearnC Proficiency Index (MPI). Scores above 100 were better than average; Less than 100 were worse than average. Measurements and Main Results 17,648 sessions were analyzed (2017-2020) comparing 77 students (residents to practicing surgeons) working in 7 different curriculums. On average, each student spent 8 hours and 24 minutes on simulation, attempted 26.5 different exercises, and became proficient in 20.6 exercises per user. The MPI mean score for all participants in all curricula was an MPI of 104.9 (SD: 15.5). 13 students were 1 standard deviation below the norm with an average MPI of 80.15. This group averaged 9 hours 27 minutes each on the simulator attempting 23.46 exercises but becoming proficient in only 10.38 (47%) of them in 224 sessions. 12 students were 1 standard deviation above the norm with an average MPI of 127.05. This group averaged 6 hours 31 minutes each on the simulator attempting 29.08 exercises but becoming proficient in 27.5 (95%) of them in 196 sessions. Conclusion A universal skill-based performance index (MPI) was calculated and found to be a reliable tool that could be used to identify relative proficiency among students in different robotic surgery VR Simulation curriculums. An individual user's proficiency can be utilized to identify a student's progress in a given curriculum. Future studies of MPI will determine if machine learning can provide timely personalized feedback to the user.