A Pilot Study Evaluating a Virtual Reality-Based Nontechnical Skills Training Application for Urology Trainees: Usability, Acceptability, and Impact

Abstract

This study aimed to develop and evaluate a virtual reality (VR)-based nontechnical skills (NTS) training application for urology trainees and assess its effectiveness in improving their skills and confidence. A mixed-methods study was conducted to develop and evaluate a VR-based NTS training application for 32 urology trainees. The development process involved collaboration with 5 urology experts, 2 medical education specialists, and a human factors researcher. The study evaluated the application's usability, acceptability, and efficacy through 3 phases: scenario development with expert feedback integration, storyboarding and creation processes with facilitators and urology trainees, and a final evaluation by trainees. The data were collected during a 4-day urology boot camp in October 2022. Thirty-two urology trainees participated in the study and completed 2 VR scenarios designed to enhance their NTS skills RESULTS: The System Usability Scale (SUS) showed a moderate usability score of 66. The Training Evaluation Inventory (TEI) and additional feedback demonstrated positive effects on trainees' learning and confidence in their NTS abilities. Most participants found the application easy to use, and effective and they expressed interest in using similar VR applications for other aspects of surgical training. VR-based NTS training applications show potential for enhancing urology trainees' nontechnical skills. The integration of expert feedback and immersive technology offers a promising, accessible, and cost-effective solution to the challenges of delivering NTS training. Future research should explore the long-term impact of VR-based NTS training on trainees' performance and patient outcomes and consider incorporating advanced AI technologies for personalized and dynamic learning experiences.