Assessing the validity of VR as a training tool for medical students

Abstract

The advances in Virtual Reality technologies, increased availability and reducing hardware costs have diminished many of the early challenges in the adoption of VR. However, a commonly identified gap in immersive Virtual Reality-Head Mounded Display (VR-HMD) training for medical education is the confidence in the long-term validity of the applications, in particular, the acceleration of the learning curve efficacy of learning outcomes over time and actual skills translation into real environments. Research shows a wide range of ad hoc applications, with superficial evaluations often conducted by technology vendors, based on assumed environments and tasks, envisaged (as opposed to actual) users and effectiveness of learning outcomes underpinned with little or no research focusing on a requirements-driven validation approach. This presents decision-making challenges for those seeking to adopt, implement and embed such systems in teaching practice. The current paper aims to (i) determine whether medical VR training improves the skill acquisition of training candidates, (ii) determine the factors affecting the acquisition of skills and (iii) validate the VR-based training using requirement-driven approach. In this paper, we used within- and between-subject design approaches to assess the validity of VR-based surgical training platform developed by Vantari VR against requirements which have been identified to have impact on learning processes and outcomes in VR-based training. First, study and control groups were compared based on their level of skill acquisitions. Then, by tailoring a requirements framework, the system was validated against the appropriate requirements. In total, 74 out of 109 requirements were investigated and evaluated against survey, observer and stakeholder workshop data. The training scenario covered the topic of Arterial Blood Gas (ABG) collection for second-year university medical students. In total 44 students volunteered to participate in this study, having been randomly assigned to either the study or control group. Students exposed to VR training (the study group) outperformed the control group in practical clinical skills training tasks and also adhered to better safety and hygiene practices. The study group also had a greater procedural completion rate over the control group. Students showed increased self-efficacy and knowledge scores immediately post-VR training. Prior ABG training did not impact on VR training outcomes. Low levels of simulation sickness, physical strain and stress, coupled with high levels of enjoyability, engagement, presence and fidelity were identified as factors affecting the overall training experience. In terms of learning, high scores were recorded for active learning, cognitive benefit and reflective thinking. Lastly, by validating the system against 74 system requirements, the study found a user acceptance level of 75%. This enabled the identification of weaknesses of the current system and possible future directions.