Evaluation of an immersive virtual learning environment for operator training in mechanical and plant engineering using video analysis

Abstract

AbstractA structural evaluation is imperative for developing an effective virtual learning environment. Understanding the extent to which content that has been learned virtually can be applied practically holds particular importance. A group of persons from the technical field of mechanical and plant engineering (N = 13) participated in a virtual operator training for a case application of additive manufacturing. To evaluate the virtual learning environment the participants answered quantitative questionnaires and were asked to apply what they had learned virtually to the real machine. Both the virtual training and testing phase on the real machine were recorded by video (800 minutes in total). The category system resulting from a structured qualitative video analysis with a total of 568 codes contains design‐, instruction‐ and interaction‐related optimisation potentials for further development of the virtual learning sequence. Mistakes, difficulties and other anomalies during the application on the real machine provide further revision options. The study uses video data for the first time to derive optimisation potentials and to investigate the learning transfer of virtually learned action knowledge to the real‐world activity. Practitioner NotesWhat is already known about the topic? A combination of quantitative and qualitative data for formative evaluation is feasible to improve the quality of virtual learning environments and maintain critical constructive feedback. Relevant aspects for evaluating virtual learning environments include usability, technology acceptance, motivation and learning efficiency. Different methods for measuring the learning efficiency of VR training exist, eg, subjective measures or pretests/posttests. What this paper adds: An immersive virtual environment with its specific methodical‐didactic concept is introduced. Video data are used for the first time to derive optimisation potentials in a formative evaluation. The procedure of extracting and analysing the data in a structured video analysis is introduced in the paper. Learning transfer of the virtually learned content to the real‐world context of action is assessed and applied in the field of mechanical and plant engineering on a complex real machine for the first time. Implications for practice and/or policy: The described methodical procedure using video analysis enables deriving design‐, instruction‐ and interaction‐related optimisation potentials. The results show that spatial information was transferred to the real machine without any issues, whereby the main difficulties appeared during steps that require haptic feedback. The use of IVR should be critically assessed due to the effort required in creation.