Real-time hand interaction and self-directed machine learning agents in immersive learning environments

Abstract

Integration of extended reality (XR) in education is becoming popular to transform the traditional classroom with immersive learning environments. The adoption of immersive learning is accelerating as an innovative approach for science and engineering subjects. With new powerful interaction techniques in XR and the latest developments in artificial intelligence, interactive and self-directed learning are becoming important. However, there is a lack of research exploring these emerging technologies research with kinesthetic learning or “hands-one learning" as a pedagogical approach using real-time hand interaction and agent-guided learning in immersive environments. This paper proposes a novel approach that uses machine learning agents to facilitate interactive kinesthetic learning in science and engineering education through real-time hand interaction in the virtual world. To implement the following approach, this paper uses a chemistry-related case study and presents a usability evaluation conducted with 15 expert reviewers and 2 subject experts. NASA task load index is used for cognitive workload measurement, and the technology acceptance model is used for measuring perceived ease of use and perceived usefulness in the evaluations. The evaluation with expert reviewers proposed self-directed learning using trained agents can help in the end-user training in learning technical topics and controller-free hand interaction for kinesthetic tasks can improve hands-on learning motivation in virtual laboratories. This success points to a novel research area where agents embodied in an immersive environment using machine learning techniques can forge a new pedagogical approach where they can act as both teacher and assessor.