Empowering industrial robotic training with kinesthetic learning and digital twins in human-centric industrial systems

Abstract

This paper presents a human-centric mixed reality (MR) collaborative training platform that employs a kinesthetic learning technique in industrial robotic training, specifically focusing on robot pick-and-place (RPP) operations. Collaborating with a leading robotics industry partner (masked for review), we conducted a user study to investigate the user experiences and practical perceptions of university students and novice trainees via the human-centric training assessment. The study compares the traditional training (TT) RPP classroom as a conventional method with a new collaborative MR RPP training approach (N = 50). The MR training features a digital twin (DT) collaborative robot (cobot) in an Immersive 360° Digital-Objects-Based Augmented Training Environment (360-ATE) using Microsoft HoloLens devices. The research evaluated the impact of MR and DT on human–robot interaction, user experience, task performance, knowledge retention, and interpretation while interacting with 360-ATE, as well as differences in perceptions between the two novice cohorts under each training condition (TT and MR). The primary research question explores whether the collaborative MR training platform integrated DT can be an effective alternative for novice learners in RPP operations. The findings indicate that collaborative MR training is more engaging and effective in enhancing participant safety, confidence, and task performance, which augments cognitive capabilities. The virtual content on the MR devices, especially the digital twin, captured the attention and stimulated active learning abilities. Overall, participants in the MR cohort find the proposed training platform useful and easy to use. The platform has a positive influence on their intention to use similar 360-ATE-assisted training platforms in the future.