An immersive situational group learning system with body movement and emotion recognition combined with subject knowledge

Abstract

In the current digital situational learning, such as digital game-based learning or Second Life, learners use virtual characters to enter the situational learning, instead of using their own bodies. In addition, many subject matters requiring situational learning need a combination of subject knowledge, body language, and emotional expression, and require students to conduct group learning. However, most of the current situational systems can only provide evaluation and feedback on text or language, and cannot provide feedback on body movements and emotional expressions. To address these issues, this study establishes a situational learning system that enables students to enter the learning situation with their whole body and in small groups. Furthermore, the cognitive services of artificial intelligence are also adopted to conduct immediate evaluation and feedback on students’ learning performance, such as language, body movement, and emotional display, and students’ learning outcomes can be displayed in front of their peers. The experimental results show that the proposed situational learning system can improve students’ learning effect for subject matters that need to combine body movements and emotional expression.