Learning experience assessment through players chat content in multiplayer online games

Abstract

Assessing players’ learning experiences in a proper manner is a fundamental aspect of successful game-based learning programs. One notable characteristic of these programs is stealth assessment, which involves integrating formative assessment into the learning environment without disrupting the learning process. In multiplayer online games (MOGs), the in-game online chat system is a commonly used tool that enables players to communicate through text or voice messages during gameplay. However, there is a lack of specific research on incorporating players’ in-game chat content for computational learning experience assessment, which could enhance the validity of stealth assessment. This study proposes a stealth assessment method based on natural language processing to highlight the significance of players’ in-game chat data in estimating learners’ skills in MOGs. A natural language processing model is developed using a distilled version of the Google BERT pre-trained model. The evaluations demonstrate that the proposed method accurately estimates a player’s skill level by analyzing a few chat messages from the player. This method has the potential to make a profound impact on the field of game-based learning by enabling more precise assessment and supporting the design of tailored interventions and adaptive learning systems. This study pioneers computational skill assessment through chats in MOGs, opening up new opportunities for future investigations in skill assessment and having the potential to transform the field of game-based learning.