Obtaining evidence of learning in digital games through a deep learning neural network to classify facial expressions of the players

Abstract

This Emotions play a significant role in determining human behavior, including the involvement and motivation of students, in the learning process. A digital game have become an important option, as a pedagogical tool, for the development of skills currently required for the insertion of an individual in labor market. This paper reports a research on a computational architecture, based on the techniques and methods of Learning Analytics, to assist educators, who use digital games as pedagogical tools, in assessing the impacts of a player's emotional states on his motivation to play and, consequently, in their learning. Such architecture aims to identify data on players' emotional states, through their facial expressions, and cross them with data on the players ‘actions, game events or states, which can evidence their learning. A deep learning neural network is used in the identification of the players´ facial expressions, through images of the faces of the students playing. The capture of these images is done by a webcam, installed on the computer where the game is playing, which is triggered by an intelligent software agent, according to pre-established conditions. This intelligent software agent also captures the data about the learning evidences. A software module, to analyze learning and the possible impacts of players' emotional states on their learning, completes the proposed computational architecture. The work was developed in the discipline on digital games applied to Education in a postgraduate course in Computer Science.