Facial Affect Sensing for T-learning

Abstract

Interactive Digital TV has arisen new forms of interaction not traditionally associated with this medium. One of its applications is the so-called “t-learning” or TV-based interactive learning. To date, most existing t-learning applications confine themselves to edutainment more than to more complex and pedagogical forms of learning, due to the technological constraints imposed by set-top boxes. This paper overcomes those technological barriers to propose the first t-learning affective aware tutoring tool. The tool allows to capture by means of a camera the facial expressions of the student while performing evaluations (tests) through a broadcasted interactive t-learning application at home. It integrates a novel Artificial Intelligence method for facial affect recognition, where facial expressions are evaluated with a psychological 2-dimensional continuous affective approach. Thanks to it, the t-learning tool is able to automatically extract emotional information from the learner, which is further presented in a simple and efficient way to the distance tutor so that he can be aware of student’s encountered difficulties and emotional progression during the learning process.