Multi-modal affect detection for learning applications

Abstract

Affect detection is a key factor needed for automated Human Computer Interaction(HCI) applications. Affective computing helps understand the user's state of mind or emotions and hence helps to provide a better service. In this paper, we propose the use of multimodal affect detection to capture human attention for learning environment. For capturing affective state, we use facial features and brain signal of the user. To identify the human attention level, machine learning was used. The system is an assistant tool to identify attention levels of user for distant learning courses as well as for any e-learning applications. Experiments were conducted with volunteers to understand user affective state with different course materials. Results indicate that the proposed system can be used to detect affective state automatically. The proposed approach can be extended to other HCI applications where attention detection is needed.