Emotions and gesture recognition using affective computing assessment with deep learning

Abstract

Emotions have an important role in education. Affective development, attitudes, and emotions in learning are measured using affective assessment. This method is the right way to determine the student’s affective development. However, the process did not run optimally because the teacher found it difficult to collect student’s affective data. This paper describes the development of a system that can assist teachers in carrying out affective assessment. The system was developed using a v-model that aligns the verification phase with the validation. The use of the system is carried out during learning activities. The emotion detection system detects through body gestures using PoseNet to generate emotional data for each student. The detection results are then processed and displayed on an information system in the form of a website for affective assessment. The accuracy of emotion detection got validation values of 84.4% and 80.95% after being tested at school. In addition, the acceptance test with the usability aspect of the system by the teacher got a score of 77.56% and a score of 79.85% by the students. Based on several tests carried out, this developed system can assist the process of implementing affective assessment.