STUDENT ACTION RECOGNITION AS A MEANS OF SUPPORTING REMOTE LEARNING

Abstract

Due to the COVID-19 pandemic, many schools worldwide are using tele-education for class delivery. However, this causes a problem related to students’ active class participation, especially in the cases that due to privacy issues no videos of students are provided to the teacher. We propose to address the problem with a system that recognizes student’s actions and informs the teacher accordingly, while preserving the privacy of students. In the proposed action recognition system, the actions of being absent, writing, attending, hand raising, telephone call, using phone and looking elsewhere are recognized with the use of a deep network. The actions considered were defined by considering the relevant literature and educator’s views and ensure that they provide information about the physical presence, active participation, and distraction of students, that constitute important pedagogical aspects of class delivery. In our proposed system, real-time feedback will be provided to educators to inform them if the students are adopting any of the actions considered, so that educators know at any time the level of student engagement. Although the recognition of student's actions has been addressed in the literature before, to the best of our knowledge, the proposed formation of the application within the domain of tele-education has not been considered before.As part of the experimental evaluation, videos of primary school students were captured while they were attending a class through teleconferencing. Students participating in the experiment were asked to perform the seven actions considered in this work. For each action, a ten-second video was recorded and for each student participating in the experiment, two sets of videos have been recorded (Session A and Session B), in which the students had been wearing different clothes while the background had been different for the two sessions. The data collected was used for training and testing a deep network model capable of recognizing different actions depicted in video frames. Since we aim to keep computational load and memory requirements to the minimum, the Squeezenet deep network architecture was selected for the recognition process. Initial experimental results indicate that the proposed action recognition system provides promising classification results, when dealing with new instances of previously enrolled students or when dealing with previously unseen students. The proposed model achieved the best performance when dealing with the recognition of actions related to the behavioural disaffection class, which is crucial for the teaching-learning process during teleconferencing. The results indicate the feasibility of this approach that can be used as the basis of implementing a system that helps educators to monitor actions of participants during remote class delivery. An important aspect of the system is the fact that the privacy of students is protected since teachers receive only real time information about students' actions rather than receiving images of the students. Apart from monitoring student behaviour and participation during tele-education, the proposed system can be used for the implementation of flipped classrooms and virtual clones of physical classes.