Intelligent Teaching Evaluation System Integrating Facial Expression and Behavior Recognition in Teaching Video

Abstract

The student's listening status in the classroom is an important indicator to evaluate that if he takes an active participation in the classroom and study seriously. However, the main challenge in the teaching evaluation is that the teacher in class cannot timely, objectively and accurately evaluate each student's state of listening in accordance with the facial expression or behavior of the students. Along with the advance of deep learning algorithms, artificial intelligence technology is more and more widely applied in the field of education. Based on the above challenges, this paper proposes an intelligent teaching evaluation method that integrates student facial expressions and behaviors in teaching videos, designs and implements a deep learning based intelligent teaching evaluation system. We construct the face detection and recognition model based on deep convolutional neural network and triple loss function to realize the detection and recognition of face regions of students. And then the student facial expression recognition model and the student behavior recognition model based on the deep separable convolutional neural network are constructed. Finally, we propose a novel comprehensive teaching evaluation algorithm by fusion of the student facial expression and behavior, aiming at calculating the comprehensive evaluation value and obtain the corresponding evaluation level. Also, we construct the first teaching video database, student facial expression database and student behavior database for intelligent teaching evaluation. In this paper, the evaluation of students fully combines the students' specific facial expressions under certain behaviors in the classroom. Therefore, the final teaching assessment results are more comprehensive and accurate.