ET-YOLOv5s: Toward Deep Identification of Students’ in-Class Behaviors

Abstract

Recognizing students’ behaviors in classes based on videos of their activity plays an important role in improving teaching quality and paying attention to the healthy growth of students. The existing student behavior recognition methods mainly focus on the behavior of the single student and have low performance and efficiency. To recognize the behaviors of multiple students in the classroom at the same time, we propose a fast and effective solution, called ET-YOLOv5s, which is an improved YOLOv5s with ESRGAN (Enhanced Super-Resolution Generative Adversarial Network) and a tiny object detection module in this paper. First, the ESRGAN is adopted to generate high-definition images from original images of real classroom environments in colleges to improve the recognition accuracy. Then a tiny object detection module is added to the YOLOv5s to detect the smaller objects on high-definition images, such as phones, books, and students sitting at the back of the classroom. Finally, the experimental results show that the proposed solution can detect 11 kinds of students’ behaviors in classrooms, including playing mobile phones, having class, sitting with hands on face, sitting turn right, sitting turn left, bowing, sleeping, and so on. Compared with YOLOv4, YOLOv5s, and other tiny target detection algorithms, it has better detection performance and can effectively deal with the behavior recognition of multiple students.