PSU-CNN: Prediction of student understanding in the classroom through student facial images using convolutional neural network

Abstract

Facial expressions are a set of symbols of great importance for human-to-human communication. This communication can be assisted/optimized through use of artificial intelligence. Student-teacher interaction is important human-to-human communication. Spontaneous in nature, diverse and personal, facial expressions demand real-time, complex, robust and adaptable student’s facial expression recognition systems to facilitate the student-teacher interaction. This paper investigates the idea of performing automated analysis of a student’s understanding of the class participating in active face-to-face classroom teaching. Every student can understand the lecture which is crucial in classroom teaching. In the current study the facial images of the student were collected during the lecture. The facial images were tagged with help of student according to their understanding of lecture at that particular moment. This problem pursued by analyzing the faces of students attending the class and classifying the facial image as “Understanding” or “Not Understanding” in the lecture. Different machine learning (ML) methods were used for development of the model for the classification and the proposed deep learning framework (convolutional neural network (CNN)) outperformed the other methods i.e. Support Vector Machine, Naive Bayes classifier, achieving the test accuracy of 92%, Current work concludes that deep learning like CNN can be the better method for the classification of facial images as compared to other ML methods such as SVM and Naive Bayes for student understanding. Further high accuracy of the methods justifies its importance in optimization of classroom teaching.