Visual attention based evaluation for multiple-choice tests in e-learning applications

Abstract

Multiple-choice (MC) question is an important form of test to assess the students' academic achievement, especially in the e-learning applications. However, the classical evaluation metrics on MC questions (such as the correctness ratio) only consider the correctness of the final selection but ignore the solving progress of the testee. In the existing literature, the eye-tracking based visual attention was studied to infer the testee's cognitive progress towards a specific MC question. However, there is little work on the visual attention based evaluation of one complete MC test. In this paper, we measure the eye movement data of a group of students in an online test, which consists of forty more MC questions. We divide the screen area into five AOIs (area of interests), including one for the question and four for the candidate options. The fixation duration as well as the gaze sequence on these AOIs are recorded and studied. In the case study on the most difficult question, we observe the great differences among the eye movement of the testees in different academic levels. A new metric, namely Visual-Attention-assisted Score (VAS), is proposed to assess the student's performance with the bias of his fixations on the correct options. Experiment results show that, this metric can reflect the difference of gaze movement of testees, and thus it is helpful for the teachers to infer the real level of the students' academic achievement.