Evaluation of an Automatic Essay Correction System Used as an Assessment Tool

Abstract

In this paper, we evaluate an automatic correction essay system used as an assessment tool on a gamified course. The gamified course uses a question/answer battle as its main strategy to engage and empower students’ learning. As educational methodology, it uses peer review strategy on flipped classrooms. In such context, it was developed an automatic essay correction system, called Milsa, to be used by students out off the classroom. Milsa is used to insert questions and template answers, to automatically correct the questions based on template answers, to show the students the question, the answer and the resulting grade and, finally, to learn from the users’ feedback on the answer’s evaluation. Milsa is used as an assessment tool to measure students’ development at the gamified course. Then, we evaluate the contribution of Milsa to the students’ learning process at the course. We conducted and analyzed tests based on data collected at classes and Milsa: individual flow aligned between the classes, the assessments and an Intrinsic Motivation Inventory (IMI) questionnaire. Finally, we discusses the advantages and disadvantages of the use of Milsa as a social network that helps students with disabilities.