Short Answer Scoring in English Grammar Using Text Similarity Measurement

Abstract

Assessment in educational institution is an important system to evaluate academic performance among students. The assessment is done by teacher either manually or using automated scoring technology. This study employed natural language processing approach to automated short answer scoring system using textual similarity. There are various types of questions in an examination paper. These include multiple choices question, short answer based question, fill-in-the-blanks questions and essay questions. In this study, the focus is on fill-in-the-blanks questions type. Students are required to answer each question with 2-5 words. The scope of the subject is narrowed to English grammar for secondary school as a datasets for this study. The datasets included 240 responses for 10 questions selected randomly. Students’ answers are mapped with model answers to measure the textual similarities. The mappings were done using Levenshtein distance (LD) and Cosine similarity measures. Both textual similarity techniques assigned marks to each response according to the similarity distance of student answer and model answer. Certain range of distance values is restricted for both textual similarity techniques. The effectiveness of textual similarity in scoring short based answer is compared with human grader scoring. Both textual similarity techniques show high agreement with human grader for assigning full marks where the maximum percentage is 92 and 94 percent for LD and Cosine similarity respectively. This work should be useful to assist teacher to ease the onerous task of grading.