AraScore: Investigating Response-Based Arabic Short Answer Scoring

Abstract

There are more than 80 million students in the Arab world. Students take tests and examinations as an intrinsic part of their educational journey to assess the quality of learning and understanding of the examined material. Exams require an immense amount of resources to be conducted, but even a bigger amount of resources to be scored. Scoring of exams and papers takes up enormous time and effort from teachers all over the Arab world. This time and effort can be better utilized in other teaching activities that would elevate the quality of learning in the region. For that reason, the work presented in this paper investigates multiple supervised learning approaches for the domain of Arabic short answer scoring. Moreover, due to the scarcity of Arabic datasets, and to our knowledge, there is only one publicly available dataset for this specific task(AR-ASAG). Thus, we introduce a new publicly available dataset AraScore-Dataset. The model proposed is evaluated on 3 different datasets: 1-AraScore-Dataset 2-AR-ASAG 3-Two translated answer sets taken from the Hewlett Foundation SAS dataset. Based on the investigation, the most effective approach for Arabic automated short answer scoring is proposed, and the performance of using our dataset is compared to other publicly available datasets. The aim is to scale efficient and unbiased content scoring applications across different Arabic educational domains. The results showed that our newly developed response-based system and AraScore-Dataset have achieved state-of-the-art performance on Arabic Automated Short Answer Scoring.