Short-Answer Grading for German: Addressing the Challenges

Abstract

AbstractShort-Answer Grading (SAG) is a time-consuming task for teachers that automated SAG models have long promised to make easier. However, there are three challenges for their broad-scale adoption: A technical challenge regarding the need for high-quality models, which is exacerbated for languages with fewer resources than English; a usability challenge in adapting high-quality research prototypes to the needs of non-expert users, and a trust challenge in communicating the abilities and limitations of the tools. We propose to meet the technical challenge for German with a robust Transformer-based SAG model. We address the usability challenge with an easy-to-use graphical user interface for the SAG model, and the trust challenge with a workflow that allows teachers to evaluate the model on their own data, to choose on the basis of this evaluation which model predictions to trust, and in consequence to stay in control of grading their students while saving grading effort.