Grammatical error simulation for computer-assisted language learning

Abstract

This paper presents an automated method to generate realistic grammatical errors that can perform crucial functions for advanced technologies in computer-assisted language learning (CALL), including generating corrective feedback in dialog-based CALL (DB-CALL) systems, simulating a language learner to optimize tutoring strategies, and generating context-dependent grammar quizzes as educational materials. The goal of this study is to make grammatical errors generated by automatic simulators more realistic. To generate realistic errors, expert knowledge of language learners’ error characteristics was imported into a statistical modeling system that uses Markov logic, which provides a theoretically sound way to encode knowledge into probabilistic first-order logic. We learned the weights of first-order formulas from a learner corpus. The improved quality of the proposed method was demonstrated through comparative experiments using automatic evaluations (precision and recall rate and Kullback–Leibler divergence between error distributions) and human assessments. The proposed method increased precision by 6% and recall by 8.33% averaged across all proficiency levels. It also exhibited a relative improvement of 37.5% in the average Kullback–Leibler divergence. Judgment by human evaluators showed that the proposed method increased the average scores in two different evaluation tasks by 7 and by 0.411. Finally, we present a measure of labor savings to help predict the time and cost associated with this method for those who plan to exploit grammatical error simulation for their applications. The results indicate that using the proposed method could reduce the grammatical error generation time by 59% in average.