Abstract

Grammatical error correction (GEC) systems deployed in language learning environments are expected to accurately correct errors in learners’ writing. However, in practice, they often produce spurious corrections and fail to correct many errors, thereby misleading learners. This necessitates the estimation of the quality of output sentences produced by GEC systems so that instructors can selectively intervene and re-correct the sentences which are poorly corrected by the system and ensure that learners get accurate feedback. We propose the first neural approach to automatic quality estimation of GEC output sentences that does not employ any hand-crafted features. Our system is trained in a supervised manner on learner sentences and corresponding GEC system outputs with quality score labels computed using human-annotated references. Our neural quality estimation models for GEC show significant improvements over a strong feature-based baseline. We also show that a state-of-the-art GEC system can be improved when quality scores are used as features for re-ranking the N-best candidates.

Highlights

  • The task of automatically correcting various kinds of errors in written text, termed as grammatical error correction (GEC), is primarily aimed at assisting language learning and providing corrective feedback to second-language learners

  • We propose a neural approach to automatic quality estimation of GEC output

  • The contributions of this paper are: (1) we propose the first supervised approach to quality estimation (QE) of GEC system outputs, (2) we present neural QE models that outperform a strong feature-based baseline for estimating post-editing effort and an automatic GEC evaluation metric, (3) we propose new convolutional neural architectures for QE that can be potentially utilized for QE tasks in other language applications, and (4) we show that the performance of a state-of-the-art GEC system can be improved by adding QE scores as features in re-ranking the N-best candidates

Read more

Summary

Introduction

The task of automatically correcting various kinds of errors in written text, termed as grammatical error correction (GEC), is primarily aimed at assisting language learning and providing corrective feedback to second-language learners. Having quality estimates for the system’s output sentences can help instructors to decide whether to check and fix the system’s corrections (for higher quality corrections) or to ignore the system’s corrections altogether and recorrect the original learner-written sentences (for lower quality ones) instead. This can significantly make the process of post-editing easier and faster. Such quality estimates can directly help end users — the language learners — to decide on the extent to which the system’s corrections can be trusted and seek assistance from instructors and other sources to get better corrective feedback if needed. We propose a neural approach to automatic quality estimation of GEC output

Results
Discussion
Conclusion
Full Text
Paper version not known

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.