Abstract

We introduce our system that is submitted to the restricted track of the BEA 2019 shared task on grammatical error correction1 (GEC). It is essential to select an appropriate hypothesis sentence from the candidates list generated by the GEC model. A re-ranker can evaluate the naturalness of a corrected sentence using language models trained on large corpora. On the other hand, these language models and language representations do not explicitly take into account the grammatical errors written by learners. Thus, it is not straightforward to utilize language representations trained from a large corpus, such as Bidirectional Encoder Representations from Transformers (BERT), in a form suitable for the learner’s grammatical errors. Therefore, we propose to fine-tune BERT on learner corpora with grammatical errors for re-ranking. The experimental results of the W&I+LOCNESS development dataset demonstrate that re-ranking using BERT can effectively improve the correction performance.

Highlights

  • Grammatical error correction (GEC) systems may be used for language learning to detect and correct grammatical errors in text written by language learners

  • We used First Certificate in English (FCE), Lang-8, NUCLE, and Write & Improve (W&I)+LOCNESS training set as training data and we split the W&I+LOCNESS development set into development and test data by random selection from each Common European Framework of Reference for Languages (CEFR) levels for the transformer and Bidirectional Encoder Representations from Transformers (BERT)

  • By using BERT based on self-attention for re-ranking, which is effective for long distance information, our system became better at solving long distance errors; there is a room for improvement

Read more

Summary

Introduction

Grammatical error correction (GEC) systems may be used for language learning to detect and correct grammatical errors in text written by language learners. GEC has grown in importance over the past few years due to the increasing need for people to learn new languages. GEC has been addressed in the Helping Our Own (HOO) (Dale and Kilgarriff, 2011; Dale et al, 2012) and Conference on Natural Language Learning (CoNLL) (Ng et al, 2013, 2014) shared tasks between 2011 and 2014. There are three main types of neural network models for GEC, namely, recurrent neural networks (Ge et al, 2018), a multi-layer convolutional model based on convolutional neural networks (Chollampatt and Ng, 2018a) and a transformer model based on self-attention (JunczysDowmunt et al, 2018). We follow the best practices to develop our system based on the transformer model, which has achieved better performance for GEC (Zhao et al, 2019)

Methods
Results
Conclusion

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.