Chinese Grammatical Error Correction Research Articles

Chinese Grammatical Error Correction Using Pre-trained Models and Pseudo Data

In recent studies, pre-trained models and pseudo data have been key factors in improving the performance of the English grammatical error correction (GEC) task. However, few studies have examined the role of pre-trained models and pseudo data in the Chinese GEC task. Therefore, we develop Chinese GEC models based on three pre-trained models: Chinese BERT, Chinese T5, and Chinese BART, and then incorporate these models with pseudo data to determine the best configuration for the Chinese GEC task. On the natural language processing and Chinese computing (NLPCC) 2018 GEC shared task test set, all our single models outperform the ensemble models developed by the top team of the shared task. Chinese BART achieves an F score of 37.15, which is a state-of-the-art result. We then combine our Chinese GEC models with three kinds of pseudo data: Lang-8 (MaskGEC), Wiki (MaskGEC), and Wiki (Backtranslation). We find that most models can benefit from pseudo data, and BART+Lang-8 (MaskGEC) is the ideal setting in terms of accuracy and training efficiency. The experimental results demonstrate the effectiveness of the pre-trained models and pseudo data on the Chinese GEC task and provide an easily reproducible and adaptable baseline for future works. Finally, we annotate the error types of the development data; the results show that word-level errors dominate all error types, and word selection errors must be addressed even when using pre-trained models and pseudo data. Our codes are available at https://github.com/wang136906578/BERT-encoder-ChineseGEC .

A Chinese Grammatical Error Correction Method Based on Iterative Training and Sequence Tagging

Chinese grammatical error correction (GEC) is under continuous development and improvement, and this is a challenging task in the field of natural language processing due to the high complexity and flexibility of Chinese grammar. Nowadays, the iterative sequence tagging approach is widely applied to Chinese GEC tasks because it has a faster inference speed than sequence generation approaches. However, the training phase of the iterative sequence tagging approach uses sentences for only one round, while the inference phase is an iterative process. This makes the model focus only on the current sentence’s current error correction results rather than considering the results after multiple rounds of correction. In order to address this problem of mismatch between the training and inference processes, we propose a Chinese GEC method based on iterative training and sequence tagging (CGEC-IT). First, in the iterative training phase, we dynamically generate the target tags for each round by using the final target sentences and the input sentences of the current round. The final loss is the average of each round’s loss. Next, by adding conditional random fields for sequence labeling, we ensure that the model pays more attention to the overall labeling results. In addition, we use the focal loss to solve the problem of category imbalance caused by the fact that most words in text error correction do not need error correction. Furthermore, the experiments on NLPCC 2018 Task 2 show that our method outperforms prior work by up to 2% on the F0.5 score, which verifies the efficiency of iterative training on the Chinese GEC model.

MaskGEC: Improving Neural Grammatical Error Correction via Dynamic Masking

Grammatical error correction (GEC) is a promising natural language processing (NLP) application, whose goal is to change the sentences with grammatical errors into the correct ones. Neural machine translation (NMT) approaches have been widely applied to this translation-like task. However, such methods need a fairly large parallel corpus of error-annotated sentence pairs, which is not easy to get especially in the field of Chinese grammatical error correction. In this paper, we propose a simple yet effective method to improve the NMT-based GEC models by dynamic masking. By adding random masks to the original source sentences dynamically in the training procedure, more diverse instances of error-corrected sentence pairs are generated to enhance the generalization ability of the grammatical error correction model without additional data. The experiments on NLPCC 2018 Task 2 show that our MaskGEC model improves the performance of the neural GEC models. Besides, our single model for Chinese GEC outperforms the current state-of-the-art ensemble system in NLPCC 2018 Task 2 without any extra knowledge.

Chinese Grammatical Error Correction Based on Convolutional Sequence to Sequence Model

Chinese grammatical error correction (CGEC) is practically useful for learners of Chinese as a second language, but it is a rather challenging task due to the complex and flexible nature of Chinese language so that existing methods for English cannot be directly applied. In this paper, we introduce a convolutional sequence to sequence model into the CGEC task for the first time, since many Chinese grammatical errors are concentrated between three and four words and convolutional neural network can better capture the local context. A convolution-based model can obtain the representations of the context by fixed size kernel. By stacking convolution layers, long-term dependences can be obtained. We also propose two optimization methods, shared embedding and policy gradient, to optimize the convolutional sequence to sequence model through sharing parameters and reconstructing loss function. Besides, we collate the existing Chinese grammatical correction corpus in detail. The results show that the models we proposed two different optimization methods both achieve large improvement compared with the natural machine translation model based on a recurrent neural network.

A Two-Stage Model for Chinese Grammatical Error Correction

Chinese grammatical error correction (GEC) is more challenging than English GEC due to its language characteristics. In this paper, a two-stage model was proposed to solve the Chinese GEC problem. The model consists of two components: a spelling check model and a GEC model. The spelling check model based on language model focuses on correcting spelling errors, while the GEC model based on neural sequence-to-sequence (seq2seq) model focuses on correcting grammatical errors. In addition, two generative methods allow the seq2seq model to correct an erroneous sentence incrementally with repeated inference steps. Furthermore, only one seq2seq model is used for grammatical correction rather than ensemble multiple models, which greatly speeds up the generation of final results and saves computing resources. The two-stage model achieves 31.01 F 0.5 on NLPCC 2018 test set, significantly outperforms all prior approaches on this task.