Abstract
In Grammatical Error Correction (GEC), sequence labeling models enjoy fast inference compared to sequence-to-sequence models; however, inference in sequence labeling GEC models is an iterative process, as sentences are passed to the model for multiple rounds of correction, which exposes the model to sentences with progressively fewer errors at each round. Traditional GEC models learn from sentences with fixed error rates. Coupling this with the iterative correction process causes a mismatch between training and inference that affects final performance. In order to address this mismatch, we propose a GAN-like sequence labeling model, which consists of a grammatical error detector as a discriminator and a grammatical error labeler with Gumbel-Softmax sampling as a generator. By sampling from real error distributions, our errors are more genuine compared to traditional synthesized GEC errors, thus alleviating the aforementioned mismatch and allowing for better training. Our results on several evaluation benchmarks demonstrate that our proposed approach is effective and improves the previous state-of-the-art baseline.
Highlights
Sequence-to-sequence neural solutions (Parnow et al, 2020) have been quite successful in comparison to their statistical counterparts (Sutskever et al, 2014), but these approaches suffer from a couple key problems, which has given rise to sequence labeling approaches for Grammatical Error Correction (GEC) (Omelianchuk et al., 2020)
To combat this exposure bias, we propose a new approach for training a sequence labeling GEC model that draws from GANs (Goodfellow et al, 2014), which consist of a generator that generates increasingly realistic fake inputs and a discriminator that is tasked with differentiating these fake inputs from real inputs
The corrective label set is given as T = {$KEP, $DEL, $APP, $REP} ∪ {$CAS, $MRG, $SPL, $NNUM, $VFORM}, in which the first set consists of the basic text editing transformation operations and the second consists of g-transformations as defined by (Omelianchuk et al, 2020) for GEC1
Summary
Sequence-to-sequence neural solutions (Parnow et al, 2020) have been quite successful in comparison to their statistical counterparts (Sutskever et al, 2014), but these approaches suffer from a couple key problems, which has given rise to sequence labeling approaches for GEC (Omelianchuk et al., 2020) Such approaches task models with generating a list of labels to classify the grammatical errors in a sentence before correcting these errors. The corrective label set is given as T = {$KEP, $DEL, $APP, $REP} ∪ {$CAS, $MRG, $SPL, $NNUM, $VFORM}, in which the first set consists of the basic text editing transformation operations and the second consists of g-transformations as defined by (Omelianchuk et al, 2020) for GEC1 Aligning sentences using these transformations in preprocessing, reduces what would be a sequence generation task that handles unequal source-target lengths to a set of label classification problems.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
