Abstract
Neural machine translation usually adopts autoregressive models and suffers from exposure bias as well as the consequent error propagation problem. Many previous works have discussed the relationship between error propagation and the accuracy drop (i.e., the left part of the translated sentence is often better than its right part in left-to-right decoding models) problem. In this paper, we conduct a series of analyses to deeply understand this problem and get several interesting findings. (1) The role of error propagation on accuracy drop is overstated in the literature, although it indeed contributes to the accuracy drop problem. (2) Characteristics of a language play a more important role in causing the accuracy drop: the left part of the translation result in a right-branching language (e.g., English) is more likely to be more accurate than its right part, while the right part is more accurate for a left-branching language (e.g., Japanese). Our discoveries are confirmed on different model structures including Transformer and RNN, and in other sequence generation tasks such as text summarization.
Highlights
Neural machine translation (NMT) has attracted much research attention in recent years (Bahdanau et al, 2014; Shen et al, 2018; Song et al, 2018; Xia et al, 2018; He et al, 2016; Wu et al, 2017, 2018)
If error propagation is the main cause of accuracy drop, the right part words in the translation results generated by right-toleft NMT models should be more accurate than the left part words
We studied the problem of accuracy drop between the left half and the right half of the results generated by neural machine translation models
Summary
Neural machine translation (NMT) has attracted much research attention in recent years (Bahdanau et al, 2014; Shen et al, 2018; Song et al, 2018; Xia et al, 2018; He et al, 2016; Wu et al, 2017, 2018). If error propagation is the main cause of accuracy drop, the right part words in the translation results generated by right-toleft NMT models should be more accurate than the left part words. We observe the opposite phenomenon that the accuracy of the right part words of the translated sentences in both leftto-right and right-to-left models is lower than that of the left part, which contradicts with error propagation. This shows that error propagation alone cannot well explain the accuracy drop and even.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.