Abstract

Neural machine translation usually adopts autoregressive models and suffers from exposure bias as well as the consequent error propagation problem. Many previous works have discussed the relationship between error propagation and the accuracy drop (i.e., the left part of the translated sentence is often better than its right part in left-to-right decoding models) problem. In this paper, we conduct a series of analyses to deeply understand this problem and get several interesting findings. (1) The role of error propagation on accuracy drop is overstated in the literature, although it indeed contributes to the accuracy drop problem. (2) Characteristics of a language play a more important role in causing the accuracy drop: the left part of the translation result in a right-branching language (e.g., English) is more likely to be more accurate than its right part, while the right part is more accurate for a left-branching language (e.g., Japanese). Our discoveries are confirmed on different model structures including Transformer and RNN, and in other sequence generation tasks such as text summarization.

Highlights

  • Neural machine translation (NMT) has attracted much research attention in recent years (Bahdanau et al, 2014; Shen et al, 2018; Song et al, 2018; Xia et al, 2018; He et al, 2016; Wu et al, 2017, 2018)

  • If error propagation is the main cause of accuracy drop, the right part words in the translation results generated by right-toleft NMT models should be more accurate than the left part words

  • We studied the problem of accuracy drop between the left half and the right half of the results generated by neural machine translation models

Read more

Summary

Introduction

Neural machine translation (NMT) has attracted much research attention in recent years (Bahdanau et al, 2014; Shen et al, 2018; Song et al, 2018; Xia et al, 2018; He et al, 2016; Wu et al, 2017, 2018). If error propagation is the main cause of accuracy drop, the right part words in the translation results generated by right-toleft NMT models should be more accurate than the left part words. We observe the opposite phenomenon that the accuracy of the right part words of the translated sentences in both leftto-right and right-to-left models is lower than that of the left part, which contradicts with error propagation. This shows that error propagation alone cannot well explain the accuracy drop and even.

Exposure Bias and Error Propagation
Error Propagation is Not the Only Cause
The Influence of Error Propagation
Language Branching Matters
Correlation between Language Branching and Accuracy Drop
N-gram Statistics
Dependency Statistics
Extended Analyses and Discussions
More Languages on Left-Branching
Other Model Structures
Other Sequence Generation Tasks
Conclusion
Full Text
Paper version not known

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.