Effective Use of Target-side Context for Neural Machine Translation

Abstract

Through the progress made in a sentence-level neural machine translation (NMT), a context- aware NMT has been rapidly developed to exploit previous sentences as context. Recent work in the context-aware NMT incorporates source- or target-side contexts. In contrast to the source- side context, the target-side context causes a gap between training that utilizes a ground truth sentence and inference using a machine-translated sentence as context. This gap leads to transla- tion quality deteriorating because the translation model is trained with only the ground truth data that cannot be used in the inference. In this paper, we propose sampling both the ground truth and the machine-translated previous sentences of the target-side for the context-aware NMT. The proposed method can make the translation model robust against mistakes and biases made at the inference. Models using our proposed approach show improvements over models using the previous approaches in English ↔ Japanese and English ↔ German translation tasks.