Neural Machine Translation with Dynamic Selection Network

Abstract

Neural Machine Translation (NMT) has made remarkable progress in recent years, and the attention mechanism has become the dominant approach with the state-of-the-art records in many language pairs. However, the attention based NMT models have two shortcomings: First, due to the lack of effective control over the influence from source and target contexts, conventional NMT tends to yield fluent but inadequate translations. Second, with the extensive application of NMT model in empirical research, its long-term weaknesses in dealing with scarce and extra vocabulary have become increasingly prominent. To address this problem, we employ our dynamic selection network which consists of context gate that dynamically controls the amount of information flowing from the source and target contexts, and dynamic vocabulary that additionally considers copying words directly from the source. Experiments are conducted on three machine translation tasks, English-to-German IWLST 2014, English-to-Vietnamese IWLST 2015 and Turkish-to-English WMT 2017. Experiments show that the proposed model outperforms the traditional NMT model with a large margin.