Recursive annotations for attention-based neural machine translation

Abstract

The last few years have witnessed the success of attention-based Neural Machine Translation (NMT), and many of variant models have been used to improve the performance. Most of the proposed attention-based NMT models encode the source sentence into a sequence of annotations which are kept fixed for the following steps. In this paper, we conjecture that the use of fixed annotations is the bottleneck in improving the performance ofconventional attention-based NMT. To tackle this shortcoming, we propose a novel model for attention-based NMT, which is intended to update the source annotations recursively when generating the target word at each time step. Experimental results show that the proposed approach achieves significant performance improvement over multiple test sets.