A teacher-student model with the bilingual word embedding for zero-shot machine translation

Abstract

Neural machine translation has achieved state-of-art performance with sufficient data, but it still suffers from the data scarcity problem for low-resource language pairs. Teacher-student model that transfers knowledge from a pivot→target model to a source→target model with the source↔pivot parallel data has shown its effectiveness. In this paper, we improve the model with the bilingual word embedding and transformer architecture. Experiments are carried out on three commonly used translation datasets and the result shows the improvement over a baseline teacher-student model.