A Comparison of Transformer, Recurrent Neural Networks and SMT in Tamil to Sinhala MT

Abstract

Neural Machine Translation (NMT) is currently the most promising approach for machine translation. The attention mechanism is a successful technique in modern Natural Language Processing (NLP), especially in tasks like machine translation. The recently proposed network architecture of the Transformer is based entirely on attention mechanisms and achieves a new state of the art results in neural machine translation, outperforming other sequence-to-sequence models. Although it is successful in a resource-rich setting, its applicability for low-resource language pairs is still debatable. Additionally when the language pair is morphologically rich and also when the corpora is multi-domain, the lack of a large parallel corpus becomes a significant barrier. In this study, we explore different NMT algorithms – Long Short Term Memory (LSTM) and Transformer based NMT, to translate the Tamil to Sinhala language pair. Where we clearly see transformer outperforms LSTM by 2.43 BLEU score for Tamil to Sinhala direction. And this work provides a preliminary comparison of statistical machine translation (SMT) and Neural Machine Translation (NMT) for Tamil to Sinhala in the open domain context.