An evaluation of Google Translate for Sanskrit to English translation via sentiment and semantic analysis

Abstract

Google Translate has been prominent for language translation; however, limited work has been done in evaluating the quality of translation when compared to expert translators. Sanskrit, one of the oldest written languages in the world was added to the Google Translate engine in 2022. Sanskrit is known as the mother of North Indian languages such as Hindi, Punjabi, and Bengali. Sanskrit has been used for composing sacred Hindu texts such as the Bhagavad Gita and the Upanishads. In this study, we present a framework that evaluates Google Translate for Sanskrit using the Bhagavad Gita. We first published a translation of the Bhagavad Gita in Sanskrit using Google Translate and then compared it with selected prominent translations by experts. Our framework features a BERT-based language model that implements sentiment and semantic analysis. The results indicate that there is a low level of similarity between the Bhagavad Gita by Google Translate and expert translators in terms of sentiment and semantic analyses. We found major inconsistencies in the translation of philosophical terms and metaphors. We further implemented a qualitative evaluation and found that Google Translate was unsuitable for the translation of certain Sanskrit words and phrases due to the poetic nature, contextual significance, metaphor and imagery. The mistranslations are not surprising since the Bhagavad Gita is known as a difficult text not only to translate but also to interpret since it relies on contextual, philosophical and historical information. It is difficult to distinguish different names and philosophical concepts such as karma and dharma without knowing the context and having a background of Hindu philosophy. Our framework lays the foundation for the automatic evaluation of other languages that can be translated by Google Translate.