Quality Evaluation of C-E Translation of Legal Texts by Mainstream Machine Translation Systems—An Example of DeepL and Metasota

Abstract

Despite significant progress made in machine translation technology and the ongoing efforts in practical and commercial application of neural machine translation systems, their performance in vertical fields remains unsatisfactory. To avoid misunderstandings and excessive expectations of a specific machine translation system, this research selected legal texts as its real data research object. The text translation tasks were accomplished using two popular neural machine translation systems, DeepL and Metasota, both domestically and internationally, and evaluated using internationally recognized BLEU algorithm to reflect their Chinese-to-English translation performance in legal fields. Based on the determined BLEU score, the study adopted an artificial analysis method to analyze the grammatical aspects of the machine translation output, including the accuracy of terminology usage, word order, subject-verb agreement, sentence structure, tense, and voice to enable readers to have a rational understanding of the gap between machine translation and human translation in legal text translation, and objectively assess the application and future development prospects of machine translation in legal text fields. The experimental results indicate that machine translation systems still face challenges in achieving high-quality legal text translations and meeting practical needs, and that further post-translation editing research is needed to improve the accuracy of legal text translation.