How do machine translators measure up to human literary translators in stylometric tests?

Abstract

Abstract This study aims to test the classification of literary translations by human and machine translators in a bid to explore stylistic distinctions between the two groups and among machine translators and their evolution over 1 year between 2019 and 2020. For this, the study takes a stylometric approach by employing two statistical methods popularly used in authorship attribution—support vector machine (SVM) and principal component analysis (PCA), which are applied to analyzing three types of features—1-grams, 2-grams, and 3 grams. The results show that over the researched period, the three machine translators investigated (Google, Bing, and Papago) moved in the direction of converging in style while advancing toward human translators. The distance gained by them on human translators, however, was not significant enough to challenge the clear-cut divide between the two groups. The PCA tests additionally revealed some characteristics of the machine translator that might be responsible for the stylistic distinction between them and their human counterparts including their tendency of overusing basic standard language.