Gutenberg Goes Neural: Comparing Features of Dutch Human Translations with Raw Neural Machine Translation Outputs in a Corpus of English Literary Classics

Rebecca Webster,Lieve Macken,Margot Fonteyne,Joke Daems,Arda Tezcan

doi:10.3390/informatics7030032

Abstract

Due to the growing success of neural machine translation (NMT), many have started to question its applicability within the field of literary translation. In order to grasp the possibilities of NMT, we studied the output of the neural machine system of Google Translate (GNMT) and DeepL when applied to four classic novels translated from English into Dutch. The quality of the NMT systems is discussed by focusing on manual annotations, and we also employed various metrics in order to get an insight into lexical richness, local cohesion, syntactic, and stylistic difference. Firstly, we discovered that a large proportion of the translated sentences contained errors. We also observed a lower level of lexical richness and local cohesion in the NMTs compared to the human translations. In addition, NMTs are more likely to follow the syntactic structure of a source sentence, whereas human translations can differ. Lastly, the human translations deviate from the machine translations in style.

Highlights

Machine translation (MT) is widely accepted in everyday translation situations, using it to translate literary texts can still cause some hesitation [1]
Aside from the types of errors in the neural machine translation (NMT), one could wonder how these machine generated translations compare to the actual human translations: Do they achieve the same level of lexical richness and cohesion, the same syntactic structures, or even the same style? Our results indicate that lexical richness decreases from human translation to machine translation, indicating a certain homogenization of the lexicon used by the NMT systems
It would appear that literary human translations are lexically richer, more cohesive, and syntactically more diverse than their literary NMTs

Summary

Introduction

Machine translation (MT) is widely accepted in everyday translation situations, using it to translate literary texts can still cause some hesitation [1]. With the continuous advancements made in the field of neural machine translation (NMT), some researchers have tried to remove this stigma and provide more information on the applicability of NMT to literary texts [1,3,4,5]. The aim of this article is to assess the performance of generic NMT systems on literary texts, by providing insights into the quality of the NMT, but by examining more in-depth features such as lexical richness, local cohesion, syntactic and stylistic difference. In order to discuss the performance of NMT on literary texts, we completed two types of analyses. The first concerns an error analysis, which is the more traditional approach of assessing translation quality

Objectives

Methods

Results

Discussion

Conclusion