Abstract
This study aims to analyse how translation experts from the German department of the European Commission’s Directorate-General for Translation (DGT) identify and correct different error categories in neural machine translated texts (NMT) and their post-edited versions (NMTPE). The term translation expert encompasses translator, post-editor as well as revisor. Even though we focus on neural machine-translated segments, translator and post-editor are used synonymously because of the combined workflow using CAT-Tools as well as machine translation. Only the distinction between post-editor, which refers to a DGT translation expert correcting the neural machine translation output, and revisor, which refers to a DGT translation expert correcting the post-edited version of the neural machine translation output, is important and made clear whenever relevant. Using an automatic error annotation tool and the more fine-grained manual error annotation framework to identify characteristic error categories in the DGT texts, a corpus analysis revealed that quality assurance measures by post-editors and revisors of the DGT are most often necessary for lexical errors. More specifically, the corpus analysis showed that, if post-editors correct mistranslations, terminology or stylistic errors in an NMT sentence, revisors are likely to correct the same error type in the same post-edited sentence, suggesting that the DGT experts were being primed by the NMT output. Subsequently, we designed a controlled eye-tracking and key-logging experiment to compare participants’ eye movements for test sentences containing the three identified error categories (mistranslations, terminology or stylistic errors) and for control sentences without errors. We examined the three error types’ effect on early (first fixation durations, first pass durations) and late eye movement measures (e.g., total reading time and regression path durations). Linear mixed-effects regression models predict what kind of behaviour of the DGT experts is associated with the correction of different error types during the post-editing process.
Highlights
Background and Related ResearchWith “28 member states, 500 million citizens, 3 alphabets and 24 official languages” ([1], p. 1)—i.e., 552 possible language combinations—translation quality plays a key role in the European Union (EU)to bridge linguistic gaps and ensure successful communication between its citizens and institutions.Approximately 1600 in-house translators make the European Commissions’ Directorate- General for Translation (DGT) probably the world’s largest translation service: In 2016 alone, the Directorate-General for Translation (DGT) received73,000 translation requests, produced 2.2 million pages, and outsourced more than 650,000 pages to freelancers ([2], p. 43)
This paper reports on two consecutive studies: (1) a published corpus analysis of PE changes and was not significant, the post-hoc analysis revealed the only dissociation between Error Type categories their revisions performed by DGT professionals to explore the quality of neural machine translated texts (NMT) outputs and neural machine translation post-editing (NMTPE) of for corrected items in our data
Previous studies on NMT quality reported omissions and mistranslations/incorrect lexis as the correction of Misterm and Function words (Func) errors which are recognised during the early processes has an problematic NMT error categories, e.g., [13,17,18,19], and a similar picture is painted in our analysis effect on later processes: large early effects have an effect on the later measure
Summary
Background and Related ResearchWith “28 member states, 500 million citizens, 3 alphabets and 24 official languages” ([1], p. 1)—i.e., 552 possible language combinations—translation quality plays a key role in the European Union (EU)to bridge linguistic gaps and ensure successful communication between its citizens and institutions.Approximately 1600 in-house translators make the European Commissions’ Directorate- General for Translation (DGT) probably the world’s largest translation service: In 2016 alone, the DGT received73,000 translation requests, produced 2.2 million pages, and outsourced more than 650,000 pages to freelancers ([2], p. 43). Machine translation users choose between three possibilities according to the purpose, life cycle, and (the size of) the target group of the text: (1) not correcting the MT output at all, (2) applying light corrections to ensure the understandability of the text or (3) applying a full post-editing which requires linguistic as well as stylistic changes to achieve a text of publishable quality that reads well and does not contain any errors. In the case of the DGT and its extremely high-quality standards, only full post-editing is applied and even the post-edits are revised by a second person This is partly due to the DGT workflow which combines the use of translation memory systems (TMS) and machine translations (MT) in the same working environment.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.