Abstract
Translation Environment Tools make translators’ work easier by providing them with term lists, translation memories and machine translation output. Ideally, such tools automatically predict whether it is more effortful to post-edit than to translate from scratch, and determine whether or not to provide translators with machine translation output. Current machine translation quality estimation systems heavily rely on automatic metrics, even though they do not accurately capture actual post-editing effort. In addition, these systems do not take translator experience into account, even though novices’ translation processes are different from those of professional translators. In this paper, we report on the impact of machine translation errors on various types of post-editing effort indicators, for professional translators as well as student translators. We compare the impact of MT quality on a product effort indicator (HTER) with that on various process effort indicators. The translation and post-editing process of student translators and professional translators was logged with a combination of keystroke logging and eye-tracking, and the MT output was analyzed with a fine-grained translation quality assessment approach. We find that most post-editing effort indicators (product as well as process) are influenced by machine translation quality, but that different error types affect different post-editing effort indicators, confirming that a more fine-grained MT quality analysis is needed to correctly estimate actual post-editing effort. Coherence, meaning shifts, and structural issues are shown to be good indicators of post-editing effort. The additional impact of experience on these interactions between MT quality and post-editing effort is smaller than expected.
Highlights
In order to improve Translation Environment Tools, we need to find objective ways to assess postediting effort before presenting machine translation output to the translator
We focus on the following research questions: (i) are all effort indicators influenced by machine translation quality, (ii) is the product effort indicator human-targeted translation error rate (HTER) influenced by different machine translation error types than the process effort indicators, (iii) is there an overlap between the error types that influence the different process effort indicators, and (iv) is the impact of machine translation error types on effort indicators different for student translators than for professional translators?
We looked at the impact of fine- and coarse-grained machine translation quality on different post-editing effort indicators in two different analyses
Summary
In order to improve Translation Environment Tools, we need to find objective ways to assess postediting effort before presenting machine translation output to the translator. Assessing PE Effort via Process Analysis According to Krings (2001), there are three main types of processbased post-editing effort Of these three, the easiest to define and measure is temporal effort: how much time does a posteditor need to turn machine translation output into a high quality translation? On the basis of the above-mentioned research, we expect that a decrease in machine translation quality will lead to an increase in post-editing effort, as expressed by an increase in HTER (Specia and Farzindar, 2010), the number of production units (Koponen, 2012; Popovic et al, 2014), the number of fixations (Doherty and O’Brien, 2009), post-editing time (Koponen et al, 2012), fixation duration (Stymne et al, 2012), pause ratio (O’Brien, 2006), and a decrease in average pause ratio (Lacruz et al, 2012). This means that we expect to see a greater increase in post-editing effort with students than with professional translators when there is an increase in grammatical and lexical issues in the text, and we expect a greater increase in post-editing effort with professional translators than with students when there is an increase in coherence, meaning, or structural issues
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
