Quality assessment of machine-translated post-edited subtitles: an analysis of Brazilian translators’ perceptions

Abstract

This study focuses on the translation product in the form of subtitles and, in particular, investigates the perceived quality of machine-translated post-edited interlingual subtitles. Basing this study on data collected from Brazilian professional translators, we analyse whether the use of machine translation has an impact on the perceived quality and acceptability of interlingual subtitles. We also examine those technical parameters and linguistic aspects that the study participants considered troublesome and whether translators would report any evidence that the subtitles were machine-translated. Sixty-eight Brazilian translators volunteered to participate in this study. They completed a questionnaire, watched a movie trailer with post-edited subtitles, and then assessed the quality of the subtitles by rating them using a Likert-type scale. Finally, the participants responded to a written verbal protocol. Our analysis relies on the qualitative and quantitative data provided by the participants. The qualitative and quantitative data from the protocol answers were triangulated with the quantitative results from the Likert-type scale. IBM SPSS Statistics was used for statistical analysis. Our findings show that 28.1% of the translators assessed the post-edited subtitles as very satisfactory whereas 48.4% judged them as satisfactory. These results seem to be an indication of the acceptability of post-editing machine-translated subtitles as they find support in the written verbal protocol answers. Regarding the technical parameters, 43% of the participants reported no issues whereas 22.8% commented on the font colour and the font size. When asked about any linguistic aspects that could have disturbed the viewing experience, 64.1% of them reported that no linguistic issues were evident in the subtitles. A small percentage of the participants reported some linguistic problems, such as a lack of accuracy, the use of literal translation, and unnatural subtitles. Despite this apparent linguistic evidence of machine translation, no participant explicitly affirmed that the subtitles were machine-translated and post-edited.