Ecological validity in corpus-based and experimental translation research

Abstract

This study provides insights into the ecological validity of experimental results in translation process research (TPR) by comparing translation products under three different conditions: produced in a translation research lab, in the translators' usual working environment, and included as part of a corpus. The results will test the ecological validity of experimental results in TPR and shed light on some of the discrepancies between findings based on corpus data and lab experiments. Data collected in rigorous translation process experiments outside the translators’ usual working environment may not represent authentic translation behaviour. For example, in the experimental setting, translators will not have access to the tools and resources they usually work with, they will be immediately aware of their participation in a scientific study (which may affect their choices), and the stimuli under investigation will often be artificially constructed. These caveats call into question the meaningfulness of conclusions about translation and the translation process based on target texts produced in such artificial settings. This is furthermore reinforced by the differences found between experimental and corpus-based results. In order to test the comparability of different translation settings, 20 translators were commissioned to translate texts according to their usual workflow, in their usual working environment and at their own pace with access to resources of their preference. The source texts were previously used in experiments (Heilmann et al., 2021; Heilmann et al., 2022) and they contain stimuli that may represent a contrastive or cognitive challenge to the translators. The translators were also asked to provide information about the resources they used and the time and breaks they took to finish the translations. Their translations were compared to the target texts collected in the lab experiments and to translations obtained from a corpus with regard to the linguistic stimuli as well as linguistic characteristics such as average sentence length and lexical density. The results highlight the effect of the experimental setting on the translation product and illuminate the differences between experimental data and corpus data.