Automatic subtitles increase accuracy and decrease cognitive load in simultaneous interpreting

Abstract

Abstract This study examines the effect of real-time subtitles generated by automatic speech recognition (ASR) technology on interpreting accuracy and interpreters’ cognitive load. Multiple measurements — including interpreting accuracy, the NASA-TLX for subjective ratings of cognitive load, eye-tracking and theta power as indicated by EEG recordings — were applied. Twenty-three professional simultaneous interpreters worked with a video recording of a speech presented in five conditions: a baseline without subtitles and then with subtitles of varying levels of precision (100%, 95%, 90% and 80%). The results reveal that the presence of subtitles significantly improved interpreting accuracy, with a suggested optimal precision rate of 90% or higher. The interpreters looked more at the subtitles, regardless of their level of precision, than the speaker. Contrary to our predictions, the presence of subtitles decreased, rather than increased, the cognitive load (although this outcome was shown by the EEG data only and not by the self-reported data). We conclude that the cognitive cost of processing subtitles as an additional information channel is offset by the cognitive gain achieved through visual prompting. The study highlights a complex effect of subtitles on interpreting, with such factors as subtitle presence and precision modulating the interpreters’ cognitive load in such a workflow.