Native language subtitling of educational videos: A multimodal analysis with eye tracking, EEG and self‐reports

Abstract

AbstractLanguage assistance becomes an integral component of the teaching–learning process in non‐English– speaking multilingual societies where education takes place predominantly in local languages. Subtitles have been found to be beneficial in facilitating understanding of English language media in such situations. However, when accompanied with subtitles, information‐rich instructional materials can add to the learner's cognitive load as their attention must be split between the content and the subtitle. To evaluate the cognitive and affective impacts of subtitle language on learning, we conducted a multimodal study using eye tracking, electroencephalogram (EEG), self‐report, and pre–post test data of 51 individuals watching a 12‐minute educational video with either no subtitles, English subtitles (L2) or subtitles in their native language (L1). We discovered (a) positive learning gains for groups with subtitles, whether L1 or L2, which suggested positive impact of subtitles on learning outcome; (b) native language subtitling yielded the highest instructional efficiency and supported effective distribution of visual attention between slide content and subtitle; (c) native language subtitles elicited the lowest cognitive load, as evidenced by both EEG measurements and self‐report data; and (d) gaze data highlighted key strategies of high and low performers during interaction with subtitled media. Practitioner notesWhat is already known about this topic Use of subtitles for language acquisition is an extensively researched topic. It is well established that subtitles are beneficial for language acquisition. Use of subtitles for content learning is relatively less understood with no robust finding. What this paper adds Provides a rich literature review about the impact of subtitles, namely, beneficial, detrimental or neutral, for content learning. Offers empirical results demonstrating positive impact of native language subtitles for content learning. Adds to the limited literature of multimodal investigation of learning with subtitles. Implications for practice and/or policy English language e‐learning content designers may consider adding native language subtitles for global adoption of learning products.