Eye movement patterns differ while watching captioned videos of second language vs. mathematics lessons

Abstract

BackgroundExtant eye-tracking studies suggest that foreign-language learners tend to read the native language captions while watching foreign-language videos. However, it remains unclear how the captions affect the learners' eye movements when watching Math videos. PurposeWhile watching teaching videos, we seek to determine how the lesson type (English or Math), cognitive load (high or low), and caption type (meaningful, no captions, or meaningless) affect the dwell times and fixation counts on the captions. MethodsOne hundred and eighty undergraduate students were randomly and equally assigned to six (2 lesson type × 3 caption type) conditions. Each participant watched two short teaching videos (one low load and one high load). After watching each video, a comprehension test and three self-reported items (fatigue, effort, and difficulty) regarding this particular video were given. ResultsWe reported more dwell times and fixation counts on the meaningful captions, compared to the meaningless captions and no captions. In the high-load condition, viewers watching an English lesson relied more on the meaningful captions than they did when watching a Math lesson. In the low-load condition, the dwell times and fixation counts on the captions were similar between the English and Math lessons. Finally, the captions did not affect the comprehension test performances after ruling out individual differences in the prior performances of English and Math. ConclusionsEnglish language learning may rely more on the captions than is the case in learning Math. This study provides the direction for designing multimedia teaching materials in the current trend of multimedia teaching.