Exploring Test Takers’ Cognition in a High-stakes Reading Test : An Eye-tracking Study

Abstract

Learning how test takers derive their answers, referred to as the test-taking process, is essential to test validation. However, research on test takers’ mental process has been impeded by the excessive popularity of quantitative analysis and the limitations of research methods. By using eye-tracking technology, this study investigated Chinese ESL learners’ reading processes in the iBT TOEFL reading section for the purpose of test validation. Ninety Chinese ESL students at a large Midwestern university were invited to an eye-tracking lab and asked to take a 20-minute reading test on a computer screen to which an eye tracker was attached. As in Bax and Weir (2012), this study analyzed only high scorers’ eye movement data (e.g., gaze plots and fixation durations) based on the predetermined criteria. The reading time spent on the intended reading types was also calculated as evidence of validity. The results showed that participants predominantly exercised careful reading skills below the paragraph level, vocabulary questions seemingly failed to elicit inferencing ability, and expeditious reading rarely occurred. Learners did not spend more time reading a text associated with implicit questions compared to a text associated with explicit questions. The implication of the findings for test validity and test development are also discussed.