Temporal dynamics of eye‐tracking and EEG during reading and relevance decisions

Abstract

Assessment of text relevance is an important aspect of human–information interaction. For many search sessions it is essential to achieving the task goal. This work investigates text relevance decision dynamics in a question‐answering task by direct measurement of eye movement using eye‐tracking and brain activity using electroencephalography EEG. The EEG measurements are correlated with the user's goal‐directed attention allocation revealed by their eye movements. In a within‐subject lab experiment (N = 24), participants read short news stories of varied relevance. Eye movement and EEG features were calculated in three epochs of reading each news story (early, middle, final) and for periods where relevant words were read. Perceived relevance classification models were learned for each epoch. The results show reading epochs where relevant words were processed could be distinguished from other epochs. The classification models show increasing divergence in processing relevant vs. irrelevant documents after the initial epoch. This suggests differences in cognitive processes used to assess texts of varied relevance levels and provides evidence for the potential to detect these differences in information search sessions using eye tracking and EEG.