Towards Learning to Read Like Humans

Abstract

Previous NLP works were successful in using human gaze behavior to improve task performance on their data sets. However, having to repeatedly collect gaze data for every new data set is impractical. Thus, there is a need for a method that will allow the utilization of available gaze data without the overhead. Our work presents a novel attempt to directly predict gaze features for each word with respect to its sentence. We take on a multi-corpus and task-agnostic approach: using four different eye-tracking data sets, regardless of reading task, material, and experiment design. Using only the word sequence as input to a 2-layer bidirectional LSTM, we achieve \(R^2\) scores in the range of 76.80 to 95.59 for the following five gaze features: Number of Fixations (NFIX), First Fixation Duration (FFD), Total Reading Time (TRT), Go-Past Time (GPT), and Gaze Duration (GD). In addition, we use the model to predict gaze features for words in seen and unseen sentences in an attempt to improve performance in two NLP tasks. This led to a slight increase in performance, supporting the potential of such a model. Our paper presents an exploratory experiment into this methodology.