Eye movement analysis with hidden Markov models (EMHMM) with co-clustering

Janet H Hsiao,Yueyuan Zheng,Antoni B Chan,Hui Lan

doi:10.3758/s13428-021-01541-5

Janet H Hsiao, Yueyuan Zheng + Show 2 more

Open Access

https://doi.org/10.3758/s13428-021-01541-5

Copy DOI

Abstract

The eye movement analysis with hidden Markov models (EMHMM) method provides quantitative measures of individual differences in eye-movement pattern. However, it is limited to tasks where stimuli have the same feature layout (e.g., faces). Here we proposed to combine EMHMM with the data mining technique co-clustering to discover participant groups with consistent eye-movement patterns across stimuli for tasks involving stimuli with different feature layouts. Through applying this method to eye movements in scene perception, we discovered explorative (switching between the foreground and background information or different regions of interest) and focused (mainly looking at the foreground with less switching) eye-movement patterns among Asian participants. Higher similarity to the explorative pattern predicted better foreground object recognition performance, whereas higher similarity to the focused pattern was associated with better feature integration in the flanker task. These results have important implications for using eye tracking as a window into individual differences in cognitive abilities and styles. Thus, EMHMM with co-clustering provides quantitative assessments on eye-movement patterns across stimuli and tasks. It can be applied to many other real-life visual tasks, making a significant impact on the use of eye tracking to study cognitive behavior across disciplines.

Highlights

Eye-movement behavior has been shown to reflect underlying cognitive processes, and can potentially reveal individual differences in perception styles and cognitive abilities
In scene perception, Chua et al (2005) observed that Westerners made more eye fixations on foreground objects than Asians, whereas Asians looked at the backgrounds more often, and this difference was reflected in their object recognition performance
We used one HMM to summarize a participant’s eye-movement pattern when viewing a particular stimulus, and used coclustering to discover participants sharing similar eyemovement patterns across stimuli. By applying this method to young Asian adults’ eye movements during scene perception, we discovered explorative and focused eye-movement patterns

Summary

Introduction

Eye-movement behavior has been shown to reflect underlying cognitive processes, and can potentially reveal individual differences in perception styles and cognitive abilities. People with cognitive deficits, such as neurodegenerative or psychotic disorders, have been reported to have atypical eyemovement patterns in visual tasks (e.g., Daffner et al, 1992), suggesting eye movements may be used for early detection of cognitive deficits. Traditional approaches for analyzing eyemovement data, such as the use of predefined regions of interests (ROIs) on the stimuli (e.g., Barton et al, 2006) or the use of fixation heat maps/salience maps (e.g., Caldara & Miellet, 2011; Toet, 2011) do not adequately reflect individual differences in either spatial dimension (such as ROI choices) or temporal dimension (such as gaze transition among the ROIs) of eye movements. There have been attempts in using temporal information of eye movements in the analysis, such as using Levenshtein distance or sequence alignment algorithms to quantify and compare similarities of scan paths defined as a sequence of predefined ROIs visited

Objectives

Methods

Results

Conclusion