Analysis Of Eye-tracking Data Research Articles

GazeEnViz4D: 4-D Gaze-in-Environment Visualization Pipeline

Eye-tracking data visualization and analysis is often performed in three dimensions (x,y,t). It involves overlaying the gaze point (x,y) on scene-camera images and creating a pipeline to process the gaze-overlaid spatio-temporal data. In this project, we present a pipeline (called GazeEnViz4D) to extract 3D data from raw 2D eye-tracking data, and we have developed a custom ENvironment VIsualiZer (EnViz4D§) to allow researchers to visualize the pipeline-processed data in four dimensions (x,y,z,t) for extensive, interactive analysis. GazeEnViz4D consists of creating a 3D point cloud of the environment, calculating the observer motion, locating the 2D gaze obtained from the eye-tracker in the 3D model, and visualizing the data over time using EnViz4D. EnViz4D allows a researcher to zoom in the environment at any instance, pause or play the 3D data, speed up or slow down, forward or reverse, essentially recreating the data collection episode in four dimensions from an arbitrary modifiable viewpoint.

Machine learning analysis of speech and eye tracking data to distinguish Alzheimer's clinic patients from healthy controls

AbstractBackgroundClinical trials of disease‐modifying therapies for Alzheimer’s disease (AD) are increasingly focused on recruiting individuals with preclinical or early‐stage disease. Artificial intelligence may help in enriching clinical trial populations with high‐risk individuals. We analyzed prospectively‐collected speech and eye‐tracking data to distinguish individuals with mild‐moderate AD, mild cognitive impairment (MCI), and subjective memory complaints (SMC) from age and sex‐matched healthy volunteers.MethodIndividuals with known clinical diagnoses of AD, MCI, and SMC from a specialty clinic, and healthy controls from the community, were prospectively recruited. Participants described the “Cookie Theft Picture” from the Boston Aphasia Battery. Their speech was recorded and manually transcribed and eye movements were assessed using an infrared eye‐tracker. Data underwent feature extraction for language‐related features including lexical and acoustic parameters (from transcripts and speech), and fixation and saccades (from eye‐tracking). Additionally, for language and eye‐tracking, features capturing spatial neglect are explored following the approach in [1]. Separate predictive models were examined using logistic regression (LR), K‐Nearest Neighbours (KNN), and random forests (RF).ResultRecruitment and analysis are ongoing. Data from 34 clinic patients (9% SMC, 35% MCI, 56% mild‐moderate AD, mean age 7310, 44% female) and 39 controls (mean age 6811, 79% female) is reported. Best speech‐based models distinguished patients from controls with an Area Under the ROC Curve (AUC) of 79% (95% CI 0.70‐0.87). Eye‐tracking‐based models reached distinguished patients from controls with an AUC of 71% (95% CI 0.59‐0.82).ConclusionMachine‐learning mediated analysis of speech and eye‐tracking data achieved promising classification performance with the current cohort. Both increasing the size of the corpus with ongoing recruitment, and combining demographic and clinical data alongside multimodal feature fusion, may help to improve classification.

Searching for Street Parking: Effects on Driver Vehicle Control, Workload, Physiology, and Glances.

Urban areas that allow street parking exhibit a heightened crash risk that is often attributed to factors such as reduced road width, decreased visibility, and interruptions to traffic flow. No previous on-road studies have investigated how the demands of searching for parking affect driving performance, physiology, and visual attention allocation. We are interested in these effects on the driver and their possible influence on the safety of the environment. While simulator studies offer several benefits, the physical, mental and social pressures incurred by searching for parking in an urban streetscape cannot be emulated in a simulator. We conducted an on-road instrumented vehicle study with 28 participants driving in downtown Toronto, Canada to explore the effect of searching for street parking on drivers. During the experiment, participants drove two routes in a counterbalanced order: one route with a parking search task, and the other route as a baseline. Speed and lane position were measured via vehicle instrumentation, heart rate and galvanic skin response were measured through physiological sensors, and gaze position was collected through a head-mounted eye-tracker. Participants completed the NASA Task Load Index after each route. It was found that while searching for parking, participants drove slower and closer to the curb, and perceived higher workload. While there were no statistically significant effects in physiological measures, there was a rise in heart rate approaching statistical significance. A detailed analysis of eye-tracking data revealed a clear change in glance behavior while searching for parking, with an increase in long off-road glances (>2 s) and decrease in shorter off-road glances (<1.6 s). Some exhibited behaviors (e.g., slowing down) may be seen to compensate for the potentially negative effects of increased demands associated with parking search, while others (e.g., increase in long off-road glances) have the potential to increase crash risk. This study acts as an important first step in revealing changes in driving performance, physiology and glance behavior brought on by searching for parking in a real-world urban environment.

The impact of engineering information format on task performance: Gaze scanning pattern analysis

The emergence of new visualization technologies such as Virtual Reality (VR) and Augmented Reality (AR) had been widely implemented in the Architecture, Engineering, and Construction (AEC) industry. Although cumulative evidence pointed out a positive impact of these visualization technologies on construction task performance, there is still an obvious disagreement on the benefits or implications of these new visualization technologies, due to the lack of understanding of the mechanisms in which the visualization affects cognitive processes related to information processing. To obtain more evidence, this paper presents a human-subject experiment (n = 90) to investigate the impact of information format on the performance of an industrial pipeline maintenance task. The investigation centers around how different engineering information formats affect the attention patterns as a potential explanation for the changes in performance. A between-group experiment design was used where the participants were randomly assigned to one of the three groups (2D group, 3D group, and VR group) depending on what type of information was given to review the pipe operation instruction. After the review session, the participants were asked to perform the operation task in the virtual environment based on their memory. The results showed that the 3D and VR groups outperformed the 2D group in task performance. The analysis of eye-tracking data further indicated that the information format significantly changed the gaze scanning pattern when participants were reviewing the operational instructions. We also found that the task performance was correlated with eye-tracking features including gaze movement and pupil dilation. Our findings provided more evidence about the mechanisms in which new visualization technologies affect the attention patterns, helped resolve the current disagreement within the literature. In addition, a prediction model was proposed to use eye-tracking features to predict construction task performance.

Is apparent fixational drift in eye-tracking data due to filters or eyeball rotation?

Eye trackers are sometimes used to study the miniature eye movements such as drift that occur while observers fixate a static location on a screen. Specifically, analysis of such eye-tracking data can be performed by examining the temporal spectrum composition of the recorded gaze position signal, allowing to assess its color. However, not only rotations of the eyeball but also filters in the eye tracker may affect the signal’s spectral color. Here, we therefore ask whether colored, as opposed to white, signal dynamics in eye-tracking recordings reflect fixational eye movements, or whether they are instead largely due to filters. We recorded gaze position data with five eye trackers from four pairs of human eyes performing fixation sequences, and also from artificial eyes. We examined the spectral color of the gaze position signals produced by the eye trackers, both with their filters switched on, and for unfiltered data. We found that while filtered data recorded from both human and artificial eyes were colored for all eye trackers, for most eye trackers the signal was white when examining both unfiltered human and unfiltered artificial eye data. These results suggest that color in the eye-movement recordings was due to filters for all eye trackers except the most precise eye tracker where it may partly reflect fixational eye movements. As such, researchers studying fixational eye movements should be careful to examine the properties of the filters in their eye tracker to ensure they are studying eyeball rotation and not filter properties.

Visual Exploration at Higher Fixation Frequency Increases Subsequent Memory Recall.

Only a small proportion of what we see can later be recalled. Up to date it is unknown how far differences in visual exploration during encoding affect the strength of episodic memories. Here, we identified individual gaze characteristics by analyzing eye tracking data in a picture encoding task performed by 967 healthy subjects during fMRI. We found a positive correlation between fixation frequency during visual exploration and subsequent free recall performance. Brain imaging results showed a positive correlation of fixation frequency with activations in regions related to vision and memory, including the medial temporal lobe. To investigate if higher fixation frequency is causally linked to better memory, we experimentally manipulated visual exploration patterns in an independent population of 64 subjects. Doubling the number of fixations within a given exploration time increased subsequent free recall performance by 19%. Our findings provide evidence for a causal relationship between fixation frequency and episodic memory for visual information.

PyTrack: An end-to-end analysis toolkit for eye tracking

Eye tracking is a widely used tool for behavioral research in the field of psychology. With technological advancement, we now have specialized eye-tracking devices that offer high sampling rates, up to 2000 Hz, and allow for measuring eye movements with high accuracy. They also offer high spatial resolution, which enables the recording of very small movements, like drifts and microsaccades. Features and parameters of interest that characterize eye movements need to be algorithmically extracted from raw data as most eye trackers identify only basic parameters, such as blinks, fixations, and saccades. Eye-tracking experiments may investigate eye movement behavior in different groups of participants and in varying stimuli conditions. Hence, the analysis stage of such experiments typically involves two phases, (i) extraction of parameters of interest and (ii) statistical analysis between different participants or stimuli conditions using these parameters. Furthermore, the datasets collected in these experiments are usually very large in size, owing to the high temporal resolution of the eye trackers, and hence would benefit from an automated analysis toolkit. In this work, we present PyTrack, an end-to-end open-source solution for the analysis and visualization of eye-tracking data. It can be used to extract parameters of interest, generate and visualize a variety of gaze plots from raw eye-tracking data, and conduct statistical analysis between stimuli conditions and subject groups.

Using eye-tracking to examine the role of first and second language glosses

This study employs eye-tracking to investigate how first (L1) and second language (L2) glosses affect lexical uptake and reading behaviors in L2 learners of English. The study also explores the relationship between lexical uptake and reading behaviors as a function of gloss type. To investigate this, 81 Korean university students were asked to read a baseline passage with no gloss or the same passage with glosses in the study’s L1 (Korean) or L2 (English). Their eye movements were recorded with an eye tracker as they read, and they were subsequently asked to respond to two vocabulary tests. Analyses of eye-tracking data and vocabulary test scores revealed that the presence or absence of L1 and L2 glosses might produce differences in lexical uptake and dissimilar attentional mechanisms. For instance, the study found that L1 and L2 glosses failed to significantly enhance the acquisition of visual word forms, whereas both types of glosses were significantly effective in consolidating form–meaning associations. Additionally, correlation analyses indicated that the relationship between reading behaviors and lexical acquisition might differ depending on gloss type. Ultimately, our findings provide a more comprehensive picture of L1 and L2 gloss effects, and have significant implications for L2 pedagogy.

The identification of children with autism spectrum disorder by SVM approach on EEG and eye-tracking data

ObjectiveTo identify autistic children, we used features extracted from two modalities (EEG and eye-tracking) as input to a machine learning approach (SVM). MethodsA total of 97 children aged from 3 to 6 were enrolled in the present study. After resting-state EEG data recording, the children performed eye-tracking tests individually on own-race and other-race stranger faces stimuli. Power spectrum analysis was used for EEG analysis and areas of interest (AOI) were selected for face gaze analysis of eye-tracking data. The minimum redundancy maximum relevance (MRMR) feature selection method combined with SVM classifiers were used for classification of autistic versus typically developing children. ResultsResults showed that classification accuracy from combining two types of data reached a maximum of 85.44%, with AUC = 0.93, when 32 features were selected. LimitationsThe sample consisted of children aged from 3 to 6, and no younger patients were included. ConclusionsOur machine learning approach, combining EEG and eye-tracking data, may be a useful tool for the identification of children with ASD, and may help for diagnostic processes.

A hidden Markov model for analyzing eye-tracking of moving objects : Case study in a sustained attention paradigm.

Eye-tracking provides an opportunity to generate and analyze high-density data relevant to understanding cognition. However, while events in the real world are often dynamic, eye-tracking paradigms are typically limited to assessing gaze toward static objects. In this study, we propose a generative framework, based on a hidden Markov model (HMM), for using eye-tracking data to analyze behavior in the context of multiple moving objects of interest. We apply this framework to analyze data from a recent visual object tracking task paradigm, TrackIt, for studying selective sustained attention in children. Within this paradigm, we present two validation experiments to show that the HMM provides a viable approach to studying eye-tracking data with moving stimuli, and to illustrate the benefits of the HMM approach over some more naive possible approaches. The first experiment utilizes a novel 'supervised' variant of TrackIt, while the second compares directly with judgments made by human coders using data from the original TrackIt task. Our results suggest that the HMM-based method provides a robust analysis of eye-tracking data with moving stimuli, both for adults and for children as young as 3.5-6 years old.

ISeeColor: Method for Advanced Visual Analytics of Eye Tracking Data

Recent advances in head-mounted eye-tracking technology have allowed researchers to monitor eye movements during locomotion in real-world environments, increasing the ecological validity of research on human gaze behavior. While collecting eye-tracking data is becoming more accessible, visual analytics of eye-tracking data remains difficult and time-consuming. As such, there is a significant need for developing efficient visualization and analysis tools for large-scale eye-tracking data. This work develops a first-of-its-kind eye-tracking data visualization and analysis system that allows for automatic recognition of independent objects within field-of-vision, using deep-learning-based semantic segmentation. This system recolors the fixated objects-of-interest by integrating gaze fixation information with semantic maps. The system effectively allows researchers to automatically infer what objects users view and for how long in dynamic contexts. The contributions are 1) a data visualization and analysis system that uses deep-learning technology along with eye-tracking data to automatically recognize objects-of-interest from head-mounted eye-tracking video recordings, and 2) a graphical user interface that presents objects-of-interest annotation along with eye-tracking data information. The architecture is tested with an outdoor case study of users walking around the Tufts University campus as part of a navigation study, which was administered by a team of research psychologists.

The Effects of Content Ephemerality on Information Processing

Many forms of communication, from verbal conversations to messaging and content sharing via apps such as Snapchat and Telegram, limit the number of times people can examine content. We investigate how this restriction affects information processing. Building on the notion that people strategically allocate cognitive effort, we propose that receivers increase allocation of cognitive resources when processing ephemeral content. Five preregistered studies and exploratory analysis of eye-tracking data (total N = 6,543) demonstrate that making content ephemeral – that is, restricting people to view it once (versus multiple times) – increases top-down attention allocation, prolongs voluntary viewing time, and magnifies focus on relevant information. These effects facilitate more accurate recall (both cued and free recall), improve comprehension, and generate more positive attitudes towards likable content. Taken together, these findings suggest that sharers can communicate information more effectively by merely telling their audience that they could not view it again.

건축·공간디자인 분야의 학제간 융합 연구를 위한 시선추적 관련 선행연구의 통합적 경향 분석

This study aims to suggest implications for the field of architecture and space design by analyzing the tendencies of journal articles that have been conducted using the eye tracking technique, from various angles. For this study, years, research fields, research purposes, reasons for introduction of the technique and research methods of the final articles, selected in accordance with the data selection criteria were analyzed and the study results are as follows. First, in relation to the research purposes of the previous studies, ‘analyses’ occupied 42.4% of them from a quantitative perspective, but there was a relative lack of the studies for ‘evaluating’, ‘suggesting’, ‘developing’ and ‘seeking’. It is considered that when eye tracking is applied to architecture and space design in future, rather than simple analyses, study results containing practical implications need to be provided. Second, holding a majority, there were 125 papers(55.6%) that introduced the eye tracking technique, ‘because this was an useful technique as an objective and quantitative analysis tool’. This implies the eye tracking technique can be a helpful research method too for research on behaviors of human beings that use consumed spaces or spaces used by the majority. Third, the subjects of the eye tracking experiments were divided into architectural and street spaces, behaviors, images and documents. It was hard to find the validity of experimental design in many of the studies, since there was an absence of descriptions on the study subjects and the experiment time. This demonstrates that eye tracking experimental studies in the future field of architecture and space design have to describe experimental design processes clearly not to lose the objectivity of research. Fourth, when it comes to analyzing eye tracking data, in the initial stages of research, most of the analyses focused on the observation time, whereas since 2010, data have been analyzed using more than two analysis methodologies. This means in the future field of architecture and space design, research tasks need to be verified in an objective way, based on integrated analyses using a variety of analysis methodologies. Lastly, when comparing the tendencies of the studies on architecture and space design and on other fields, the field of architecture and space design has been focused on visual and perceptual response analysis of the subjects, while other fields have been carried out with diverse topics. In the future architecture and space design, the scope of research needs to expand as well in order to seek proper evaluation plans, verify and inquire into design elements, suggest design guidelines and develop education tools.

Should online math learning environments be tailored to individuals’ cognitive profiles?

Online learning environments are well-suited for tailoring the learning experience of children individually and on a large scale. An environment such as Math Garden allows children to practice exercises adapted to their specific mathematical ability; this is thought to maximize their mathematical skills. In the current experiment, we investigated whether learning environments should also consider the differential impact of cognitive load on children’s math performance depending on their individual verbal working memory (WM) and inhibitory control (IC) capacity. A total of 39 children (8–11 years old) performed a multiple-choice computerized arithmetic game. Participants were randomly assigned to two conditions where the visibility of time pressure, a key feature in most gamified learning environments, was manipulated. Results showed that verbal WM was positively associated with arithmetic performance in general but that higher IC predicted better performance only when the time pressure was not visible. This effect was mostly driven by the younger children. Exploratory analyses of eye-tracking data (N = 36) showed that when time pressure was visible, children attended more often to the question (e.g., 6 × 8). In addition, when time pressure was visible, children with lower IC, in particular younger children, attended more often to answer options representing operant confusion (e.g., 9 × 4 = 13) and visited more answer options before responding. These findings suggest that tailoring the visibility of time pressure, based on a child’s individual cognitive profile, could improve arithmetic performance and may in turn improve learning in online learning environments.

Contributions of filtering and attentional allocation to working memory performance in individuals with autism spectrum disorder.

Past findings on working memory (WM) ability in individuals with autism spectrum disorder (ASD) are mixed. The present objective was to assess not only the integrity of WM capacity, but also the potential contribution of filtering ability and attentional selection to WM performance, in individuals with ASD. A sample of 24 participants with ASD (Mage = 19.6 years) and 24 typically developing participants without ASD (Mage = 20.3 years) participated. Participants completed a computerized paradigm designed to systematically assess WM capacity, visual filtering ability, and attentional selection. In brief, participants were shown visual arrays consisting of 2-8 colored stimuli (circles and/or squares). After a short delay, memory for one of the stimuli was probed. Importantly, participants were informed beforehand that one of the shape types (e.g., circles) was more likely to be probed compared to the other shape type (e.g., squares) - thus making it strategically advantageous to focus on the high frequency shapes and to filter/ignore the low frequency shapes. Eye tracking data were simultaneously collected. The ASD group demonstrated intact WM capacity and filtering ability, but disrupted ability to efficiently allocate capacity under the demands of high WM load. Analysis of eye tracking data suggests the groups may have differed in their strategic approach to encoding stimuli which may have, in turn, contributed to the aforementioned impairment. Findings support the assertion that disruptions in secondary processes such as strategy use and attentional selection may have played a role in previous reports of WM impairment in ASD. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Eye tracking in developmental cognitive neuroscience – The good, the bad and the ugly

Eye tracking is a popular research tool in developmental cognitive neuroscience for studying the development of perceptual and cognitive processes. However, eye tracking in the context of development is also challenging. In this paper, we ask how knowledge on eye-tracking data quality can be used to improve eye-tracking recordings and analyses in longitudinal research so that valid conclusions about child development may be drawn. We answer this question by adopting the data-quality perspective and surveying the eye-tracking setup, training protocols, and data analysis of the YOUth study (investigating neurocognitive development of 6000 children). We first show how our eye-tracking setup has been optimized for recording high-quality eye-tracking data. Second, we show that eye-tracking data quality can be operator-dependent even after a thorough training protocol. Finally, we report distributions of eye-tracking data quality measures for four age groups (5 months, 10 months, 3 years, and 9 years), based on 1531 recordings. We end with advice for (prospective) developmental eye-tracking researchers and generalizations to other methodologies.

Gaze analysis of user characteristics in magazine style narrative visualizations

Previous research has shown that various user characteristics (e.g., cognitive abilities, personality traits, and learning abilities) can influence user experience during information visualization tasks. These findings have prompted researchers to investigate user-adaptive information visualizations that can help users by providing personalized support based on their specific needs. Whereas existing work has been mostly limited to tasks involving just visualizations, the aim of our research is to broaden this work to include scenarios where users process textual documents with embedded visualizations, i.e., Magazine Style Narrative Visualizations, or MSNVs for short. In this paper, we analyze eye tracking data collected from a user study with MSNVs to uncover processing behaviors that are negatively impacting user experience (i.e., time on task) for users with low abilities in these user characteristics. Our analysis leverages Linear Mixed-Effects Models to evaluate the relationships among user characteristics, gaze processing behaviors, and task performance. Our results identify several MSNV processing behaviors within the visualization that contribute to poor task performance for users with low reading proficiency. For instance, we identify that users with low reading proficiency transition significantly more often compared to their counterparts between relevant and non-relevant bars, and transition more often from bars to the labels. We present our findings as a step toward designing user-adaptive support mechanisms to alleviate these difficulties with MSNVs, and provide suggestions on how our results can be leveraged for creating a set of meaningful interventions for future evaluation (e.g., dynamically highlighting relevant bars and labels in the visualization to help users with low reading proficiency locate them more effectively).

Generalization Strategies in Finding the nth Term Rule for Simple Quadratic Sequences

In this study, we identify ways in which a sample of 18 graduates with mathematics-related first degrees found the nth term for quadratic sequences from the first values of a sequence of data, presented on a computer screen in various formats: tabular, scattered data pairs and sequential. Participants’ approaches to identifying the nth term were recorded with eye-tracking technology. Our aims are to identify their strategies and to explore whether and how format influences these strategies. Qualitative analysis of eye-tracking data offers several strategies: Sequence of Differences, Building a Relationship, Known Formula, Linear Recursive and Initial Conjecture. Sequence of Differences was the most common strategy, but Building a Relationship was more likely to lead to the right formula. Building from Square and Factor Search were the most successful methods of Building a Relationship. Findings about the influence of format on the range of strategies were inconclusive but analysis indicated sporadic evidence of possible influences.

Effects of Individuality, Education, and Image on Visual Attention: Analyzing Eye-tracking Data using Machine Learning.

Machine learning, particularly classification algorithms, constructs mathematical models from labeled data that can predict labels for new data. Using its capability to identify distinguishing patterns among multi-dimensional data, we investigated the impact of three factors on the observation of architectural scenes: individuality, education, and image stimuli. An analysis of the eye-tracking data revealed that (1) a velocity histogram was unique to individuals, (2) students of architecture and other disciplines could be distinguished via endogenous parameters, but (3) they were more distinct in terms of seeking structural versus symbolic elements. Because of the reverse nature of the classification algorithms that automatically learn from data, we could identify relevant parameters and distinguishing eye-tracking patterns that have not been reported in previous studies.

Application of hidden Markov models to eye tracking data analysis of visual quality inspection operations

Visual inspection is used in many areas due to the potential high costs of inspection error such as injury, fatality, loss of expensive equipment, scrapped items, rework, or failure to procure repeat business. This study presents an application of hidden Markov models (HMM) to fixations’ sequences analysis during visual inspection of front panels in a home appliance facility. The eye tracking data are gathered when quality control operators perform their tasks. The results support the difference between expert and novice operator. Moreover, the article demonstrates four HMMs with two and three hidden states both for novice and experienced operators and provides analysis and discussion of the outcomes.