Analysis Of Eye Movement Data Research Articles

Efficient diagnostic classification of diverse pathologies through contextual eye movement data analysis with a novel hybrid architecture

The analysis of eye movements has proven valuable for understanding brain function and the neuropathology of various disorders. This research aims to utilize eye movement data analysis as a screening tool for differentiation between eight different groups of pathologies, including scholar, neurologic, and postural disorders. Leveraging a dataset from 20 clinical centers, all employing AIDEAL and REMOBI eye movement technologies this study extends prior research by considering a multi-annotation setting, incorporating information from recordings from saccade and vergence eye movement tests, and using contextual information (e.g. target signals and latency of the eye movement relative to the target and confidence level of the quality of eye movement recording) to improve accuracy while reducing noise interference. Additionally, we introduce a novel hybrid architecture that combines the weight-sharing feature of convolution layers with the long-range capabilities of the transformer architecture to improve model efficiency and reduce the computation cost by a factor of 3.36, while still being competitive in terms of macro F1 score. Evaluated on two diverse datasets, our method demonstrates promising results, the most powerful discrimination being Attention & Neurologic; with a macro F1 score of up to 78.8%; disorder. The results indicate the effectiveness of our approach in classifying eye movement data from different pathologies and different clinical centers accurately, thus enabling the creation of an assistant tool in the future.

Разработка методик на основе айтрекинга для диагностики когнитивных функций у детей

<p style="text-align: justify;">The development of eye-movement-based diagnostic techniques is especially relevant for assessing cognitive function in children who are unable to provide a verbal or motor response due to their age or developmental disabilities. For these children, the use of objective methods for assessing cognitive function based on eye-tracking can be a useful diagnostic tool to complement traditional behavioural questionnaires and scales. In addition, such diagnostic techniques can be used to quickly screen for problems in the development of cognitive functions. In this article, we review researches on the development of such techniques. Studies show that eye-tracking-based techniques can assess attention, receptive speech, especially in young children, and provide screening assessments of intelligence in children with developmental delays. The data obtained using eye-tracking techniques generally significantly correlate with those obtained using traditional behavioural techniques, indicating that further development of such techniques is promising. The application of machine learning methods may help to create simpler and more effective screening techniques for assessing cognitive processes based on the analysis of eye movement data.</p>

The smell of paper or the shine of a screen? Students’ reading comprehension, text processing, and attitudes when reading on paper and screen

With the increasing prevalence of digital devices such as smartphones, tablets and e-readers, more and more reading is happening in digital formats – also in classrooms across the world. The present study focuses on lower secondary school students and their reading comprehension and attitudes toward reading on paper and screens. The study uses an innovative methodological approach were eye-tracking technology is used to observe ten carefully sampled eight grade students with different reading levels. The students read a selection of texts and answered questions from the Norwegian national reading assessment on comparable versions on paper and on screen. By analyzing eye movement data (reading transitions), including more than 25000 fixations, in combination with text reading comprehension outcomes, students' cued retrospective reporting from reading, and interview data we have obtained detailed and comprehensive data on students' reading comprehension, reading behavior, and reading experiences across different media. A key result is that reading on screen leads to more shallow processing and can hinder reading comprehension. Importantly, our results from the students' cued retrospective reporting of their eye tracking, showed that they were unaware of their reading behavior and didn't reflect much on reading in different media. These findings have implications for the increasing shift to digital learning environments in the educational context. It is important to recognize the difference between reading processes, and policymakers and practitioners cannot assume that these processes are the same across individuals and different delivery modes.

Evaluating the Usability of mHealth Apps: An Evaluation Model Based on Task Analysis Methods and Eye Movement Data.

Advancements in information technology have facilitated the emergence of mHealth apps as crucial tools for health management and chronic disease prevention. This research work focuses on mHealth apps for the management of diabetes by patients on their own. Given that China has the highest number of diabetes patients in the world, with 141 million people and a prevalence rate of 12.8% (mentioned in the Global Overview of Diabetes), the development of a usability research methodology to assess and validate the user-friendliness of apps is necessary. This study describes a usability evaluation model that combines task analysis methods and eye movement data. A blood glucose recording application was designed to be evaluated. The evaluation was designed based on the model, and the feasibility of the model was demonstrated by comparing the usability of the blood glucose logging application before and after a prototype modification based on the improvement suggestions derived from the evaluation. Tests showed that an improvement plan based on error logs and post-task questionnaires for task analysis improves interaction usability by about 24%, in addition to an improvement plan based on eye movement data analysis for hotspot movement acceleration that improves information access usability by about 15%. The results demonstrate that this study presents a usability evaluation model for mHealth apps that enables the effective evaluation of the usability of mHealth apps.

PACMHCI V8, ETRA, May 2024 Editorial

We are excited to provide the third issue of the Proceedings of the ACM on Human-Computer Interaction focusing on contributions from the ACM Eye Tracking Research and Applications (ETRA) community. ETRA is the premier eye-tracking conference bringing together researchers from across disciplines to present advances in eye-tracking systems and methods, oculomotor research, eye movement data analysis, gaze-based interaction, and eye-tracking applications. A total of 25 full papers were accepted for ETRA 2024 (June 4 - 7, 2024, in Glasgow, U.K.), selected from 68 submissions after a rigorous reviewing process (37% acceptance rate). Depending on the topic fit and authors' preferences, accepted papers were split into two issues. 19 accepted papers are presented in this issue, and 6 will be included in an issue of the Proceedings of the ACM on Computer Graphics and Interactive Techniques. We want to thank all members of the Editorial Board and all external reviewers for their effort and dedication, as well as all authors for their high-quality contributions.

Exploring Cognitive Dysfunction in Long COVID Patients: Eye Movement Abnormalities and Frontal-Subcortical Circuits Implications via Eye-Tracking and Machine Learning

BackgroundCognitive dysfunction is regarded as one of the most severe aftereffects following coronavirus disease 2019 (COVID-19). Eye movements, controlled by several brain areas, such as the dorsolateral prefrontal cortex and frontal-thalamic circuits, provide a potential metric for assessing cortical networks and cognitive status. We aimed to examine the utility of eye movement measurements in identifying cognitive impairments in long COVID patients. MethodsWe recruited 40 long COVID patients experiencing subjective cognitive complaints and 40 healthy controls and used a certified eye-tracking medical device to record saccades and antisaccades. Machine learning was applied to enhance the analysis of eye movement data. ResultsPatients did not differ from the healthy controls regarding age, sex, and years of education. However, the patients' Montreal Cognitive Assessment total score was significantly lower than healthy controls. Most eye movement parameters were significantly worse in patients. These included the latencies, gain (computed as the ratio between stimulus amplitude and gaze amplitude), velocities, and accuracy (evaluated by the presence of hypermetric or hypometria dysmetria) of both visually and memory-guided saccades; the number of correct memory saccades; the latencies and duration of reflexive saccades; and the number of errors in the antisaccade test. Machine learning permitted distinguishing between long COVID patients experiencing subjective cognitive complaints and healthy controls. ConclusionOur findings suggest impairments in frontal subcortical circuits among long COVID patients who report subjective cognitive complaints. Eye-tracking, combined with machine learning, offers a novel, efficient way to assess and monitor long COVID patients' cognitive dysfunctions, suggesting its utility in clinical settings for early detection and personalized treatment strategies. Further research is needed to determine the long-term implications of these findings and the reversibility of cognitive dysfunctions.

An Eye Movement Classification Method based on Cascade Forest.

Eye tracking technology has become increasingly important in scientific research and practical applications. In the field of eye tracking research, analysis of eye movement data is crucial, particularly for classifying raw eye movement data into eye movement events. Current classification methods exhibit considerable variation in adaptability across different participants, and it is necessary to address the issues of class imbalance and data scarcity in eye movement classification. In the current study, we introduce a novel eye movement classification method based on cascade forest (EMCCF), which comprises two modules: (1) a feature extraction module that employs a multi-scale time window method to extract features from raw eye movement data; (2) a classification module that innovatively employs a layered ensemble architecture, integrating the cascade forest structure with ensemble learning principles, specifically for eye movement classification. Consequently, EMCCF not only enhanced the accuracy and efficiency of eye movement classification but also represents an advancement in applying ensemble learning techniques within this domain. Furthermore, experimental results indicated that EMCCF outperformed existing deep learning-based classification models in several metrics and demonstrated robust performance across different datasets and participants.

For higher accuracy: 3D eye movement gaze depth estimation method based on Mix-TCN

Currently, the biggest obstacle in applying eye tracking technology in Virtual Reality (VR) and Augmented Reality (AR) scenes is the particular difficulty in choosing distance and object distance in 3D scenes. In previous research, geometric calculation methods using vestibular ocular reflex (VOR) and research on binocular visual angle have been studied to some extent, but unfortunately, their effects have not reached a practical level. The paper proposes a new research idea to estimate the depth of binocular gaze using a method of time series eye movement data analysis based on depth learning, and proposes a Mix-Temporal Convolutional Network (TCN) optical time series network. By combining VOR and deep learning theory, the paper has realized the current state-of-the-art technology for estimating the depth of gaze from the movement of the gaze.

Alleviating design fixation with Alternative Uses Task: The role of an integrated and design-independent intervention in promoting creative incubation

Fixation hampers creativity and impedes progress in product styling design. Alternative Uses Task (AUT) is a promising intervention to mitigate design fixation as it involves both associative and suppressive cognitive processes and is distant from product styling design in both form and content. This article describes a designer-centred experiment investigating whether and how AUT can mitigate design fixation by comparing the impact of AUT and a descriptive task (DT) on the subsequent design phases. The analysis of design schemes and questionnaire feedback shows that AUT improved design schemes and reduced design fixation better than the DT. Further analysis of eye movement data revealed that AUT facilitated attention control and enhanced arousal. AUT is a cognitively beneficial intervention task applicable to design contexts that can yield benefits after short-term exposure.

PACMHCI V7, ETRA, May 2023 Editorial

In 2022, ETRA moved its publication of full papers to a journal-based model, and we are delighted to present the second issue of the Proceedings of the ACM on Human-Computer Interaction to focus on contributions from the Eye Tracking Research and Applications (ETRA) community. ETRA is the premier eye-tracking conference that brings together researchers from across disciplines to present advances and innovations in oculomotor research, eye tracking systems, eye movement data analysis, eye tracking applications, and gaze-based interaction. This issue presents 13 full papers accepted for presentation at ETRA 2023 (May 30 - June 2, 2023, in Tübingen, Germany) selected from 37 submissions (35% acceptance rate). We are grateful to all authors for the exciting contributions they have produced and to the Editorial Board and external reviewers for their effort during the entire rigorous reviewing process which resulted in high-quality and insightful reviews for all submitted articles.

Eye tracking method based on mobile big data in computer environment

The human brain needs to rely on the information obtained by the eyes to process information. To a certain extent, the eyes are the reflection of the human brain activity. Based on this, many people pursue eyeball changes to judge a person’s psychological state, and eye tracking technology has also been proposed, playing an increasingly important role in different fields. The purpose is to study the human attention distribution system in the computer environment. We use the eye tracking technology of the eye tracker to record any actions of each learner during learning, use the function of the eye tracker to obtain various data, and use statistical analysis to analyze it. The experimental results show that the eyes can reflect human behavior information well. Analyzing eye movement data can deepen people’s understanding of the mechanism of attention distribution and improve learning efficiency by 20%.

AN IMPROVED DYNAMIC TIME WARPING METHOD COMBINING DISTANCE DENSITY CLUSTERING FOR EYE MOVEMENT ANALYSIS

Analyzing eye movement data to evaluate learning status has become crucial in intelligent education. The eye movement scanning path can directly or indirectly reflect changes in thinking patterns and psychological states. By analyzing the scanning path, we can explore the commonality and differences in learners’ eye movement behaviors and provide essential references for improving visual content and giving guidance. This paper first studies the time series representation and clustering of the learner’s scanning path under the same task. Then, the three learning states of concentration, mind-wandering, and information wandering are evaluated through the clustering results. Specifically, the improved DBA algorithm (iDBA) is proposed to extract group eye movement patterns, combined with the dynamic time warping (DTW) algorithm to calculate the similarity of scanning paths and determine the clustering seeds, while the distance density clustering (DDC) algorithm is used for clustering. Experiments show that time series-based eye movement pattern mining can identify group viewing behaviors. Meanwhile, clustering reveals different reading strategies and provides the ability to assess learning status.

Mining attention distribution paradigm: Discover gaze patterns and their association rules behind the visual image

Background and ObjectiveAttention allocation reflects the way of humans filtering and organizing the information. On one hand, different task scenarios seriously affect human's rule of attention distribution, on the other hand, visual attention reflecting the cognitive and psychological process. Most of the previous studies on visual attention allocation are based on cognitive models, predicted models, or statistical analysis of eye movement data or visual images, however, these methods are inadequate to provide an inside view of gaze behavior to reveal the attention distribution pattern within scenario context. Moreover, they seldom study the association rules of these patterns. Therefore, we adopted the big data mining approach to discover the paradigm of visual attention distribution. MethodsWe applied the data mining method to extract the gaze patterns to discover the regularities of attention distribution behavior within the scenario context. The proposed method consists of three components, tasks scenario segmented and clustered, gaze pattern mining, and association rule of frequent pattern mining. ResultsThe proposed approach is tested on the operation platform. The complex operation task is simultaneously segmented and clustered with the TICC-based method and evaluated by the BCI index. The operator's eye movement frequent patterns and their association rule are discovered. The results demonstrate that our method can associate the eye-tracking data with the task-oriented scene data. DiscussionThe proposed method provides the benefits of being able to explicitly express and quantitatively analyze people's visual attention patterns. The proposed method can not only be applied in the field of aerospace medicine and aviation psychology, but also can likely be applied to computer-aided diagnosis and follow-up tool for neurological disease and cognitive impairment related disease, such as ADHD (Attention Deficit Hyperactivity Disorder), neglect syndrome, social attention differences in ASD (Autism spectrum disorder).

A machine learning approach to primacy-peak-recency effect-based satisfaction prediction

Data retrieval systems supporting the discovery and reuse of open data are emerging as important tools in the open data ecosystem. However, user satisfaction with them is relatively low. This study proposes the primacy-peak-recency effect to investigate the cognitive mechanisms underlying data searchers’ overall satisfaction. To test the primacy-peak-recency effect, primacy-peak-recency cubes consisting of eye movement indicators at primacy, peak, and recency moments and their combinations are constructed as the theoretical model. A user experiment was conducted to collect eye movement data and satisfaction scores generated during 48 doctoral students’ interactions with data retrieval systems. An ensemble machine learning framework was then applied to analyze eye movement data to assess the theoretical model. The results indicate that the primacy-peak-recency cubes are salient predictors of data searchers’ satisfaction (the prediction accuracy=0.682 and regression R2=0.749). This finding suggests that data searchers’ complex cognitive processes at primacy, peak, and recency moments measured by uni-, bi-, and three-dimensional eye movement indicators are predictors of overall satisfaction, confirming the primacy-peak-recency effect. In addition, combinations of varying types of influential moments and multidimensional eye movement events are the best predictors of overall satisfaction. This suggests that influential moments and cognitive processes have additive effects on overall satisfaction. Combining theory-driven and data-driven approaches, this study sheds light on the potential of machine learning approaches for analyzing neuropsychological data for heuristics examination. With these insights, practical strategies to predict data searchers’ satisfaction and optimize the user-experience design of data retrieval systems are proposed.

An improved algorithm for saccadic intrusion qualification under air traffic control situation

Saccadic intrusions are saccadic deviations that interrupt fixations. Saccadic intrusions have the unique characteristics of regular saccades and a small amplitude and round trip, providing a foundation for accurately identifying and quantifying saccadic intrusions. However, currently, rapid and accurate identification algorithms for saccadic intrusions of air traffic controllers are lacking. This study aimed to improve Tokuda's quantification algorithm for saccadic eye movement deviation in terms of data preprocessing, saccadic eye movement identification, and saccade amplitude threshold. The eye movement data were recorded using an eye tracker at a frequency of 100 Hz and obtained from the control simulation experiment. A total of 20 participants with basic control knowledge participated in this study, and finally 54 sets of data were obtained. The analysis of eye movement data showed that the saccadic intrusion quantification value of 36 of 54 sets of data obtained by the improved algorithm was closer to the value output by the manual identification method compared with the original algorithm. The results showed that these improvements had positive effects on the algorithm.

Discourse complexity in the light of eye-tracking: a pilot Russian language study

The paper explores the influence of discourse structure on text complexity. We assume that certain types of discourse units are easier to read than others, due to their explicit discourse structure, which makes their informational input more accessible. As a data source, we use the dataset from the MECO corpus, which contains eye movement data for 12 Russian texts read by 35 native speakers. We demonstrate that the approach relying on elementary discourse units (EDUs) can be felicitously used in the analysis of eye movement data, since fixation patterns on EDUs are similar to those on whole sentences. Our analysis has identified EDU outliers, which show shorter time of first fixation than estimated. We arranged these outliers into several groups associated with different discourse structures. First, these are statements with nominal predicates that set exposition of the text or macroproposition and, following those, EDUs that elaborate on the previous statement and signal the beginning of the narrative. Second, they are EDUs that serve as the middle component of a listing or a group of coordinated clauses or phrases. The final group represents EDUs that are part of an opposition, contrast or comparison. Discourse analysis based on EDUs has never been applied to eye movement data, so our project opens many avenues for further research of complexity of discourse structure.

What can a line of sight tell us? Working with a gantry crane at 50 m above ground level

With the support of gantry crane operators who load and unload containers between vessels and trailers, we visualized the gap in the line of sight between beginners and experienced operators. The aim was to explore training methods for skills transfer. Through an analysis of eye-movement data, we found that the working environment at 50 m above ground level was psychologically oppressive, so the beginners had an imbalanced gaze to the vital parts of the object. On the other hand, experienced operators made efforts to collect information by widening their gaze range.When it comes to training, in a stressful environment, too much input may instead lead to panic for beginners. Gradually expanding the scope of the inputs becomes important. In addition, beginners also need to carefully imitate the experienced operators in focusing on objects, so that they may truly grasp the skills.

EyeBox: A Toolbox based on Python3 for Eye Movement Analysis

Eye tracking technology can reflect human attention and cognition, widely used as a research tool. To analyze eye movement data, users need to determine a specific area known as areas of interests (AOIs). Although existing tools offer dynamic AOIs functions to process visual behavior on moving stimuli, they may ask users to use markers to specify contours of moving stimuli in the physical environment or define AOIs manually on screen. This paper proposes a toolbox named Eyebox to 1) recognize dynamic AOIs automatically based on SIFT and extract eye movement indicators, as well as 2) draw fixations. We also design a user-friendly interface for this toolbox. Eyebox currently supports processing data recorded from the Pupil Core device. We compared results processed by manual with by Eyebox in a custom eye-tracking dataset to evaluate this toolbox. The accuracy of 3/4 data for AOI1 is above 90%, and the accuracy of 4/5 data for AOI2 is higher than 90%. Finally, we conducted a user study to test the usability and user experience of EyeBox.

Effects of different interaction modes on fatigue and reading effectiveness with mobile phones

Reading on touchscreen mobile devices is common in modern society. However, it is difficult to determine which interaction mode has the best reading effect when reading texts of different lengths. Using memory tests, eye-movement data analysis, and task load scale, we studied the effects of paging and vertical scrolling modes on the memory level, visual fatigue, task load, and reading speed of 28 participants reading both long and short texts. The results showed that vertical scrolling has a better memory effect than paging mode when reading short texts, but there was no difference in effect when reading long texts; the blink frequency in paging mode is higher than in scrolling mode whether reading long or short texts; no significant differences were found in reading speed in scrolling or paging mode when reading short or long texts; and in task load, the mental and temporal demands in scrolling mode were lower than in paging mode whether reading long or short texts. Considering that most current mobile readings do not differ in interaction mode according to the length of reading material, our conclusions provide valuable design recommendations to designers working on mobile reading apps.

Eye movements in real and simulated driving and navigation control - Foreword to the Special Issue.

The control of technological systems by human operators has been the object of study for many decades. The increasing complexity in the digital age has made the optimization of the interaction between system and human operator particularly necessary. In the present thematic issue, ten exemplary articles are presented, ranging from observational field studies to experimental work in highly complex navigation simulators. For the human operator, the processes of attention play a crucial role, which are captured in the contributions listed in this thematic issue by eye-tracking devices.For many decades, eye tracking during car driving has been investigated extensively (e.g. 6; 5). In the present special issue, Cvahte Ojsteršek & Topolšek (4) provide a literature review and scientometric analysis of 139 eye-tracking studies investigating driver distraction. For future studies, the authors recommend a wider variety of distractor stimuli, a larger number of tested participants, and an increasing interdisciplinarity of researchers.In addition to most studies investigating bottom-up processes of covered attention, Tuhkanen, Pekkanen, Lehtonen & Lappi (10) include the experimental control of top-down processes of overt attention in an active visuomotor steering task. The results indicate a bottom-up process of biasing the optic flow of the stimulus input in interaction with the top-down saccade planning induced by the steering task.An expanding area of technological development involves autonomous driving where actions of the human operator directly interact with the programmed reactions of the vehicle. Autonomous driving requires, however, a broader exploration of the entire visual input and less gaze directed towards the road centre. Schnebelen, Charron & Mars (9) conducted experimental research in this area and concluded that gaze dynamics played the most important role in distinguishing between manual and automated driving.Through a combination of advanced gaze tracking systems with the latest vehicle environment sensors, Bickerdt, Wendland, Geisler, Sonnenberg & Kasneci (2021) conducted a study with 50 participants in a driving simulator and propose a novel way to determine perceptual limits which are applicable to realistic driving scenarios.Eye-Computer-Interaction (ECI) is an interactive method of directly controlling a technological device by means of ocular parameters. In this context, Niu, Gao, Xue, Zhang & Yang (8) conducted two experiments to explore the optimum target size and gaze-triggering dwell time in ECI. Their results have an exemplary application value for future interface design.Aircraft training and pilot selection is commonly performed on simulators. This makes it possible to study human capabilities and their limitation in interaction with the simulated technological system. Based on their methodological developments and experimental results, Vlačić, Knežević, Mandal, Rođenkov & Vitsas (11) propose a network approach with three target measures describing the individual saccade strategy of the participants in this study.In their analysis of the cognitive load of pilots, Babu, Jeevitha Shree, Prabhakar, Saluja, Pashilkar & Biswas (3) investigated the ocular parameters of 14 pilots in a simulator and during test flights in an aircraft during air to ground attack training. Their results showed that ocular parameters are significantly different in different flying conditions and significantly correlate with altitude gradients during air to ground dive training tasks.In maritime training the use of simulations is per international regulations mandatory. Mao, Hildre & Zhang (7) performed a study of crane lifting and compared novice and expert operators. Similarities and dissimilarities of eye behavior between novice and expert are outlined and discussed.The study of Atik & Arslan (2) involves capturing and analyzing eye movement data of ship officers with sea experience in simulation exercises for assessing competency. Significant differences were found between electronic navigation competencies of expert and novice ship officers. The authors demonstrate that the eye tracking technology is a valuable tool for the assessment of electronic navigation competency.The focus of the study by Atik (1) is the assessment and training of situational awareness of ship officers in naval Bridge Resource Management. The study shows that eye tracking provides the assessor with important novel data in simulator based maritime training, such as focus of attention, which is a decisive factor for the effectiveness of Bridge Resource Management training.The research presented in the different articles of this special thematic issue cover many different areas of application and involve specialists from different fields, but they converge on repeated demonstrations of the usefulness of measuring attentional processes by eye movements or using gaze parameters for controlling complex technological devices. Together, they share the common goal of improving the potential and safety of technology in the digital age by fitting it to human capabilities and limitations.