Eye Movement Analysis Research Articles

Imaging, biomarkers, and vascular cognitive impairment in China: Rationale and design for the VICA study.

Vascular cognitive impairment (VCI) is highly heterogeneous, with unclear pathogenesis. Individuals with vascular risk factors (VRF), cerebral small vessel disease (CSVD), and stroke are all at risk of developing VCI. To address the growing challenges posed by VCI, the "Vascular, Imaging and Cognition Association of China" (VICA) was established. VICA aims to recruit 10,000 participants, including 2000 with VRF, 3000 with CSVD, and 5000 stroke patients, to form a nationwide multicenter cohort. The study integrates clinical, neuroimaging, and multi-omics data to better understand VCI heterogeneity, improve disease prediction, and ensure timely diagnosis. VICA has screened 2045 eligible VRF participants from six communities in Wuhan, Shanghai, and Taizhou, along with 602 CSVD and 1269 stroke patients from 135 hospitals nationwide. Baseline enrollment and follow-up work are still ongoing. Establishing a high-quality longitudinal cohort is crucial for understanding VCI pathogenesis and developing novel markers for early screening and diagnosis. Establish a large-scale prospective longitudinal cohort comprising 10,000 participants, focusing on the high-risk population of vascular cognitive impairment (VCI) in China. Establish a nationwide three-tier medical network, make full use of resources, and achieve extensive enrollment of patients with cerebral small vessel disease and stroke patients. Utilize multimodal imaging and biomarkers to lay the foundation for constructing more-precise risk models. Introduce eye movement and gait analysis as new methods for assessing cognitive function. Use positron emission tomography to further investigate the interaction between vascular factors and neurodegeneration.

The impact of point symbol similarity on visual search on maps

ABSTRACT The study presented in this paper investigates the impact of the similarity level of cartographic point symbols on visual search performance. Specifically, we explore the application of an algorithm for quantitatively assessing symbols’ shape similarity, and apply the results of such automated analysis in a simple target search task on a map with distractors of varying similarity levels. Our primary hypothesis posits that the more similar the target symbol and distractors, the more challenging the visual search becomes in terms of search efficiency and eye metric performance. In a controlled laboratory experiment utilizing eye-tracking device, 33 participants were recruited, and 60 map stimuli were tested with each participant. The results confirmed the hypothesis assumptions. However, symbols based on a round shape were consistently the easiest to find at every similarity level. Eye movement analysis revealed differences in absolute pupil size between male and female participants, possibly associated with cognitive load, although without affecting search time performance. Interestingly, scan-path speed decreased with an increasing level of similarity between the target symbol and distractors. The practical implication of this study lies in the potential development of tools to assist map designers in selecting symbols with low similarity, thereby ensuring improved map readability.

Towards Automatic Object Detection and Activity Recognition in Indoor Climbing

Rock climbing has propelled from niche sport to mainstream free-time activity and Olympic sport. Moreover, climbing can be studied as an example of a high-stakes perception-action task. However, understanding what constitutes an expert climber is not simple or straightforward. As a dynamic and high-risk activity, climbing requires a precise interplay between cognition, perception, and precise action execution. While prior research has predominantly focused on the movement aspect of climbing (i.e., skeletal posture and individual limb movements), recent studies have also examined the climber’s visual attention and its links to their performance. To associate the climber’s attention with their actions, however, has traditionally required frame-by-frame manual coding of the recorded eye-tracking videos. To overcome this challenge and automatically contextualize the analysis of eye movements in indoor climbing, we present deep learning-driven (YOLOv5) hold detection that facilitates automatic grasp recognition. To demonstrate the framework, we examined the expert climber’s eye movements and egocentric perspective acquired from eye-tracking glasses (SMI and Tobii Glasses 2). Using the framework, we observed that the expert climber’s grasping duration was positively correlated with total fixation duration (r = 0.807) and fixation count (r = 0.864); however, it was negatively correlated with the fixation rate (r = −0.402) and saccade rate (r = −0.344). The findings indicate the moments of cognitive processing and visual search that occurred during decision making and route prospecting. Our work contributes to research on eye–body performance and coordination in high-stakes contexts, and informs the sport science and expands the applications, e.g., in training optimization, injury prevention, and coaching.

Gaze transition entropy as a measure of attention allocation in a dynamic workspace involving automation

Real-world work environments require operators to perform multiple tasks with continual support from an automated system. Eye movement is often used as a surrogate measure of operator attention, yet conventional summary measures such as percent dwell time do not capture dynamic transitions of attention in complex visual workspace. This study analyzed eye movement data collected in a controlled a MATB-II task environment using gaze transition entropy analysis. In the study, human subjects performed a compensatory tracking task, a system monitoring task, and a communication task concurrently. The results indicate that both gaze transition entropy and stationary gaze entropy, measures of randomness in eye movements, decrease when the compensatory tracking task required more continuous monitoring. The findings imply that gaze transition entropy reflects attention allocation of operators performing dynamic operational tasks consistently.

Eye movements in conceptual and non-conceptual thinking

AbstractHow to influence and assess whether students engage in conceptual thinking are longstanding methodological problems in mathematics education. Recently, eye-tracking technology has fueled a discussion on whether eye movement analysis can support valid inferences about mathematical thinking. This study investigates whether eye movement analysis can distinguish between conceptual and non-conceptual thinking in a geometric classification task where both modes of thinking lead to identical responses. Participants were asked to classify geometric shapes while we tracked their eye movements and to report their thinking verbally. Our findings indicate that self-reported conceptual thinking is characterised by fewer eye movements between task shapes and response shapes, and that self-reported non-conceptual thinking involves comparing the shapes’ similarity directly. A logistic regression model correctly classified the self-reported ways of thinking in 80.3% of the cases. We conclude that eye-tracking can contribute to making inferences about mathematical thought processes and facilitate research on how to engage students in conceptual thinking and development.

Eyes as a window to brain pathology in parkinson's disease: a narrative review.

There is a renewed interest on eye movements analysis and retinal alterations in Parkinson's disease. This may identify markers for at-risk subpopulation, early diagnosis and evolutive profiles for research or personalized medicine.

An integrated framework for eye tracking-assisted task capability recognition of air traffic controllers with machine learning

To effectively address the continuously increasing demands of air transport, air traffic management (ATM) systems are evolving towards a human-artificial intelligence (AI) hybrid automation paradigm. In this paradigm, air traffic controllers (ATCOs) play a crucial role in ensuring safe and efficient operations. Recognizing ATCOs’ task capability is essential for evaluating performance, optimizing task assignments, and personalizing training strategies. Physiological signals, such as eye movements, offer objective insights into human behavior and cognitive processes, making them valuable for identifying the task capability of ATCOs. In this study, an integrated framework leveraging machine learning is proposed to achieve this goal. First, a Transformer-attentional long short-term memory (LSTM) network is developed to analyze eye movement patterns, capturing both global and long-term dependencies for precise skill level detection. Next, manual parameters are extracted from the raw eye tracking data and the SHAP (SHapley Additive exPlanation) method is utilized to determine the importance of these parameters, aiding in the selection of relevant performance metrics. Furthermore, a radar chart is implemented to intuitively visualize and compare performance metrics across different skill levels based on the selected parameters. A case study and extensive experiments are conducted to validate the effectiveness of the proposed framework. This research advances task capability recognition for ATCOs in a human-in-the-loop scenario, with a focus on expertise level detection, parameter importance identification, and performance metric comparison.

Online Metaphor Comprehension in Adults with Autism Spectrum Disorders: An Eye Tracking Study

AbstractThe aim of this study was to investigate novel metaphor comprehension in adults with autism spectrum disorder (ASD). Previous literature is conflicting about whether individuals with ASD have impairment in this particular type of figurative language. Participants in the study completed a visual world paradigm eye-tracking task, which involved selecting an interpretation of an auditorily presented sentence (i.e. a picture-sentence matching task), where images corresponded to literal and metaphorical interpretations. Thus, the study also investigated online processing, via reaction times and eye movements. Forty adults participated in the study (18 with ASD and 22 typically-developing controls). Each participant completed the AQ questionnaire and had their vocabulary assessed. Results showed that participants with ASD comprehended metaphorical utterances with the same accuracy as controls. However, they had significantly slower reaction times, and specifically, were approximately 800 ms slower. Analysis of eye movements revealed that participants with ASD showed significantly longer fixation times on both the target and distractor image, the latter of which suggests difficulty overcoming the literal interpretation. Consistent with some prior studies, we showed that adults with ASD are not impaired in novel metaphor comprehension, but they were clearly less efficient. Verbal abilities did not significantly relate to performance. Finally, our online processing measure (eye tracking) provided us with insights into the nature of the ASD inefficiency (i.e. a literality bias).

Efficient diagnostic classification of diverse pathologies through contextual eye movement data analysis with a novel hybrid architecture

The analysis of eye movements has proven valuable for understanding brain function and the neuropathology of various disorders. This research aims to utilize eye movement data analysis as a screening tool for differentiation between eight different groups of pathologies, including scholar, neurologic, and postural disorders. Leveraging a dataset from 20 clinical centers, all employing AIDEAL and REMOBI eye movement technologies this study extends prior research by considering a multi-annotation setting, incorporating information from recordings from saccade and vergence eye movement tests, and using contextual information (e.g. target signals and latency of the eye movement relative to the target and confidence level of the quality of eye movement recording) to improve accuracy while reducing noise interference. Additionally, we introduce a novel hybrid architecture that combines the weight-sharing feature of convolution layers with the long-range capabilities of the transformer architecture to improve model efficiency and reduce the computation cost by a factor of 3.36, while still being competitive in terms of macro F1 score. Evaluated on two diverse datasets, our method demonstrates promising results, the most powerful discrimination being Attention & Neurologic; with a macro F1 score of up to 78.8%; disorder. The results indicate the effectiveness of our approach in classifying eye movement data from different pathologies and different clinical centers accurately, thus enabling the creation of an assistant tool in the future.

Auxiliary diagnostic method of Parkinson’s disease based on eye movement analysis in a virtual reality environment

Eye movement dysfunction is one of the non-motor symptoms of Parkinson’s disease (PD). An accurate analysis method for eye movement is an effective way to gain a deeper understanding of the nervous system function of PD patients. However, currently, there are only a few assistive methods available to help physicians conveniently and consistently assess patients suspected of having PD. To solve this problem, we proposed a novel visual behavioral analysis method using eye tracking to evaluate eye movement dysfunction in PD patients automatically. This method first provided a physician task simulation to induce PD-related eye movements in Virtual Reality (VR). Subsequently, we extracted eye movement features from recorded eye videos and applied a machine learning algorithm to establish a PD diagnostic model. Then, we collected eye movement data from 66 participants (including 22 healthy controls and 44 PD patients) in a VR environment for training and testing during visual tasks. Finally, on this relatively small dataset, the results reveal that the Support Vector Machine (SVM) algorithm has better classification potential.

Understanding the Role of Eye Movement Pattern and Consistency in Isolated English Word Reading Through Hidden Markov Modeling.

In isolated English word reading, readers have the optimal performance when their initial eye fixation is directed to the area between the beginning and word center, that is, the optimal viewing position (OVP). Thus, how well readers voluntarily direct eye gaze to this OVP during isolated word reading may be associated with reading performance. Using Eye Movement analysis with Hidden Markov Models, we discovered two representative eye movement patterns during lexical decisions through clustering, which focused at the OVP and the word center, respectively. Higher eye movement similarity to the OVP-focusing pattern predicted faster lexical decision time in addition to cognitive abilities and lexical knowledge. However, the OVP-focusing pattern was associated with longer isolated single letter naming time, suggesting conflicting visual abilities required for identifying isolated letters and multi-letter words. In contrast, in both word and pseudoword naming, although clustering did not reveal an OVP-focused pattern, higher consistency of the first fixation as measured in entropy predicted faster naming time in addition to cognitive abilities and lexical knowledge. Thus, developing a consistent eye movement pattern focusing on the OVP is essential for word orthographic processing and reading fluency. This finding has important implications for interventions for reading difficulties.

Classifying musical reading expertise by eye-movement analysis using machine learning

Classifying musical reading expertise by eye-movement analysis using machine learning

ORASIS-MAE Harnesses the Potential of Self-Learning from Partially Annotated Clinical Eye Movement Records

Self-supervised learning (SSL) has gained significant attention in the past decade for its capacity to utilize non-annotated datasets to learn meaningful data representations. In the medical domain, the challenge of constructing large annotated datasets presents a significant limitation, rendering SSL an ideal approach to address this constraint. In this study, we introduce a novel pretext task tailored to stimulus-driven eye movement data, along with a denoising task to improve the robustness against simulated eye tracking failures. Our proposed task aims to capture both the characteristics of the pilot (brain) and the motor (eye) by learning to reconstruct the eye movement position signal using up to 12.5% of the unmasked eye movement signal patches, along with the entire REMOBI target signal. Thus, the encoder learns a high-dimensional representation using a multivariate time series of length 8192 points, corresponding to approximately 40 s. We evaluate the learned representation on screening eight distinct groups of pathologies, including dyslexia, reading disorder, and attention deficit disorder, across four datasets of varying complexity and size. Furthermore, we explore various head architecture designs along with different transfer learning methods, demonstrating promising results with improvements of up to approximately 15%, leading to an overall macro F1 score of 61% and 61.5% on the Saccade and the Vergence datasets, respectively. Notably, our method achieves macro F1 scores of 64.7%, 66.1%, and 61.1% for screening dyslexia, reading disorder, and attention deficit disorder, respectively, on clinical data. These findings underscore the potential of self-learning algorithms in pathology screening, particularly in domains involving complex data such as stimulus-driven eye movement analysis.

Effects of various in-vehicle human–machine interfaces on drivers’ takeover performance and gaze pattern in conditionally automated vehicles

With the era of automated driving approaching, designing an effective and suitable human–machine interface (HMI) to present takeover requests (TORs) is critical to ensure driving safety. The present study conducted a simulated driving experiment to explore the effects of three HMIs (instrument panel, head-up display [HUD], and peripheral HMI) on takeover performance, simultaneously considering the TOR type (informative and generic TORs). Drivers’ eye movement data were also collected to investigate how drivers distribute their attention between the HMI and surrounding environment during the takeover process. The results showed that using the peripheral HMI to present TORs can shorten takeover time, and drivers rated this HMI as more useful and satisfactory than conventional HMIs (instrument panel and HUD). Eye movement analysis revealed that the peripheral HMI encourages drivers to spend more time gazing at the road ahead and less time gazing at the TOR information than the instrument panel and HUD, indicating a better gaze pattern for traffic safety. The HUD seemed to have a risk of capturing drivers’ attention, which resulted in an ‘attention tunnel,’ compared to the instrument panel. In addition, informative TORs were associated with better takeover performance and prompted drivers to spend less time gazing at rear-view mirrors than generic TORs. The findings of the present study can provide insights into the design and implementation of in-vehicle HMIs to improve the driving safety of automated vehicles.

Research on visual experience evaluation of fortress heritage landscape by integrating SBE–SD method and eye movement analysis

Urbanization has led to homogenizing heritage site landscapes, and the protective measures have become disconnected from public needs. Additionally, the complex and diverse overall characteristics of heritage sites and the uneven distribution of values across different areas are related to the lack of landscape experience assessment. The “subjective + objective” cognitive evaluation and visual perception framework that adopts the Scenic Beauty Estimation Procedure-Semantic Differential (SBE–SD) method and eye-tracking analysis can compensate for the limitations of a single evaluation method by integrating quantitative and qualitative analysis. This research takes the Yi’an Fortress in Zhangpu County, Fujian Province, as the object and examines the visual experiences of different areas and types of landscapes within the Yi’an Fortress. The findings reveal several key insights: (1) Significant differences were found in the landscape experiences of different areas within the heritage site. The visual experience score of the core building area of Yi’an Fortress is (1.01) > the heritage entrance area (0.897) > the residential area (0.841) > the natural ecological area (0.784), indicating that the natural ecological area should be the focus of future protection and development efforts, with a particular emphasis on enhancing the ‘landscape aesthetic’ and ‘landscape cultural’ aspects. (2) The landscape experience scores can be used to understand the reasons for the differences in participants’ experiences of different landscapes. The architectural heritage landscape of the Yi’an Fortress scored highest in the experience evaluation due to its superior performance in terms of ‘landscape form’ factors. The cultural decorative landscape scored next, while the garden greening landscape scored the lowest, due to their poorer performance in terms of ‘landscape form’ and ‘landscape aesthetics’. (3) The eye-tracking data was consistent with the results of the subjective evaluation, validating the “subjective + objective” cognitive evaluation and visual perception framework, employing the SBE–SD method and eye-tracking analysis is a scientific and effective method for assessing the visual experience of heritage landscapes. These results provide a scientific basis for the heritage planners and managers of the Yi’an fortress to improve the landscape environment, better meet public needs, and preserve the unique character of this vital cultural resource. Furthermore, this study offers a new research method and approach for the protection of other heritage landscapes.

Real-Time Deepfake Video Detection Using Eye Movement Analysis with a Hybrid Deep Learning Approach

Deepfake technology uses artificial intelligence to create realistic but false audio, images, and videos. Deepfake technology poses a significant threat to the authenticity of visual content, particularly in live-stream scenarios where the immediacy of detection is crucial. Existing Deepfake detection approaches have limitations and challenges, prompting the need for more robust and accurate solutions. This research proposes an innovative approach: combining eye movement analysis with a hybrid deep learning model to address the need for real-time Deepfake detection. The proposed hybrid deep learning model integrates two deep neural network architectures, MesoNet4 and ResNet101, to leverage their respective architectures’ strengths for effective Deepfake classification. MesoNet4 is a lightweight CNN model designed explicitly to detect subtle manipulations in facial images. At the same time, ResNet101 handles complex visual data and robust feature extraction. Combining the localized feature learning of MesoNet4 with the deeper, more comprehensive feature representations of ResNet101, our robust hybrid model achieves enhanced performance in distinguishing between manipulated and authentic videos, which cannot be conducted with the naked eye or traditional methods. The model is evaluated on diverse datasets, including FaceForensics++, CelebV1, and CelebV2, demonstrating compelling accuracy results, with the hybrid model attaining an accuracy of 0.9873 on FaceForensics++, 0.9689 on CelebV1, and 0.9790 on CelebV2, showcasing its robustness and potential for real-world deployment in content integrity verification and video forensics applications.

Eye Movements and Autonomic Regulation of Cognitive Activity during Reading in Adolescence. Part I. Functional “Cost” of Cognitive Activity when Reading Text from the Screen in Adolescence

Reading from the screen of an electronic device (ED) is a significant cognitive activity for adolescences, and its complexity affects visceral functions. We conducted an analysis of the heart rate variability (HRV) and eye movements (EM) in adolescents while they were reading complex text on an ED screen. The aim was to assess the functional state characteristics under these conditions and reveal the intensity (“cost”) associated with this activity The study involved 22 adolescences with an average age of 15 years (М = 15.46, SD = 0.44). Reading text from an ED screen in adolescents was associated with a high functional “cost,” characterized by a decrease in overall HRV, an increase in the tension index and heart rate. These changes indicate the functional tension of regulatory systems during cognitive activity. The study revealed varying levels of parafoveal processing involvement. For the majority of adolescents (86.4%), word-by-word reading and a low percentage of regressions (12.0%) were observed, suggesting developed average reading skill. However, a qualitative analysis of individual EM tracks indicated varying level of reading skill development among adolescents, possibly due to a limited vocabulary and a lack of understanding of syntax. Additionally, 40.9% of adolescents demonstrated poor text comprehension. Our results showed that reading remains a complex cognitive task for adolescents, despite the expectation that their reading skills should be well-developed and automated by this age. Individual analysis of HRV and EM in adolescents with varying levels of text comprehension during reading demonstrated different strategies of adaptive behavior and autonomic reactions when performing a complex cognitive task. The functional “cost” of information processing when reading text from the ED screen results from a combination of age-related and individual adaptation characteristics, language competence and the psycholinguistic complexity of the text.

Recognition of marine navigators’ workload based on eye movement features using bridge simulation

The human factor accounts for 80% of marine traffic accidents, either directly or indirectly. Workload significantly impacts the reliability of marine navigators’ behavior. Employing eye movement measurement as an objective assessment tool can effectively evaluate seafarers’ workload. This study conducts bridge simulation experiments to analyze eye movement features and investigate marine navigators’ workload levels across different tasks. By categorizing marine navigators’ workload into high and low level based on specific navigation duties. The study employs an independent sample T-test to select blink frequency, fixation time percentage, and pupil area standard deviation as key eye movement features for workload recognition. A marine navigators’ workload recognition model is developed utilizing the support vector machine (SVM) algorithm, which is subsequently trained and evaluated with experimental data. Results demonstrate a high recognition rate of 90.91% for marine navigators’ workload, indicating the SVM method’s accuracy in workload assessment based on eye movement features. These findings lay the foundation for the advancement of intelligent monitoring systems for marine navigators’ workload and the implementation of human–computer interaction active early warning technology and equipment.

Analysis of Eye Movements Reveals Longer Visual Saccades and Abnormal Preference for Social Images in Autism Spectrum Disorder: Longer Saccades and Fewer Fixations on Social Stimuli in Autism

Analysis of Eye Movements Reveals Longer Visual Saccades and Abnormal Preference for Social Images in Autism Spectrum Disorder: Longer Saccades and Fewer Fixations on Social Stimuli in Autism

The smell of paper or the shine of a screen? Students’ reading comprehension, text processing, and attitudes when reading on paper and screen

With the increasing prevalence of digital devices such as smartphones, tablets and e-readers, more and more reading is happening in digital formats – also in classrooms across the world. The present study focuses on lower secondary school students and their reading comprehension and attitudes toward reading on paper and screens. The study uses an innovative methodological approach were eye-tracking technology is used to observe ten carefully sampled eight grade students with different reading levels. The students read a selection of texts and answered questions from the Norwegian national reading assessment on comparable versions on paper and on screen. By analyzing eye movement data (reading transitions), including more than 25000 fixations, in combination with text reading comprehension outcomes, students' cued retrospective reporting from reading, and interview data we have obtained detailed and comprehensive data on students' reading comprehension, reading behavior, and reading experiences across different media. A key result is that reading on screen leads to more shallow processing and can hinder reading comprehension. Importantly, our results from the students' cued retrospective reporting of their eye tracking, showed that they were unaware of their reading behavior and didn't reflect much on reading in different media. These findings have implications for the increasing shift to digital learning environments in the educational context. It is important to recognize the difference between reading processes, and policymakers and practitioners cannot assume that these processes are the same across individuals and different delivery modes.