Eye Movement Patterns Research Articles

Design of a LabVIEW-Based Virtual Instrument for the Disabled

The eye is a prominent organ with complex functions that play a crucial role in our daily lives. The ability to perceive objects from varying distances and angles is vital for navigation, interaction, and understanding the world around us. The coordination of multiple muscles to move the eye allows us to focus on specific points of interest and gather visual information effectively. The concept of using the electrical signals generated by eye movements, known as the electrooculogram (EOG), has been studied in this research. EOG signals can be captured by placing electrodes around the eyes and measuring the potential differences generated as the eyes move. It was noted that a limited number of studies were carried out on EOG-based designs. However, continued research and development in this area have the potential to enhance the quality of life for individuals with disabilities and contribute to our understanding of both the visual system and human-computer interaction. This paper focuses on addressing the relatively underexplored areas of vertical and horizontal eye movements, as well as eye blinking, in the context of electrooculogram (EOG)-based designs. In this research, sharp EOG signals were observed in different eye movement patterns, and those signals were utilized to control assistive devices for individuals with mobility impairments with the LabVIEW software interface.

Prediction of time to contact under perceptual and contextual uncertainties.

Accurately estimating time to contact (TTC) is crucial for successful interactions with moving objects, yet it is challenging under conditions of sensory and contextual uncertainty, such as occlusion. In this study, participants engaged in a prediction motion task, monitoring a target that moved rightward and an occluder. The participants' task was to press a key when they predicted the target would be aligned with the occluder's right edge. We manipulated sensory uncertainty by varying the visible and occluded periods of the target, thereby modulating the time available to integrate sensory information and the duration over which motion must be extrapolated. Additionally, contextual uncertainty was manipulated by having a predictable and unpredictable condition, meaning the occluder either reliably indicated where the moving target would disappear or provided no such indication. Results showed differences in accuracy between the predictable and unpredictable occluder conditions, with different eye movement patterns in each case. Importantly, the ratio of the time the target was visible, which allows for the integration of sensory information, to the occlusion time, which determines perceptual uncertainty, was a key factor in determining performance. This ratio is central to our proposed model, which provides a robust framework for understanding and predicting human performance in dynamic environments with varying degrees of uncertainty.

Understanding Human Cognition Through Computational Modeling.

One important goal of cognitive science is to understand the mind in terms of its representational and computational capacities, where computational modeling plays an essential role in providing theoretical explanations and predictions of human behavior and mental phenomena. In my research, I have been using computational modeling, together with behavioral experiments and cognitive neuroscience methods, to investigate the information processing mechanisms underlying learning and visual cognition in terms of perceptual representation and attention strategy. In perceptual representation, I have used neural network models to understand how the split architecture in the human visual system influences visual cognition, and to examine perceptual representation development as the results of expertise. In attention strategy, I have developed the Eye Movement analysis with Hidden Markov Models method for quantifying eye movement pattern and consistency using both spatial and temporal information, which has led to novel findings across disciplines not discoverable using traditional methods. By integrating it with deep neural networks (DNN), I have developed DNN+HMM to account for eye movement strategy learning in human visual cognition. The understanding of the human mind through computational modeling also facilitates research on artificial intelligence's (AI) comparability with human cognition, which can in turn help explainable AI systems infer humans' belief on AI's operations and provide human-centered explanations to enhance human-AI interaction and mutual understanding. Together, these demonstrate the essential role of computational modeling methods in providing theoretical accounts of the human mind as well as its interaction with its environment and AI systems.

Using Synchronized Eye Movements to Predict Attention in Online Video Learning

Concerns persist about attentional engagement in online learning. The inter-subject correlation of eye movements (ISC) has shown promise as an accessible and effective method for attention assessment in online learning. This study extends previous studies investigating ISC of eye movements in online learning by addressing two research questions. Firstly, can ISC predict students’ attentional states at a finer level beyond a simple dichotomy of attention states (e.g., attending and distracted states)? Secondly, whether learners’ learning styles affect ISC’s prediction rate of attention assessment in video learning? Previous studies have shown that learners of different learning styles have different eye movement patterns when viewing static materials. However, limited research has explored the impact of learning styles on viewing patterns in video learning. An eye tracking experiment with participants watching lecture videos demonstrated a connection between ISC and self-reported attention states at a finer level. We also demonstrated that learning styles did not significantly affect ISC’s prediction rate of attention assessment in video learning, suggesting that ISC of eye movements can be effectively used without considering learners’ learning styles. These findings contribute to the ongoing discourse on optimizing attention assessment in the evolving landscape of online education.

In the Eyes of the Future: Eye Movement during Near and Distant Future Thinking.

Research has suggested that near future events are typically viewed from a first-person (an own-eyes, also known as field) perspective while distant future events are typically viewed from a third-person (an observer) perspective. We investigated whether these distinct mental perspectives would be accompanied by distinct eye movement activities. We invited participants to imagine near and distant future events while their eye movements (i.e., scan path) were recorded by eye-tracking glasses. Analysis demonstrated fewer but longer fixations for near future thinking than for distant future thinking. Analysis also demonstrated more "field" mental visual perspective responses for near than for distant future thinking. The long fixations during near future thinking may mirror a mental visual exploration involving processing of a more complex visual representation compared with distant future thinking. By demonstrating how near future thinking triggers both "field" responses and long fixations, our study demonstrates how the temporality of future thinking triggers both distinct mental imagery and eye movement patterns.

Insights into the relationship between eye movements and personality traits in restricted visual fields

Previous studies have suggested behavioral patterns, such as visual attention and eye movements, relate to individual personality traits. However, these studies mainly focused on free visual tasks, and the impact of visual field restriction remains inadequately understood. The primary objective of this study is to elucidate the patterns of conscious eye movements induced by visual field restriction and to examine how these patterns relate to individual personality traits. Building on previous research, we aim to gain new insights through two behavioral experiments, unraveling the intricate relationship between visual behaviors and individual personality traits. As a result, both Experiment 1 and Experiment 2 revealed differences in eye movements during free observation and visual field restriction. Particularly, simulation results based on the analyzed data showed clear distinctions in eye movements between free observation and visual field restriction conditions. This suggests that eye movements during free observation involve a mixture of conscious and unconscious eye movements. Furthermore, we observed significant correlations between conscious eye movements and personality traits, with more pronounced effects in the visual field restriction condition used in Experiment 2 compared to Experiment 1. These analytical findings provide a novel perspective on human cognitive processes through visual perception.

Trouble Sleeping?

For individuals of all ages, receiving high-quality sleep is essential for health and well-being. SomaSleep, a groundbreaking sleep mask by Somalytics which integrates carbon-nanotube paper composite (CPC) capacitive sensors, tracks eye movement and sleep patterns, eliminating the need for sleep clinics and providing users with real-time sleep data. Despite this revolutionary technology, concerns pertaining to cost, accessibility, and safety must be addressed for this technology to be inclusive for all users. Moving forward, this technology has numerous applications in societal and healthcare frameworks.

Eye movements reflect active statistical learning.

What is the link between eye movements and sensory learning? Although some theories have argued for an automatic interaction between what we know and where we look that continuously modulates human information gathering behavior during both implicit and explicit learning, there exists limited experimental evidence supporting such an ongoing interplay. To address this issue, we used a visual statistical learning paradigm combined with a gaze-contingent stimulus presentation and manipulated the explicitness of the task to explore how learning and eye movements interact. During both implicit exploration and explicit visual learning of unknown composite visual scenes, spatial eye movement patterns systematically and gradually changed in accordance with the underlying statistical structure of the scenes. Moreover, the degree of change was directly correlated with the amount and type of knowledge the observers acquired. This suggests that eye movements are potential indicators of active learning, a process where long-term knowledge, current visual stimuli and an inherent tendency to reduce uncertainty about the visual environment jointly determine where we look.

Distinct eye movement patterns to complex scenes in Alzheimer's disease and Lewy body disease.

Alzheimer's disease (AD) and Lewy body disease (LBD), the two most common causes of neurodegenerative dementia with similar clinical manifestations, both show impaired visual attention and altered eye movements. However, prior studies have used structured tasks or restricted stimuli, limiting the insights into how eye movements alter and differ between AD and LBD in daily life. We aimed to comprehensively characterize eye movements of AD and LBD patients on naturalistic complex scenes with broad categories of objects, which would provide a context closer to real-world free viewing, and to identify disease-specific patterns of altered eye movements. We collected spontaneous viewing behaviors to 200 naturalistic complex scenes from patients with AD or LBD at the prodromal or dementia stage, as well as matched control participants. We then investigated eye movement patterns using a computational visual attention model with high-level image features of object properties and semantic information. Compared with matched controls, we identified two disease-specific altered patterns of eye movements: diminished visual exploration, which differentially correlates with cognitive impairment in AD and with motor impairment in LBD; and reduced gaze allocation to objects, attributed to a weaker attention bias toward high-level image features in AD and attributed to a greater image-center bias in LBD. Our findings may help differentiate AD and LBD patients and comprehend their real-world visual behaviors to mitigate the widespread impact of impaired visual attention on daily activities.

Auxiliary Diagnosis of Children with Attention-Deficit/Hyperactivity Disorder: An Eye-Tracking Study with Novel Digital Biomarkers.

Attention-deficit/hyperactivity disorder (ADHD) is a common neurodevelopmental disorder in school-aged children. The lack of objective biomarkers for ADHD often results in missed diagnoses or misdiagnoses, which lead to inappropriate or delayed interventions. Eye-tracking technology provides an objective method to assess children's neuropsychological behavior. The purpose of this research was to develop an objective and reliable auxiliary diagnostic system for ADHD using eye-tracking technology. This system would be valuable for screening for ADHD in schools and communities and may help identify objective biomarkers for the clinical diagnosis of ADHD. We conducted a case-control study of children with ADHD and typically developing (TD) children. We designed an eye-tracking assessment paradigm based on the core cognitive deficits of ADHD and extracted various digital biomarkers that represented participant behaviors. These biomarkers and developmental patterns were compared between the ADHD and TD groups. Machine learning (ML) was implemented to validate the ability of the extracted eye-tracking biomarkers to predict ADHD. The performance of the ML models was evaluated using k-fold cross-validation. We recruited 216 participants, of whom 94 were children with ADHD and 122 were TD children. The ADHD group showed significantly poorer performance (for accuracy and completion time) than the TD group in the pro-, anti-, and delayed-saccade tasks. Additionally, there were significant group differences in digital biomarkers, such as pupil diameter fluctuation, regularity of gaze trajectory, and fixations on uninterested areas. Although the accuracy and task completion speed of the ADHD group increased over time, their eye movement patterns remained irregular. The 5-6-year-old TD group outperformed the 9-10-year-old ADHD group, and this difference remained relatively stable over time, which indicated that the ADHD group followed a unique developmental pattern. The ML model was effective in discriminating the groups, achieving an area under the curve of 0.965 and an accuracy of 0.908. The eye-tracking biomarkers proposed in this study effectively identified differences in various aspects of eye movement patterns between the ADHD and TD groups. In addition, the ML model constructed using these digital biomarkers achieved high accuracy and reliability in identifying ADHD. Our system can facilitate early screening for ADHD in schools and communities and provide clinicians with objective biomarkers as a reference. This study has been registrated at Chinese Clinical Trail Registry(No. ChiCTR2400087697).

Toward viewing behavior for aerial scene categorization

Previous work has demonstrated similarities and differences between aerial and terrestrial image viewing. Aerial scene categorization, a pivotal visual processing task for gathering geoinformation, heavily depends on rotation-invariant information. Aerial image-centered research has revealed effects of low-level features on performance of various aerial image interpretation tasks. However, there are fewer studies of viewing behavior for aerial scene categorization and of higher-level factors that might influence that categorization. In this paper, experienced subjects’ eye movements were recorded while they were asked to categorize aerial scenes. A typical viewing center bias was observed. Eye movement patterns varied among categories. We explored the relationship of nine image statistics to observers’ eye movements. Results showed that if the images were less homogeneous, and/or if they contained fewer or no salient diagnostic objects, viewing behavior became more exploratory. Higher- and object-level image statistics were predictive at both the image and scene category levels. Scanpaths were generally organized and small differences in scanpath randomness could be roughly captured by critical object saliency. Participants tended to fixate on critical objects. Image statistics included in this study showed rotational invariance. The results supported our hypothesis that the availability of diagnostic objects strongly influences eye movements in this task. In addition, this study provides supporting evidence for Loschky et al.’s (Journal of Vision, 15(6), 11, 2015) speculation that aerial scenes are categorized on the basis of image parts and individual objects. The findings were discussed in relation to theories of scene perception and their implications for automation development.

Raters’ scoring process in assessment of interpreting: an empirical study based on eye tracking and retrospective verbalisation

ABSTRACT Human raters’ assessment of interpreting is a complex process. Previous researchers have mainly relied on verbal reports to examine this process. To advance our understanding, we conducted an empirical study, collecting raters’ eye-movement and retrospection data in a computerised interpreting assessment in which three groups of raters (n = 35) used an analytic rubric to assess 12 English-to-Chinese consecutive interpretations. We examined how the raters interacted with the source text, the rating scale, and the audio player displayed on the computer screen when they were assessing. We found that a) the source text and the rating scale were competing for the raters’ visual attention, with the former attracting more attention than the latter across the rater groups; b) when the raters were consulting the rating scale, they fixated less frequently on the sub-scale of target language quality than the other two sub-scales; c) the rater groups did not seem to exhibit substantially discrepant gazing behaviours overall, although there emerged different eye-movement patterns concerning certain sub-scales; and d) the raters utilised an array of strategies and shortcuts to facilitate their assessment. We discuss these findings in relation to rater training and validation of score meaning for interpreting assessment.

Intra-V1 functional networks and classification of observed stimuli.

Previous studies suggest that co-fluctuations in neural activity within V1 (measured with fMRI) carry information about observed stimuli, potentially reflecting various cognitive mechanisms. This study explores the neural sources shaping this information by using different fMRI preprocessing methods. The common response to stimuli shared by all individuals can be emphasized by using inter-subject correlations or de-emphasized by deconvolving the fMRI with hemodynamic response functions (HRFs) before calculating the correlations. The latter approach shifts the balance towards participant-idiosyncratic activity. Here, we used multivariate pattern analysis of intra-V1 correlation matrices to predict the Level or Shape of observed Navon letters employing the types of correlations described above. We assessed accuracy in inter-subject prediction of specific conjunctions of properties, and attempted intra-subject cross-classification of stimulus properties (i.e., prediction of one feature despite changes in the other). Weight maps from successful classifiers were projected onto the visual field. A control experiment investigated eye-movement patterns during stimuli presentation. All inter-subject classifiers accurately predicted the Level and Shape of specific observed stimuli. However, successful intra-subject cross-classification was achieved only for stimulus Level, but not Shape, regardless of preprocessing scheme. Weight maps for successful Level classification differed between inter-subject correlations and deconvolved correlations. The latter revealed asymmetries in visual field link strength that corresponded to known perceptual asymmetries. Post-hoc measurement of eyeball fMRI signals did not find differences in gaze between stimulus conditions, and a control experiment (with derived simulations) also suggested that eye movements do not explain the stimulus-related changes in V1 topology. Our findings indicate that both inter-subject common responses and participant-specific activity contribute to the information in intra-V1 co-fluctuations, albeit through distinct sub-networks. Deconvolution, that enhances subject-specific activity, highlighted interhemispheric links for Global stimuli. Further exploration of intra-V1 networks promises insights into the neural basis of attention and perceptual organization.

Stay Alert: Drowsiness Detection with IoT Technology

The "Drowsiness Detection using IoT with Eye Blink Sensor and Buzzer" project aims to address the critical issue of driver fatigue and drowsiness, a major cause of road accidents. Leveraging Internet of Things (IoT) technologies and a combination of eye blink sensors and a buzzer, this system provides a real-time solution for monitoring and alerting drivers when signs of drowsiness are detected. The project focuses on enhancing road safety by preventing accidents caused by driver fatigue Drowsiness detection is a critical aspect of ensuring safety, particularly in domains such as transportation where fatigue-related accidents pose significant risks. In recent years, the proliferation of Internet of Things (IoT) devices has enabled innovative approaches to drowsiness detection systems. This abstract outlines a novel framework for drowsiness detection utilizing IoT technologies. Our proposed system integrates various sensors, including facial recognition cameras, heart rate monitors, and accelerometers, into a networked IoT infrastructure. These sensors continuously monitor key physiological and behavioral indicators associated with drowsiness, such as eye movement patterns, blink frequency, heart rate variability, and head position changes.

Autistic Individuals Do Not Alter Visual Processing Strategy During Encoding Versus Recognition of Faces: A Hidden Markov Modeling Approach.

Visual face recognition-the ability to encode, discriminate, and recognize the faces of others-is fundamentally supported by eye movements and is a common source of difficulty for autistic individuals. We aimed to evaluate how visual processing strategies (i.e., eye movement patterns) directly support encoding and recognition of faces in autistic and neurotypical (NT) individuals. We used a hidden Markov modeling approach to evaluate the spatiotemporal dynamics of eye movements in autistic (n = 15) and neurotypical (NT) adolescents (n = 17) during a face identity recognition task. We discovered distinct eye movement patterns among all participants, which included a focused and exploratory strategy. When evaluating change in visual processing strategy across encoding and recognition phases, autistic individuals did not shift their eye movement patterns like their NT peers, who shifted to a more exploratory visual processing strategy during recognition. These findings suggest that autistic individuals do not modulate their visual processing strategy across encoding and recognition of faces, which may be an indicator of less efficient face processing.

Exploring the Connection between Eye Movement Parameters and Eye Fatigue

Eye fatigue, a prominent symptom of computer vision syndrome (CVS), has gained significant attention in various domains due to the increasing diversification of electronic display devices and their widespread usage scenarios. The COVID-19 pandemic has further intensified the reliance on these devices, leading to prolonged screen time. This study aimed to investigate the effectiveness of utilizing eye movement patterns in discriminating fatigue during the usage of electronic display devices. Eye movement data was collected from subjects experiencing different levels of fatigue, and their fatigue levels were recorded using the T/CVIA-73-2019 scale. The analysis revealed that features related to the pupils demonstrated a high level of confidence and reliability in distinguishing fatigue, especially related to pupil size. However, features associated with fixations, such as fixation duration and frequency, did not significantly contribute to fatigue discrimination. Furthermore, the study explored the influence of subjective awareness on fatigue discrimination. By modifying the experimental settings and considering the subjects’ subjective perception, it was observed that individual consciousness and self-awareness played a crucial role in fatigue discrimination. The implications of these findings extend beyond the field of computer vision syndrome, offering potential applications in developing interventions and strategies to alleviate eye fatigue and promote eye health among individuals who extensively use electronic display devices.

SARS-CoV-2 Infection Impairs Oculomotor Functions: A Longitudinal Eye-tracking Study.

Although Severe Acute Respiratory Syndrome Coronavirus 2 infection (SARS-CoV-2) is primarily recognized as a respiratory disease, mounting evidence suggests that it may lead to neurological and cognitive impairments. The current study used three eye-tracking tasks (free-viewing, fixation, and smooth pursuit) to assess the oculomotor functions of mild infected cases over six months with symptomatic SARS-CoV-2 infected volunteers. Fifty symptomatic SARS-CoV-2 infected, and 24 self-reported healthy controls completed the eye-tracking tasks in an initial assessment. Then, 45, and 40 symptomatic SARS-CoV-2 infected completed the tasks at 2- and 6-months post-infection, respectively. In the initial assessment, symptomatic SARS-CoV-2 infected exhibited impairments in diverse eye movement metrics. Over the six months following infection, the infected reported overall improvement in health condition, except for self-perceived mental health. The eye movement patterns in the free-viewing task shifted toward a more focal processing mode and there was no significant improvement in fixation stability among the infected. A linear discriminant analysis shows that eye movement metrics could differentiate the infected from healthy controls with an accuracy of approximately 62%, even 6 months post-infection. These findings suggest that symptomatic SARSCoV- 2 infection may result in persistent impairments in oculomotor functions, and the employment of eye-tracking technology can offer valuable insights into both the immediate and long-term effects of SARS-CoV-2 infections. Future studies should employ a more balanced research design and leverage advanced machine-learning methods to comprehensively investigate the impact of SARSCoV- 2 infection on oculomotor functions.

Can Perceivers Differentiate Intense Facial Expressions? Eye Movement Patterns.

Recent research on intense real-life faces has shown that although there was an objective difference in facial activities between intense winning faces and losing faces, viewers failed to differentiate the valence of such expressions. In the present study, we explored whether participants could perceive the difference between intense positive facial expressions and intense negative facial expressions in a forced-choice response task using eye-tracking techniques. Behavioral results showed that the recognition accuracy rate for intense facial expressions was significantly above the chance level. For eye-movement patterns, the results indicated that participants gazed more and longer toward the upper facial region (eyes) than the lower region (mouth) for intense losing faces. However, the gaze patterns were reversed for intense winning faces. The eye movement pattern for successful differentiation trials did not differ from failed differentiation trials. These findings provided preliminary evidence that viewers can utilize intense facial expression information and perceive the difference between intense winning faces and intense losing faces produced by tennis players in a forced-choice response task.

Trial-level fluctuations in pupil dilation at encoding reflect strength of relational binding.

Eye-tracking methodologies have revealed that eye movements and pupil dilations are influenced by our previous experiences. Dynamic fluctuations in pupil size during learning reflect in part the formation of memories for learned information, while viewing behavior during memory testing is influenced by memory retrieval and drawn to previously learned associations. However, no study to date has linked fluctuations in pupil dilation at encoding to the magnitude of viewing behavior at test. The current investigation involved monitoring eye movements both in single item recognition and relational recognition tasks. In the item task, all faces were presented with the same background scene and memory for faces was subsequently tested, whereas in the relational task each face was presented with its own unique background scene and memory for the face-scene association was subsequently tested. Pupil size changes during encoding predicted the magnitude of preferential viewing during test, as well as future recognition accuracy. These effects emerged only in the relational task, but not in the item task, and were replicated in an additional experiment in which stimulus luminance was more tightly controlled. A follow-up experiment and additional analyses ruled out differences in orienting instructions or number of fixations to the encoding display as explanations of the observed effects. The results shed light on the links between pupil dilation, memory encoding, and eye movement patterns during recognition and suggest that trial-level fluctuations in pupil dilation during encoding reflect relational binding of items to their context rather than general memory formation or strength. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Human-Computer Interaction Corporate Law Education for Directors: A Machine Learning Approach

With the formation and growth of the company, corporate law is continuously established and enhanced. It significantly contributes to the company’s healthy growth and brings business operations inside the legal framework. As a result, corporate law education is crucial for employees, particularly directors. Fundamentally, corporate law education is a learning process. The metaverse period has increased learners’ needs for learning environments, and human-computer interaction technology will offer all-encompassing support for the smart learning environment. The board of directors needs to adequately monitor each director’s learning level as they study corporate law, which will inevitably be detrimental to the company’s long-term growth. The most efficient method to address this issue is to use eye movement data to mine different eye movement patterns, followed by an analysis of the learning state of corporate law of directors. The scanning path analysis is used to examine the similarities and differences of the directors’ eye movement behaviors during the study of corporate law to enhance the state of corporate law learning. However, the learning status of corporate law cannot be determined only by the eye tracking of directors. We employ the convolutional neural networks (CNN) -based emotion recognition model to provide the directors constructive criticism about their learning state and offer suggestions for the learning mode. The experimental results demonstrate that time series-based eye movement pattern mining can identify directors’ viewing habits, and clustering can reveal different learning strategies that can be used to evaluate directors’ corporate law learning status. Additionally, the CNN-based emotion recognition model experiment also shows that the established model has an accuracy of 97.0035% and an F1 of 0.9412 in the CASIA-FaceV5 dataset, which helps evaluate the emotions of directors when learning company law.