Eye Movement Characteristics Research Articles

Effect of Stimulus Regularities on Eye Movement Characteristics

Humans have the unique ability to discern spatial and temporal regularities in their surroundings. However, the effect of learning these regularities on eye movement characteristics has not been studied enough. In the present study, we investigated the effect of the frequency of occurrence and the presence of common chunks in visual images on eye movement characteristics like the fixation duration, saccade amplitude and number, and gaze number across sequential experimental epochs. The participants had to discriminate the patterns presented in pairs as the same or different. The order of pairs was repeated six times. Our results show an increase in fixation duration and a decrease in saccade amplitude in the sequential epochs, suggesting a transition from ambient to focal information processing as participants acquire knowledge. This transition indicates deeper cognitive engagement and extended analysis of the stimulus information. Interestingly, contrary to our expectations, the saccade number increased, and the gaze number decreased. These unexpected results might imply a reduction in the memory load and a narrowing of attentional focus when the relevant stimulus characteristics are already determined.

Eye-movement-prompted large image captioning model

Pretrained large vision-language models have shown outstanding performance on the task of image captioning. However, owing to the insufficient decoding of image features, existing large models sometimes lose important information, such as objects, scenes, and their relationships. In addition, the complex ”black-box” nature of these models makes their mechanisms difficult to explain. Research shows that humans learn richer representations than machines do, which inspires us to improve the accuracy and interpretability of large image captioning models by combining human observation patterns. We built a new dataset, called saliency in image captioning (SIC), to explore relationships between human vision and language representation. One thousand images with rich context information were selected as image data of SIC. Each image was annotated with five caption labels and five eye-movement labels. Through analysis of the eye-movement data, we found that humans efficiently captured comprehensive information for image captioning during their observations. Therefore, we propose an eye-movement-prompted large image captioning model, which is embedded with two carefully designed modules: the eye-movement simulation module (EMS) and the eye-movement analyzing module (EMA). EMS combines the human observation pattern to simulate eye-movement features, including the positions and scan paths of eye fixations. EMA is a graph neural network (GNN) based module, which decodes graphical eye-movement data and abstracts image features as a directed graph. More accurate descriptions can be predicted by decoding the generated graph. Extensive experiments were conducted on the MS-COCO and NoCaps datasets to validate our model. The experimental results showed that our network was interpretable, and could achieve superior results compared with state-of-the-art methods, i.e., 84.2% BLEU-4 and 145.1% CIDEr-D on MS-COCO Karpathy test split, indicating its strong potential for use in image captioning.

Detecting five-pattern personality traits using eye movement features for observing emotional faces.

The five-pattern personality traits rooted in the theory of traditional Chinese medicine (TCM) have promising prospects for clinical application. However, they are currently assessed using a self-report scale, which may have certain limitations. Eye tracking technology, with its non-intrusive, objective, and culturally neutral characteristics, has become a powerful tool for revealing individual cognitive and emotional processes. Therefore, applying this technology for personality assessment is a promising approach. In this study, participants observed five emotional faces (anger, happy, calm, sad, and fear) selected from the Chinese Facial Affective Picture System. Utilizing artificial intelligence algorithms, we evaluated the feasibility of automatically identifying different traits of the five-pattern personality traits from participants' eye movement patterns. Based on the analysis of five supervised learning algorithms, we draw the following conclusions: The Lasso feature selection method and Logistic Regression achieve the highest prediction accuracy for most of the traits (TYa, SYa, SYi, TYi). This study develops a framework for predicting five-pattern personality traits using eye movement behavior, offering a novel approach for personality assessment in TCM.

Related work analysis for determination of fatigue state based on eye movements monitoring

We have reviewed theoretical background of detecting functional state of fatigue based on the strategy of eye movements. Also, modern methods for assessing eye movements were considered. Based on our literature review, we can conclude that nowadays there are multitude numerical characteristics of eye movements, the dynamics of which can hypothetically make it possible to assess degree of fatigue. However, there are still no proposals for a method for determining the degree of fatigue based on an analysis of the strategy of eye movements. In this regard, according to the concepts of static and dynamic vision, it is proposed to consider the shift in the numerical characteristics of eye movements towards characteristics that reflect the strategy of dynamic vision as evidence of fatigue.

Eye Movement Evaluation of Pedestrians' Mobile Phone Usage at Street Crossings

Vision is one of the most important human senses, accounting for most of the external information pedestrians receive while crossing the street. However, distracted mobile phone usage during street crossing consumes pedestrians’ cognitive resources and diverts their visual attention. As a result, pedestrians may be unable to fully concentrate on observing the traffic environment and effectively planning their crossing path and behavior. This study evaluated the effect of pedestrian behavioral activities at street crossings on eye-movement (EM) characteristics. The crossing tasks were natural behavior, voice call, text messaging, and listening to music. The tasks were further categorized as simple or complex. A total of 29 participants were recruited in Nanjing: 18 males (62.1%) and 11 females (37.9%) with an average age of 23.59 years (SD = 2.44). The Friedman test was used to analyze differences in saccade frequency, fixation time, browsing number, and browsing time across different scenarios. Text messaging had the most significant impact on pedestrians’ EM characteristics, followed by voice call; music listening had a relatively weaker effect. Secondary task difficulty influenced the percentage of browsing, viewing, and to some extent gaze time. On the other hand, music rhythm and style only partially influenced the percentage of gaze and gaze time. Mobile phones substantially affected pedestrians’ EM characteristics and attention allocation for the same level of secondary task difficulty. These findings contribute to a better understanding of pedestrians’ visual characteristics under distracted mobile phone usage conditions and provide valuable insights for developing appropriate measures to enhance pedestrian safety.

A Quantitative Method of Assessing the Hazard Perception of Drivers in Human–Machine Codriving Vehicles

Hazard perception, as a dynamic indicator, intuitively characterizes drivers’ ability to detect potential hazard events in complex driving environments. However, current quantitative research on the hazard perceptions of drivers in the human–machine codriving phase is lacking. A total of 30 drivers were recruited to carry out a driving simulator experiment, and a total of 8 risky driving scenarios were experienced. The physiological characteristics and eye movement characteristics of the drivers were collected. An expert evaluation method was selected to quantify the situational hazard degree (SHD). The questionnaire was used to collect the drivers’ subjective hazard evaluations (SHEs) of the scenarios. Paired samples t test and correlation analysis were adopted to screen the evaluation indexes of the drivers’ hazard perceptions. A gray near-optimal comprehensive evaluation model was established to quantify the human–machine codrivers’ hazard perceptions in various risky driving scenarios. The results show that the drivers’ SHEs increased with the SHD. The standard deviation of heart rate, mean pupil diameter, and mean fixation duration can be used as evaluation indexes for the human–machine codrivers’ hazard perceptions. The proposed quantitative method is expected to shift the development of human–machine codrivers’ hazard perception from qualitative analysis to quantitative calculation. The research results can provide a theoretical basis for increasing safety in the human–machine codriving phase, improving road safety and reducing the incidence of accidents.

Auxiliary diagnostic method of Parkinson’s disease based on eye movement analysis in a virtual reality environment

Eye movement dysfunction is one of the non-motor symptoms of Parkinson’s disease (PD). An accurate analysis method for eye movement is an effective way to gain a deeper understanding of the nervous system function of PD patients. However, currently, there are only a few assistive methods available to help physicians conveniently and consistently assess patients suspected of having PD. To solve this problem, we proposed a novel visual behavioral analysis method using eye tracking to evaluate eye movement dysfunction in PD patients automatically. This method first provided a physician task simulation to induce PD-related eye movements in Virtual Reality (VR). Subsequently, we extracted eye movement features from recorded eye videos and applied a machine learning algorithm to establish a PD diagnostic model. Then, we collected eye movement data from 66 participants (including 22 healthy controls and 44 PD patients) in a VR environment for training and testing during visual tasks. Finally, on this relatively small dataset, the results reveal that the Support Vector Machine (SVM) algorithm has better classification potential.

AI can see you: Machiavellianism and extraversion are reflected in eye-movements.

Recent studies showed an association between personality traits and individual patterns of visual behaviour in laboratory and other settings. The current study extends previous research by measuring multiple personality traits in natural settings; and by comparing accuracy of prediction of multiple machine learning algorithms. Adolescent participants (N = 35) completed personality questionnaires (Big Five Inventory and Short Dark Triad Questionnaire) and visited an interactive museum while their eye movements were recorded with head-mounted eye tracking. To predict personality traits the eye-movement data was analysed using eight machine-learning methods: Random Forest, Adaboost, Naive Bayes, Support Vector Machine, Logistic Regression, k Nearest Neighbours, Decision Tree and a three-layer Perceptron. Extracted eye movement features introduced to machine learning algorithms predicted personality traits with above 33% chance accuracy (34%-48%). This result is comparable to previous ecologically valid studies, but lower than in laboratory-based research. Better prediction was achieved for Machiavellianism and Extraversion compared to other traits (10 and 9 predictions above the chance level by different algorithms from different parts of the recording). Conscientiousness, Narcissism and Psychopathy were not reliably predicted from eye movements. These differences in predictability across traits might be explained by differential activation of different traits in different situations, such as new vs. familiar, exciting vs. boring, and complex vs. simple settings. In turn, different machine learning approaches seem to be better at capturing specific gaze patterns (e.g. saccades), associated with specific traits evoked by the situation. Further research is needed to gain better insights into trait-situation-algorithm interactions.

Exploring the causes of frequent accidents at highway tunnel exits: Coupling analysis of the slack effect and white hole effect in extra-long tunnels

In this paper, the causes of frequent traffic accidents at the exits of highway tunnels are investigated. This study recruited 34 subjects to conduct vehicle experiments using an illumination meter, eye tracker, Smarter Eye and OBD to measure the illumination, eye movement characteristics, driving trajectory, and speed while driving in a tunnel exit. A questionnaire was used to investigate drivers’ subjective feelings while driving. The change trend of each indicator in the area near the tunnel exit was analyzed, and the relationships among the light environment, driver physiology, driving behavior, and driver psychology were explored. The psychological and physiological inconsistency phenomenon and abnormal acceleration behavior were observed at the tunnel exit, and the slack effect was verified. As the tunnel exit area is driven through, with the rapid increase in illumination, the rate of change of the driver’s pupil area increases, and the distance between the vehicle and the tunnel sidewall increases. It is unreasonable that in a tunnel exit with a poor traffic environment, some drivers experience psychological slack and tend to increase their speed and accelerate out of the tunnel. The slack effect is observed in most drivers, approximately 79.4% of the total. The white hole effect interferes with drivers’ vision and reduces their ability to obtain traffic information. The slack effect reduces drivers’ attention and increases vehicle speed. A coupling effect exists between the white hole and slack effects, which together threaten traffic safety at the tunnel exit.

Акцентуации характера у пациенток эстетической медицины как факторы восприятия своей внешности и основания для выбора стратегии взаимодействия при оказании услуг

<p><strong>Introduction</strong>. Perceptual processing of one's own face plays a role in reducing self-identity and sense of self. Despite extensive research into the ability to recognize other people's faces, self-face perception remains an understudied area. In this perceptual processing, our faces are influenced by the medical and psychophysiological conditions of the individual. For example, accentuation of character can affect the perception of one’s own face. Understanding these processes is important for developing a patient-centered approach, personalized strategies for supporting and treating patients with different characterological characteristics who seek aesthetic medicine services. <strong>Objective</strong>. The purpose of this study was to identify the relationship between character accentuations and perception of one’s own face. <strong>Methods.</strong> The study involved 86 women aged from 20 to 65 years (mean age 39±13 years). Oculomotor activity was recorded using a Gazepoint HD 150 Hz eye tracker. To measure personality accentuations, a questionnaire developed by K. Leonhard and G. Shmishek in 1970 was used. <strong>Results.</strong> When examining one's own face, the characteristics of eye movements reflect the patients' attitude towards themselves, associated with certain psychological characteristics of the individual. Thus, women with dysthymic personality traits tend to scrutinize their appearance, while women with excitable personality traits tend to focus on the positive aspects of visual information. Pedantry correlates with the number of returns and fixations on attractive features, which may reflect a desire to maintain a positive sense of self. Exaltation is associated with the length of the scanning path, the number of saccades on attractive features, which also reflects a positive self-perception. The personal profiles of the study participants are characterized by accentuated traits, among which the hyperthymic and exalted type of character accentuation predominates.The patients were divided into subgroups depending on which areas of the face they looked at more often and for longer - attractive or problem areas of the face. The study showed that, depending on their predominant personality traits, patients either looked primarily at attractive features or focused on facial flaws. <strong>Conclusion</strong>. Based on the data presented, it was concluded that it is advisable to carry out express diagnostics of the psychological characteristics of patients at the initial appointment in a cosmetology clinic. An initial psychological examination of the patient provides important information about the characterological characteristics of the individual and the perception of attractive features and flaws of his face, which will allow the doctor to predict difficulties that may arise during further medical interventions, as well as improve the level of mutual understanding and communication between patients and the doctor.</p>

Openness to experience predicts eye movement behavior during scene viewing

Individuals’ abilities to perform goal-directed spatial deployments of attention are distinguishable from their broader preferences for how they use spatial attention when circumstances do not compel a specific deployment strategy. Although these preferences are likely to play a major role in how we interact with the visual world during daily life, they remain relatively understudied. This exploratory study investigated two key questions about these preferences: firstly, are individuals consistent in their preferences for how they deploy their spatial attention when making shifts of attention versus adopting an attentional breadth? Secondly, which other factors are associated with these preferences? Across two experiments, we measured how participants preferred to deploy both attentional breadth (using an adapted Navon task) and eye movements (using a free-viewing task). We also measured participants’ working memory capacities (Experiment 1), and their personalities and world beliefs (Experiment 2). In both experiments, there were consistent individual differences in preference for attentional breadth and eye movement characteristics, but these two kinds of preference were unrelated to each other. Working memory capacity was not linked to these preferences. Conversely, the personality trait of Openness to Experience robustly predicted two aspects of eye movement behavior preference, such that higher levels of Openness predicted smaller saccades and shorter scan paths. This suggests that personality dimensions may predict preferences for more absorbed engagement with visual information. However, it appears that individuals’ preferences for shifts of attention during scene viewing do not necessarily relate to the breadth of attention they choose to adopt.

Advancements in intelligent driving assistance: A machine learning approach to identify real-time driving strategies using environmental, eye movement, control-related, and kinetic-related data

Human-vehicle collaboration (HVC) is gaining focus as intelligent vehicle technology shifts driving responsibilities from humans to automation. Effective HVC hinges on mutual understanding between parties. Driving strategies address specific driving requirements and guide maneuver decisions, offering a gateway to uncovering a driver’s immediate needs. The present study introduces the framework of adaptive driving (FAD), which uses machine learning algorithms to incorporate critical features for identifying four common driving strategies, including competitive driving, skill-based behavior, defensive driving, and urgent reactions. A total of 52 participants engaged in simulated driving sessions, with these sessions yielding 983 driving events that were used for model training. Discriminative models considering driving environment, eye movement, or combined features can satisfactorily classify four driving strategies, while control-related and kinetic-related data offer minor contributions. Environmental data significantly aids in identifying skill-based and competitive strategies, characterized by lower risk dynamics and longer Time-to-Collision (TTC), with forward targets often triggering competitive driving. For risk mitigation strategies, both environment and eye movement data are crucial. Scenarios with higher risk dynamics prompt defensive driving, while shorter TTC indicates urgent reactions. Defensive driving is revealed by saccade patterns and less variation in pupil activities, whereas urgent reactions show pronounced pupil dilation and reduced saccade activity, indicating challenges in environmental observation during emergencies.

Influence of an adjacent tunnel connecting zone shading shed on drivers’ eye movement characteristics

A tunnel shading shed is crucial in improving driving safety as a type of traffic facility to ease the transition of light environments. To study the effect of installation of a shading shed on the visual characteristics of drivers in the connecting zone of the adjacent tunnels, a total of 32 drivers were gathered to perform a real vehicle experiment. The study zone of the adjacent tunnels was divided into three sections: upstream tunnel exit; connecting zone; and downstream tunnel threshold zone. Fixation duration, saccade duration and saccade frequency were selected as research indexes. The results suggest that installation of a shading shed in the connecting zone significantly reduced the fixation (saccade) duration in the upstream tunnel exit and downstream tunnel threshold zones, with a significantly higher saccade frequency. In addition, fixation is better improved at the downstream tunnel entrance, and saccade is better enhanced at the upstream tunnel exit.

Recognition of marine navigators’ workload based on eye movement features using bridge simulation

The human factor accounts for 80% of marine traffic accidents, either directly or indirectly. Workload significantly impacts the reliability of marine navigators’ behavior. Employing eye movement measurement as an objective assessment tool can effectively evaluate seafarers’ workload. This study conducts bridge simulation experiments to analyze eye movement features and investigate marine navigators’ workload levels across different tasks. By categorizing marine navigators’ workload into high and low level based on specific navigation duties. The study employs an independent sample T-test to select blink frequency, fixation time percentage, and pupil area standard deviation as key eye movement features for workload recognition. A marine navigators’ workload recognition model is developed utilizing the support vector machine (SVM) algorithm, which is subsequently trained and evaluated with experimental data. Results demonstrate a high recognition rate of 90.91% for marine navigators’ workload, indicating the SVM method’s accuracy in workload assessment based on eye movement features. These findings lay the foundation for the advancement of intelligent monitoring systems for marine navigators’ workload and the implementation of human–computer interaction active early warning technology and equipment.

Analyses of eye movement parameters in children with anisometropic amblyopia

ObjectiveTo investigate the characteristics of eye movement in children with anisometropic amblyopia, and to compare those characteristics with eye movement in a control group.Methods31 children in the anisometropic amblyopia group (31 amblyopic eyes in group A, 31 contralateral eyes in group B) and 24 children in the control group (48 eyes in group C). Group A was subdivided into groups Aa (severe amblyopia) and Ab (mild-moderate amblyopia). The overall age range was 6–12 years (mean, 7.83 ± 1.79 years). All children underwent ophthalmic examinations; eye movement parameters including saccade latency and amplitude were evaluated using an Eyelink1000 eye tracker. Data Viewer and MATLAB software were used for data analysis.ResultsMean and maximum saccade latencies, as well as mean and maximum saccade amplitudes, were significantly greater in group A than in groups B and C before and after treatment (P < 0.05). Mean and maximum saccade latencies were significantly different among groups Aa, Ab, and C (P < 0.05). Pupil trajectories in two detection modes suggested that binocular fixation was better than monocular fixation.ConclusionsEye movement parameters significantly differed between contralateral normal eyes and control eyes. Clinical evaluation of children with anisometropic amblyopia should not focus only on static visual acuity, but also on the assessment of eye movement.

Cross-modal Guiding Neural Network for Multimodal Emotion Recognition from EEG and Eye Movement Signals.

Multimodal emotion recognition research is gaining attention because of the emerging trend of integrating information from different sensory modalities to improve performance. Electroencephalogram (EEG) signals are considered objective indicators of emotions and provide precise insights despite their complex data collection. In contrast, eye movement signals are more susceptible to environmental and individual differences but offer convenient data collection. Conventional emotion recognition methods typically use separate models for different modalities, potentially overlooking their inherent connections. This study introduces a cross-modal guiding neural network designed to fully leverage the strengths of both modalities. The network includes a dual-branch feature extraction module that simultaneously extracts features from EEG and eye movement signals. In addition, the network includes a feature guidance module that uses EEG features to direct eye movement feature extraction, reducing the impact of subjective factors. This study also introduces a feature reweighting module to explore emotion-related features within eye movement signals, thereby improving emotion classification accuracy. The empirical findings from both the SEED-IV dataset and our collected dataset substantiate the commendable performance of the model, thereby confirming its efficacy.

Predicting Chinese reading proficiency based on eye movement features and machine learning

Predicting Chinese reading proficiency based on eye movement features and machine learning

Advancements in driver fatigue detection: A comprehensive analysis of eye movement and facial feature approaches

This article goes into sophisticated tiredness detection methods that make use of computer vision algorithms, including the Eye Movement and Facial Feature approaches. Eye movement analysis, which emphasizes blink frequency, Scanning speed, and gaze length, excels in tasks requiring sustained visual attention, which is critical in high-attention vocations. Facial Feature detection, on the other hand, monitors changes in expressions, muscle activity, and emotions, providing flexibility across a wide range of settings. Eye movement enables accurate indications, real-time responsiveness, and task-specific precision, which is especially important in high-attention occupations. Facial Feature is ideal for a variety of scenarios since it gives complete assessments, flexibility, contextual analysis, and non-intrusiveness. Case examples show how eye-tracking and convolutional neural networks may be used to improve accuracy. This study helps to weariness management by detailing the advantages and disadvantages of Eye Movement and Facial Feature methods. Given the pervasiveness of fatigue, knowing these sensing systems is critical for maintaining safety, productivity, and general well-being in a variety of professional and everyday life scenarios.

PRECYSE: Predicting Cybersickness using Transformer for Multimodal Time-Series Sensor Data

Cybersickness, a factor that hinders user immersion in VR, has been the subject of ongoing attempts to predict it using AI. Previous studies have used CNN and LSTM for prediction models and used attention mechanisms and XAI for data analysis, yet none explored a transformer that can better reflect the spatial and temporal characteristics of the data, beneficial for enhancing prediction and feature importance analysis. In this paper, we propose cybersickness prediction models using multimodal time-series sensor data (i.e., eye movement, head movement, and physiological signals) based on a transformer algorithm, considering sensor data pre-processing and multimodal data fusion methods. We constructed the MSCVR dataset consisting of normalized sensor data, spectrogram formatted sensor data, and cybersickness levels collected from 45 participants through a user study. We proposed two methods for embedding multimodal time-series sensor data into the transformer: modality-specific spatial and temporal transformer encoders for normalized sensor data (MS-STTN) and modality-specific spatial-temporal transformer encoder for spectrogram (MS-STTS). MS-STTN yielded the highest performance in the ablation study and the comparison of the existing models. Furthermore, by analyzing the importance of data features, we determined their relevance to cybersickness over time, especially the salience of eye movement features. Our results and insights derived from multimodal time-series sensor data and the transformer model provide a comprehensive understanding of cybersickness and its association with sensor data. Our MSCVR dataset and code are publicly available: https://github.com/dayoung-jeong/PRECYSE.git.

Evaluation of mental load using EEG and eye movement characteristics

Mental load is a major cause of human-induced accidents. In this study, an explosive impact sensitivity experiment was used to induce mental load. A combination of subjective questionnaires and objective prospective time-distance tests were used to judge whether subjects experienced mental load. Four indicators, namely, β, γ, mean pupil diameter, and fixation time were selected by statistical analysis and PCA for the construction of a mental load assessment model. The study found that the occipital lobe was the most sensitive to mental load, especially β and γ bands. Lastly, it was found that subjects showed different degrees of mental load for the same mental load induction task. The results of the study are applicable to the evaluation and monitoring of the mental characteristics of workers and provide a scientific basis for adjusting the mental load of workers over time to reduce the rate of accidents and enhance production efficiency.