Gaze Point Research Articles

ACE-DNV: Automatic classification of gaze events in dynamic natural viewing

Eye movements offer valuable insights for clinical interventions, diagnostics, and understanding visual perception. The process usually involves recording a participant’s eye movements and analyzing them in terms of various gaze events. Manual identification of these events is extremely time-consuming. Although the field has seen the development of automatic event detection and classification methods, these methods have primarily focused on distinguishing events when participants remain stationary. With increasing interest in studying gaze behavior in freely moving participants, such as during daily activities like walking, new methods are required to automatically classify events in data collected under unrestricted conditions. Existing methods often rely on additional information from depth cameras or inertial measurement units (IMUs), which are not typically integrated into mobile eye trackers. To address this challenge, we present a framework for classifying gaze events based solely on eye-movement signals and scene video footage. Our approach, the Automatic Classification of gaze Events in Dynamic and Natural Viewing (ACE-DNV), analyzes eye movements in terms of velocity and direction and leverages visual odometry to capture head and body motion. Additionally, ACE-DNV assesses changes in image content surrounding the point of gaze. We evaluate the performance of ACE-DNV using a publicly available dataset and showcased its ability to discriminate between gaze fixation, gaze pursuit, gaze following, and gaze shifting (saccade) events. ACE-DNV exhibited comparable performance to previous methods, while eliminating the necessity for additional devices such as IMUs and depth cameras. In summary, ACE-DNV simplifies the automatic classification of gaze events in natural and dynamic environments. The source code is accessible at https://github.com/arnejad/ACE-DNV.

Calibration-Free Mobile Eye-Tracking Using Corneal Imaging.

In this paper, we present and evaluate a calibration-free mobile eye-traking system. The system's mobile device consists of three cameras: an IR eye camera, an RGB eye camera, and a front-scene RGB camera. The three cameras build a reliable corneal imaging system that is used to estimate the user's point of gaze continuously and reliably. The system auto-calibrates the device unobtrusively. Since the user is not required to follow any special instructions to calibrate the system, they can simply put on the eye tracker and start moving around using it. Deep learning algorithms together with 3D geometric computations were used to auto-calibrate the system per user. Once the model is built, a point-to-point transformation from the eye camera to the front camera is computed automatically by matching corneal and scene images, which allows the gaze point in the scene image to be estimated. The system was evaluated by users in real-life scenarios, indoors and outdoors. The average gaze error was 1.6∘ indoors and 1.69∘ outdoors, which is considered very good compared to state-of-the-art approaches.

Model-Based 3D Gaze Estimation Using a TOF Camera.

Among the numerous gaze-estimation methods currently available, appearance-based methods predominantly use RGB images as input and employ convolutional neural networks (CNNs) to detect facial images to regressively obtain gaze angles or gaze points. Model-based methods require high-resolution images to obtain a clear eyeball geometric model. These methods face significant challenges in outdoor environments and practical application scenarios. This paper proposes a model-based gaze-estimation algorithm using a low-resolution 3D TOF camera. This study uses infrared images instead of RGB images as input to overcome the impact of varying illumination intensity in the environment on gaze estimation. We utilized a trained YOLOv8 neural network model to detect eye landmarks in captured facial images. Combined with the depth map from a time-of-flight (TOF) camera, we calculated the 3D coordinates of the canthus points of a single eye of the subject. Based on this, we fitted a 3D geometric model of the eyeball to determine the subject's gaze angle. Experimental validation showed that our method achieved a root mean square error of 6.03° and 4.83° in the horizontal and vertical directions, respectively, for the detection of the subject's gaze angle. We also tested the proposed method in a real car driving environment, achieving stable driver gaze detection at various locations inside the car, such as the dashboard, driver mirror, and the in-vehicle screen.

RGBE-Gaze: A Large-scale Event-based Multimodal Dataset for High Frequency Remote Gaze Tracking.

High-frequency gaze tracking demonstrates significant potential in various critical applications, such as foveated rendering, gaze-based identity verification, and the diagnosis of mental disorders. However, existing eye-tracking systems based on CCD/CMOS cameras either provide tracking frequencies below 200 Hz or employ high-speed cameras, leading to substantial power consumption and increased device volume. While there have been some high-speed eye-tracking datasets and methods based on event cameras, they are primarily tailored for near-eye camera scenarios. They lack the advantages associated with remote camera scenarios, such as the absence of the need for direct contact, enhanced user comfort and convenience, and the freedom of head pose in natural environments. In this work, we present RGBE-Gaze, the first large-scale and multimodal dataset for remote gaze tracking in high-frequency through synchronizing RGB and event cameras. This dataset is collected from 66 participants with diverse genders and age groups. The custom hybrid RGB-Event camera setup is leveraged to collect 3.6 million full-face high spatial resolution RGB images and 26.3 billion high temporal resolution event samples. Additionally, the dataset includes 10.7 million gaze references from the Gazepoint GP3 HD eye tracker and 15,972 sparse points of gaze (PoG) ground truth obtained through manual stimuli clicks by participants. We also present the dataset distribution across characteristics such as head pose and distance, gaze direction, pupil size, and tracking frequency. Furthermore, we introduce a hybrid frame-event based gaze estimation method specifically designed for the collected dataset. Moreover, we perform extensive evaluations of this method along with existing frame-based gaze estimation methods under various gaze-related factors, including different subjects, gaze directions, head poses, head distances, and pupil diameters. The evaluation results illustrate that introducing event stream as a new modality into the dataset improves gaze tracking frequency and demonstrates greater estimation robustness across diverse gaze-related factors.

Lulu Filterized Lin’s correlative Theil-Sen regression-based fully connected deep multilayer perceptive neural network for eye gaze pattern recognition

Gaze estimation is process finding the point of gaze on observe axis of eye. Gaze tracking schemes are mainly employed in HCI and study of visual scanning samples. Traditional tracking schemes usually need accurate personal calibration procedure to evaluate the particular eye metrics. In order to improve the accurate gaze estimation, Lulu Filterized Lin?s Correlative Theil-Sen Regression-based Fully Connected Deep Multilayer Perceptive Neural Network (LFLCTR-FCDMPNN) is designed for accurate gaze pattern identification through lesser time consumption. Fully Connected Deep Multilayer Perceptive NN contains input layer, three hidden layers, output layer. In input layer, number of gaze images is collected. Then using Lulu nonlinear smoothing filtering method is applied in initial hidden layer for removing noise as well as enhancing image quality. In second hidden layer, Polar coordinate system-based eye-gaze point estimation is performed. Finally, the Gaze Pattern matching is carried out in third hidden layer using Lin?s Concordance Correlative Theil-Sen regression. The estimated gaze points are organized at gaze plane to identify gaze patterns. Then pattern matching performed by Lin?s Concordance Correlation. In this way, the eye gaze patterns are correctly recognized at the output layer. Experimental evaluation is conducted to demonstrate performance analysis of LFLCTR-FCDMPNN technique through different metrics like gaze pattern recognition accuracy, gaze pattern recognition time, and false-positive rate with different number of eye images. Explained result illustrates which LFLCTR-FCDMPNN method improves the accuracy of gaze pattern recognition and decreases the time consumption than the conventional prediction methods. Using the Synthes Eyes dataset, it turned out that the FPR of the suggested LFLCTR-FCDMPNN was 63% higher than existing.

A Study on Mobile Phone Interface Interaction Design for the Elderly Based on Digital Technology

Abstract The rapid development of digital technology significantly impacts the use of cell phone interfaces by the elderly. This paper aims to analyze the interaction design of cell phone interfaces for the elderly to enhance their user experience and ease of use. This paper investigates the evaluation system for evaluating cell phone interface interaction for the elderly, including interface size, Layout, and functionality. A group of elderly people aged 65-75 are invited to evaluate the effects of different design options by observing their performance under different interface sizes (large size 6.0 inches, medium size 5.5 inches, small size 4.5 inches) and layouts (nine-cell grid and combination type). The evaluation metrics are task completion time, number of operation errors, and eye movement data. The 6.0-inch interface experienced an average task completion time of 6.97 seconds, a decrease of approximately 6 seconds and 1 second from the 4.5-inch and 5.5-inch interfaces, respectively. The large-size interface had an average error rate of 1.11 per task, significantly less than the average error rate in the 4.5-inch and 5.5-inch interfaces. The more extensive interface significantly decreased user gaze points and gaze time, indicating more efficient information acquisition. More extensive cell phone interfaces and combined layouts can significantly improve interaction efficiency and reduce operation errors for elderly users, enhancing user experience. The findings of this study offer practical guidance for designing cell phone interfaces for the elderly and aid in promoting cell phone interface design for the elderly.

Impact of Road Central Greening Configuration on Driver Eye Movements: A Study Based on Real Vehicle Experiments

Safe driving depends on drivers’ ability to rapidly and accurately process information in varying traffic conditions. The presence of central green landscapes on roads is a key factor in this context. However, there is a gap in current research, which tends to focus on qualitative aspects of landscape design while lacking quantitative data-driven analyses. In this study, we aim to address this gap by investigating the impact of road central greening configuration on the eye movements of young novice drivers, a population particularly sensitive to external environmental changes. Specifically, we explore the influence of central green landscapes on four visual parameters: driver gaze, saccade, blinking, and pupil behavior. Through real vehicle experiments conducted on different road sections, we collected visual feature data to comprehensively analyze the patterns of driver eye movements. Our findings reveal that the introduction of central green landscapes can exert cognitive pressure on drivers, leading to specific alterations in their visual parameters. These changes include dispersed gaze points, reduced effective gaze durations, increased gaze frequencies, extended saccade durations and angles, heightened blink durations and frequencies, and reduced pupil area. By shedding light on the intricate interplay between road central greenery and driver behavior, this study provides valuable insights for optimizing landscape design in transportation planning and enhancing road safety considerations.

Design and Development of Traditional Elements in Visual Communication Design under the Integration of Virtual Reality Technology

Abstract This paper proposes a method based on visual communication to generate two-dimensional motion graphics that can enhance the perception of depth and three-dimensional structure in stereoscopic vision. Through eye tracking and calibration, gaze points in 3D space are calculated and applied to Chinese traditional elements in visual communication design. Feature vectors and improved plain Bayesian classifier methods are used to achieve user gesture recognition in virtual design and construct interaction design scenarios based on virtual reality technology. Lastly, the performance impact of augmented reality technology on traditional element design is investigated through a questionnaire survey. Based on the eight samples of conventional elements, the evaluation factors with the highest scores were found to be 10. In the subjective assessment of visual communication design, traditional components fall between mediocre and good, according to the ratings of all 18 assessment factors, which range from 0.2 to 1. The study presented in this paper effectively integrates virtual reality technology into the graphical design of traditional elements, significantly enhancing their visual expression.

Improving driving safety in freeway tunnels: A field study of linear visual guiding facilities

This study endeavors to assess the efficacy of retroreflective linear visual guiding facilities in enhancing traffic safety within freeway tunnels. While simulation tests have traditionally served as the primary means for investigating traffic engineering solutions aimed at improving driving behaviors within tunnels, empirical studies validating these methods in real tunnel environments have been notably lacking. To bridge this research gap, our study developed two cost-effective linear visual guiding facilities equipped with retroreflective features and evaluated their effectiveness in authentic freeway tunnel settings. We employed various indicators, including visual behaviors such as the speed of pupil area change and the distribution time of gaze points, as well as driving behaviors encompassing speed and lateral deviation, to assess the impact of these facilities on drivers. Our findings unequivocally demonstrate that the integration of linear visual guiding facilities significantly enhances driving safety within tunnels compared to conventional facilities. Specifically, the speed of pupil area change decreases, while the distribution time of gaze points increases following the implementation of linear visual guiding facilities. Moreover, we observe mitigations in the speed differentials among vehicles, both within and between tunnel zones, alongside reductions in vehicular lateral deviations. The synergistic deployment of linear visual guiding facilities in conjunction with conventional delineators and raised pavement markers can expedite visual perception, promote accurate driving decisions, and facilitate the safe passage of vehicles through tunnels. In conclusion, our study has proved that retroreflective linear visual guiding facilities effectively can enhance driving safety in freeway tunnels.

Impact of urban tunnel sidewall effect on drivers: Examining visual characteristics and driving behavior across different lanes

Urban tunnels have been widely used as special transportation infrastructure. This study investigated the impact of the urban tunnel sidewall effect on drivers, focusing on the drivers’ visual characteristics and driving behavior as the primary research objectives. In this study, numerous real vehicle tests were conducted in the middle section of an urban tunnel, focusing on different lanes. The drivers’ visual characteristics were examined through the distribution of gaze points and fixation duration, and driving behavior was assessed using the vehicles’ lateral offset and acceleration. The drivers’ visual characteristics and driving behavior were analysed together to investigate the impact of the sidewall effect on the relationship between the visual information input and driving behavior output. A questionnaire was administered after the experiment to assess the subjective experiences of each participant. The results indicated significant variations in the impact of the sidewall effect on driver attention distribution and vehicle stability across different lanes. Drivers in the left lane were the most affected and had the highest driving risk, followed by those in the right lane; those in the middle lane had the least driving risk. Meanwhile, when drivers in the left or right lanes focus their gaze on the adjacent sidewall or shortly afterwards, the vehicle experiences a higher lateral acceleration away from the sidewall compared to other areas.

An Eye Tracking and Brain–Computer Interface-Based Human–Environment Interactive System for Amyotrophic Lateral Sclerosis Patients

Amyotrophic Lateral Sclerosis (ALS) patients suffer a great inconvenience in their daily lives due to the gradual loss of their motion abilities. In order to help ALS patients regain their self-care abilities at a certain level, an eye-tracking and brain-computer interface (BCI) based human-environment interactive (HEI) system is proposed. Through a single-channel EEG recorder and a pair of eye-tracking glasses, the user’s attention levels and gaze points are detected. Based on the two types of intention information, the target objects are recognized by a YOLOv5 (You Only Look Once v5) model. Then according to voice instructions, users can control the household electric appliances. The system performed a high success rate of 89.3% experimentally, providing a reliable and capable assistive solution for ALS patients.

Analyzing the Effect of Diverse Gaze and Head Direction on Facial Expression Recognition With Photo-Reflective Sensors Embedded in a Head-Mounted Display.

As one of the facial expression recognition techniques for Head-Mounted Display (HMD) users, embedded photo-reflective sensors have been used. In this paper, we investigate how gaze and face directions affect facial expression recognition using the embedded photo-reflective sensors. First, we collected a dataset of five facial expressions (Neutral, Happy, Angry, Sad, Surprised) while looking in diverse directions by moving 1) the eyes and 2) the head. Using the dataset, we analyzed the effect of gaze and face directions by constructing facial expression classifiers in five ways and evaluating the classification accuracy of each classifier. The results revealed that the single classifier that learned the data for all gaze points achieved the highest classification performance. Then, we investigated which facial part was affected by the gaze and face direction. The results showed that the gaze directions affected the upper facial parts, while the face directions affected the lower facial parts. In addition, by removing the bias of facial expression reproducibility, we investigated the pure effect of gaze and face directions in three conditions. The results showed that, in terms of gaze direction, building classifiers for each direction significantly improved the classification accuracy. However, in terms of face directions, there were slight differences between the classifier conditions. Our experimental results implied that multiple classifiers corresponding to multiple gaze and face directions improved facial expression recognition accuracy, but collecting the data of the vertical movement of gaze and face is a practical solution to improving facial expression recognition accuracy.

Devalued Body of The Grobogan Tayub Dancer

Abstract
 This study aims to describe how Lasmi, a Tayub dancer Legend from Grobogan (an area near the North Kendeng Mountains) experiences others in various aspects of her life, especially when she lives in the art stage. The author uses the in-depth interview method in addition to supporting documents such as previous research, YouTube channels, news, and interviews with supporting sources by absorbing awareness of gender-based insights and feminism. The author borrows the theory of Liyan or subaltern/ the others from Gayatri Spivak and De Beauvoir which describes how women live and grow with the experience as the other group or not the main one. The research subject is Lasmi, who had her heyday in the 1980s to early 2000s. Lasmi is an example of an Indonesian postcolonial body, which means a body who does not come from a center of power, and does not come from the center of the metropolitan modern arts of its time. A body like this experiences otherness and marginalization in society and always becomes an object, especially from a male point of view or male gaze. The body of the other is always close to experiencing problems, being number two, and always being compared to others. In her life, Lasmi was experienced as the other in both public and domestic areas. Serving the public area when she was working as Waranggana Tayub was required to please many people, did not have full rights over his body, did not have enough time to rest, did not have maternity rest was adored if she was beautiful, and was bullied if she was ugly. Even so, she also experienced the others or subalterns in the domestic area. She is also a breadwinner and must also be demanded to be perfect in the house. She experienced domestic violence, was physically exhausted, and did not have full control over the money she earned. Even though she had experienced her heyday, in the era of the COVID-19 pandemic she was also included in a marginal group that was highly trained, due to the PSBB (Large-Scale Social Restrictions) so she could no longer dance and performed as she used to be. She has also suffered from an age devaluation, which has resulted in her being rarely invited for performing arts, and losing out to the young waranggana.
 Keywords: Gender, Feminism, Liyan, The Other, Subaltern, Kendeng, Tayub, Grobogan, COVID-19, Postcolonial Feminism

'Eye-Tracking' with Words for Alzheimer's Disease Detection: Time Alignment of Words Enunciation with Image Regions During Image Description Tasks.

Alzheimer's disease (AD) is a progressive neurodegenerative disease that results in cognitive decline, dementia, and eventually death. Diagnosing early signs of AD can help clinicians to improve the quality of life. We developed a non-invasive approach to help neurologists and clinicians to distinguish probable AD patients and healthy controls (HC). The patients' gaze points were followed based on the words they used to describe the Cookie Theft (CT) picture description task. We hypothesized that the timing of words enunciation aligns with the participant's eye movements. The moments that each word was spoken were then aligned with specific regions of the image. We then applied machine learning algorithms to classify probable AD and HC. We randomly selected 60 participants (30 AD and 30 HC) from the Dementia Bank (Pitt Corpus). Five main classifiers were applied to different features extracted from the recorded audio and participants' transcripts (AD and HC). Support vector machine and logistic regression had the highest accuracy (up to 80% and 78.33%, respectively) in three different experiments. In conclusion, point-of-gaze can be applied as a non-invasive and less expensive approach compared to other available methods (e.g., eye tracker devices) for early-stage AD diagnosis.

Testing different function fitting methods for mobile eye-tracker calibration.

During calibration, an eye-tracker fits a mapping function from features to a target gaze point. While there is research on which mapping function to use, little is known about how to best estimate the function's parameters. We investigate how different fitting methods impact accuracy under different noise factors, such as mobile eye-tracker imprecision or detection errors in feature extraction during calibration. For this purpose, a simulation of binocular gaze was developed for a) different calibration patterns and b) different noise characteristics. We found the commonly used polynomial regression via least-squares-error fit often lacks to find good mapping functions when compared to ridge regression. Especially as data becomes noisier, outlier-tolerant fitting methods are of importance. We demonstrate a reduction in mean MSE of 20% by simply using ridge over polynomial fit in a mobile eye-tracking experiment.

Monocular free-head gaze tracking method for driving electric sickbed

Building a free-head gaze tracking model with high accuracy, simple equipment, and not limited to wearing glasses is a challenge. In this paper, a monocular free-head gaze-tracking method based on machine learning are proposed. Two lightweight, high-precision and real-time gaze-tracking models are constructed, which can measure the 2D gaze point and 3D gaze direction respectively. In addition, we combined our gaze-tracking technology with electric sickbed to create an eye-gaze control based electric sickbed system that allows the patient to control the sickbed with their eyes. The experimental results show that the measurement errors of the two models on the MPIIGaze dataset are 4.84 cm and 4.8∘ respectively. After commissioning, user feedback has shown that this eye-gaze controlled electric sickbed system can enhance the lives of patients.

P1E5-07 Effects of different gazing points on event-related potentials to facial expression recognition

P1E5-07 Effects of different gazing points on event-related potentials to facial expression recognition

Eye detection and coarse localization of pupil for video-based eye tracking systems

A video-based eye tracking system generally captures NIR images, each of which contains one or two eyes of a subject. The subject’s point of gaze is then determined using 3D eye model and pupil centre corneal reflection technique. Eye detection and pupil localization play significant roles in video-based eye tracking systems. However, face rotation, wearing glasses, eye-shape variation and illumination variation make it difficult to detect an eye and localize a pupil accurately in the images captured by video-based eye tracking systems. In this paper, we proposed an eye detector that adopts a coarse-to-fine strategy. The eye detector consists of three classifiers: an ATLBP-THACs feature-based cascade classifier, a branch CNN and a multi-task CNN. We also proposed a method for coarse pupil localization. Coarse localization is an important step for pupil localization since it can provide initial pupil coordinates for a fine pupil localization method. Given a downscaled eye image, a shallow CNN is used to estimate the location of seven landmarks. On this basis, pupil center and radius are estimated. A method for small dim target enhancement is used to increase the contrast between pupil and background. The main goal of pupil enhancement is to make it easier to binarize an eye image. At last, component filtering is made by utilizing the estimated pupil center and radius. We collected a dataset named neepuEYE dataset that consists of 5500 NIR eye images from 109 people. The images can be used to generate augmented samples for training an eye detector since they contain eyes with different shape, orientation and pupil localization. Experimental results show that our eye detector is a fast and robust detector. Furthermore, our method for coarse pupil localization can obtain not only high detection rate but also high localization speed.

Real-time continuous EOG-based gaze angle estimation with baseline drift compensation under stationary head conditions

Objective:This work aims to propose a novel method to estimate the gaze from electrooculography (EOG) signals while compensating for the baseline drift, and which intrinsically detects fixations, saccades and blinks. In contrast to existing baseline drift mitigation techniques, the proposed framework is real-time-implementable, and does not require the average point of gaze to lie at the primary gaze position, nor does it disrupt the overall ocular pose-induced EOG signal DC characteristics. The EOG data used to validate the proposed method is also being made publicly available. Methods:The proposed method is based on the dual Kalman filter, which estimates the gaze angles (GAs) and the baseline concurrently, taking into consideration the EOG signal’s non-stationary and temporally-multimodal characteristics. In fact, it is a multiple-model technique based on a battery model of the eye wherein fixations, saccades and blinks are modelled separately. Results:When applied to short EOG data segments, a horizontal and vertical GA estimation error of 1.64 ± 0.82°and 1.97 ± 0.34°, respectively, was obtained, which compared well with the corresponding results obtained using the state-of-the-art linear regression models. Conversely, for longer data segments, the proposed framework yielded superior GA estimation performance when compared to the state-of-the-art techniques. Eye movement detection and labelling F-scores exceeding 90% were achieved. Conclusion:The proposed method yields reliable gaze estimation performance, and accurately detects fixations, saccades and blinks. Significance:This work proposes an integrated method to simultaneously estimate the GAs and address the baseline drift issue without the limitations of existing baseline drift mitigation techniques.

Driver distraction and in-vehicle interventions: A driving simulator study on visual attention and driving performance

Driving simulator studies are popular means to investigate driving behaviour in a controlled environment and test safety-critical events that would otherwise not be possible in real-world driving conditions. While several factors affect driving performance, driving distraction has been emphasised as a safety-critical issue across the globe. In this context, this study explores the impact of distraction imposed by mobile phone usage, i.e., writing and reading text messages, on driver behaviour. As part of the greater i-DREAMS project, this study uses a car driving simulator experimental design in Germany to investigate driver behaviour under various conditions: (I) monitoring scenario representing normal driving conditions, (II) intervention scenario in which drivers receive fixed timing in-vehicle intervention in case of unsafe driving manoeuvres, and (III) distraction scenario in which drivers receive in-vehicle interventions based on task completion capability, where mobile phone distraction is imposed. Besides, eye-tracking glasses are used to further explore drivers’ attention allocation and eye movement behaviour. This research focuses on driver response to risky traffic events (i.e., potential pedestrian collisions, and tailgating) and the impact of distraction on driving performance, by analysing a set of eye movement and driving performance measures of 58 participants. The results reveal a significant change in drivers’ gaze patterns during the distraction drives with significantly higher gaze points towards the i-DREAMS intervention display (the utilised advanced driver assistance systems in this study). The overall statistical analysis of driving performance measures suggests nearly similar impacts on driver behaviour during distraction drives; a higher deviation of lateral positioning was noted irrespective of the event risk levels and lower longitudinal acceleration rates were observed for pedestrian collisions and non-critical events during distracted driving.