Eye Gaze Tracking Research Articles

ShouldAR: Detecting Shoulder Surfing Attacks Using Multimodal Eye Tracking and Augmented Reality

Shoulder surfing attacks (SSAs) are a type of observation attack designed to illicitly gather sensitive data from "over the shoulder" of victims. This attack can be directed at mobile devices, desktop screens, Personal Identification Number (PIN) pads at an Automated Teller Machine (ATM), or written text. Existing solutions are generally focused on authentication techniques (e.g., logins) and are limited to specific attack scenarios (e.g., mobile devices or PIN Pads). We present ShotjldAR, a mobile and usable system to detect SSAs using multimodal eye gaze information (i.e., from both the potential attacker and victim). ShouldAR uses an augmented reality headset as a platform to incorporate user eye gaze tracking, rear-facing image collection and eye gaze analysis, and user notification of potential attacks. In a 24-participant study, we show that the prototype is capable of detecting 87.28% of SSAs against both physical and digital targets, a two-fold improvement on the baseline solution using a rear-facing mirror, a widely used solution to the SSA problem. The ShouldAR approach provides an AR-based, active SSA defense that applies to both digital and physical information entry in sensitive environments.

Longevity of a Brain–Computer Interface for Amyotrophic Lateral Sclerosis

SummaryThe durability of communication with the use of brain–computer interfaces in persons with progressive neurodegenerative disease has not been extensively examined. We report on 7 years of independent at-home use of an implanted brain–computer interface for communication by a person with advanced amyotrophic lateral sclerosis (ALS), the inception of which was reported in 2016. The frequency of at-home use increased over time to compensate for gradual loss of control of an eye-gaze–tracking device, followed by a progressive decrease in use starting 6 years after implantation. At-home use ended when control of the brain–computer interface became unreliable. No signs of technical malfunction were found. Instead, the amplitude of neural signals declined, and computed tomographic imaging revealed progressive atrophy, which suggested that ALS-related neurodegeneration ultimately rendered the brain–computer interface ineffective after years of successful use, although alternative explanations are plausible. (Funded by the National Institute on Deafness and Other Communication Disorders and others; ClinicalTrials.gov number, NCT02224469.)

An Improved Eyegaze Tracking Application for Early Prediction of Mental Health Using Convolution Neural Networks with Hybrid Optimization Approach

An Improved Eyegaze Tracking Application for Early Prediction of Mental Health Using Convolution Neural Networks with Hybrid Optimization Approach

Autologous umbilical cord blood infusion for the treatment of autism in young children: A within-subjects open label study on safety (assessed via caregiver report) and efficacy.

This study aimed to document the safety and efficacy of a single infusion of autologous umbilical cord blood (UCB) in 20 autistic children aged 24-72 months. A pre-post treatment within-subjects open label design was used. At T = 0, 6, 12, and 18 months, participants underwent detailed and structured safety evaluations (via caregiver report), Vineland Adaptive Behavior Scale (Vineland-3), Stanford Binet Intelligence Scale (SB-5), Expressive One-Word Picture Vocabulary Test, Brief Observation of Social Communication Change (BOSCC), Pervasive Developmental Disorder-Behavior Inventory, Repetitive Behavior Scale-Revised, Sensory Experience Questionnaire (SEQ-2.1), Child Behavior Checklist, Clinical Global Impression-Severity and Improvement (CGI-I) Scales, and eye-gaze tracking. UCB infusion was conducted at T = 6 months, hence, 0-6 months was the control period, and 6-18 months the follow-up period. Of 20 children recruited, 19 completed the study and 1 was withdrawn due to UCB not meeting quality control criteria for infusion. There were 15 males and 4 females with an overall mean (SD) age of 4.15 (0.62) years. Mean (SD) cell dose administered was 38.16 (9.82) million cells/kg. None suffered serious adverse events although there were mild behavioral side effects and one unit grew coagulase negative staphylococcus from a post-thaw sample. There were no significant differences in Vineland-3, SB-5, BOSCC, and SEQ-2.1 scores at T = 12 and T = 18 months. Twelve participants had T = 18 CGI-I scores of 2-3 (minimally to much improved), seven participants had scores of 4 (no change). Autologous UCB infusion in autistic children is generally safe but not without risks, including that of infection. In this within-subjects study, some children showed global symptom improvements while others showed no change. Stem cell therapies for autism should only be conducted under strict clinical trial conditions with clear risk discussions.

Visual and auditory attention in individuals with DYRK1A and SCN2A disruptive variants.

This preliminary study sought to assess biomarkers of attention using electroencephalography (EEG) and eye tracking in two ultra-rare monogenic populations associated with autism spectrum disorder (ASD). Relative to idiopathic ASD (n = 12) and neurotypical comparison (n = 49) groups, divergent attention profiles were observed for the monogenic groups, such that individuals with DYRK1A (n = 9) exhibited diminished auditory attention condition differences during an oddball EEG paradigm whereas individuals with SCN2A (n = 5) exhibited diminished visual attention condition differences noted by eye gaze tracking when viewing social interactions. Findings provide initial support for alignment of auditory and visual attention markers in idiopathic ASD and neurotypical development but not monogenic groups. These results support ongoing efforts to develop translational ASD biomarkers within the attention domain.

Head motion-corrected eye gaze tracking with the da Vinci surgical system.

To facilitate the integration of point of gaze (POG) as an input modality for robot-assisted surgery, we introduce a robust head movement compensation gaze tracking system for the da Vinci Surgical System. Previous surgical eye gaze trackers require multiple recalibrations and suffer from accuracy loss when users move from the calibrated position. We investigate whether eye corner detection can reduce gaze estimation error in a robotic surgery context. A polynomial regressor is first used to estimate POG after an 8-point calibration, and then, using another regressor, the POG error from head movement is estimated from the shift in 2D eye corner location. Eye corners are computed by first detecting regions of interest using the You Only Look Once (YOLO) object detector trained on 1600 annotated eye images (open dataset included). Contours are then extracted from the bounding boxes and a derivative-based curvature detector refines the eye corner. Through a user study (n = 24), our corner-contingent head compensation algorithm showed an error reduction in degrees of visual angle of 1.20 (p = 0.037) for the left eye and 1.26 (p = 0.079) for the right compared to the previous gold-standard POG error correction method. In addition, the eye corner pipeline showed a root-mean-squared error of 3.57 (SD = 1.92) pixels in detecting eye corners over 201 annotated frames. We introduce an effective method of using eye corners to correct for eye gaze estimation, enabling the practical acquisition of POG in robotic surgery.

Cognitive load in tele-robotic surgery: a comparison of eye tracker designs.

Eye gaze tracking and pupillometry are evolving areas within the field of tele-robotic surgery, particularly in the context of estimating cognitive load (CL). However, this is a recent field, and current solutions for gaze and pupil tracking in robotic surgery require assessment. Considering the necessity of stable pupillometry signals for reliable cognitive load estimation, we compare the accuracy of three eye trackers, including head and console-mounted designs. We conducted a user study with the da Vinci Research Kit (dVRK), to compare the three designs. We collected eye tracking and dVRK video data while participants observed nine markers distributed over the dVRK screen. We compute and analyze pupil detection stability and gaze prediction accuracy for the three designs. Head-worn devices present better stability and accuracy of gaze prediction and pupil detection compared to console-mounted systems. Tracking stability along the field of view varies between trackers, with gaze predictions detected at invalid zones of the image with high confidence. While head-worn solutions show benefits in confidence and stability, our results demonstrate the need to improve eye tacker performance regarding pupil detection, stability, and gaze accuracy in tele-robotic scenarios.

Improving the Temporal Accuracy of Eye Gaze Tracking for the da Vinci Surgical System through Automatic Detection of Decalibration Events and Recalibration

Robust and accurate eye gaze tracking can advance medical telerobotics by providing complementary data for surgical training, interactive instrument control, and augmented human–robot interactions. However, current gaze tracking solutions for systems such as the da Vinci Surgical System (dVSS) are limited to complex hardware installations. Additionally, existing methods do not account for operator head movement inside the surgeon console, invalidating the original calibration. This work provides an initial solution to these challenges that can seamlessly integrate into console devices beyond the dVSS. Our approach relies on simple and unobtrusive wearable eye tracking glasses and provides calibration routines that can contend with operator-head movements. An external camera measures movement of the glasses through trackers mounted on the glasses to detect invalidation of the prior calibration from head movement and slippage. Movements beyond a threshold of 5 cm or 9ˆ∘ prompt another calibration sequence. In a study where users moved freely in the surgeon console after an initial calibration procedure, we show that our system tracks the eye tracking glasses to initiate recalibration procedures. Recalibration can reduce the mean tracking error up to 89% compared to the current prevailing approach which relies on the initial calibration only. This work is an important first step towards incorporating user movement into gaze-based applications for the dVSS.

An analysis of perfume packaging designs on consumer’s cognitive and emotional behavior using eye gaze tracking

An analysis of perfume packaging designs on consumer’s cognitive and emotional behavior using eye gaze tracking

GazePair: Efficient Pairing of Augmented Reality Devices Using Gaze Tracking

As Augmented Reality (AR) devices become more prevalent and commercially viable, the need for quick, efficient, and secure schemes for pairing these devices has become more pressing. Current methods to securely exchange holograms require users to send this information through large data centers, creating security and privacy concerns. Existing techniques to pair these devices on a local network and share information fall short in terms of usability and scalability. These techniques either require hardware not available on AR devices, intricate physical gestures, removal of the device from the head, do not scale to multiple pairing partners, or rely on methods with low entropy to create encryption keys. To that end, we propose a novel pairing system, called GazePair, that improves on all existing local pairing techniques by creating an efficient, effective, and intuitive pairing protocol. GazePair uses eye gaze tracking and a spoken key sequence cue (KSC) to generate identical, independently generated symmetric encryption keys with 64 bits of entropy. GazePair also achieves improvements in pairing success rates and times over current methods. Additionally, we show that GazePair can extend to multiple users. Finally, we assert that GazePair can be used on any Mixed Reality (MR) device equipped with eye gaze tracking.

Eye Gaze Tracking in Robot-Assisted Minimally Invasive Surgery: A Systematic Review of Recent Advances and Applications

Eye Gaze Tracking in Robot-Assisted Minimally Invasive Surgery: A Systematic Review of Recent Advances and Applications

The Utility of Head-Mounted Eye Gaze Tracking for Vision-in-Action Assessments to Enhance Skill Acquisition and Sport Performance: A Strengths, Weaknesses, Opportunities, and Threats Analysis

Mobile eye tracking technology enables the objective evaluation of what, when, and how athletes use visual information from the environment to control action in service of skilled performance. This review highlights the strengths, weaknesses, opportunities, and threats (SWOT) to consider before adopting this technology to evaluate visual–motor control for enhanced skill acquisition and performance in sport. Strengths include increased ecological validity by recording the athlete within their natural performance environment. Weaknesses include diminished tracking fidelity based on the eye’s morphology or squinting, eye tracker movement, and lens fogging. Opportunities consist of novel assessments around research themes such as combatting the negative effects of pressure and performance enhancement through personalized training. The formfitting technologies also introduce recording opportunities during standard practices. Data processing time, synchronization of video streams, and hand coding of variables comprise the threats in addition to the hardware cost and the fees associated with crowdsourcing data coding. Currently, it is also not feasible to provide coaches with a report of their athletes’ data in a timeline suitable to create player-specific drills. Evaluating portable eye tracking technology with a SWOT framework helps translate scientific knowledge to practitioners considering implementing this technology for athlete assessment and development.

Development and validation of an Arabic language eye-tracking paradigm for the early screening and diagnosis of autism spectrum disorders in Qatar.

Abnormal eye gaze is a hallmark characteristic of autism spectrum disorder (ASD). The primary aim of the present research was to develop an Arabic version of an objective measure of ASD, the "autism index" (AI), based on eye gaze tracking to social and nonsocial stimuli validated initially in the United States. The initial phase of this study included the translation of English language eye-tracking stimuli into stimuli appropriate for an Arabic-speaking culture. During the second phase, we tested it on a total of 144 children with ASD, and 96 controls. The AI had excellent internal consistency and test-retest reliability. Moreover, the AI showed good differentiation of ASD from control cases (AUC = 0.730, SE = 0.035). The AI was significantly positively correlated with SCQ total raw scores (r = 0.46, p < 0.001). ADOS-2 scores were only available in the ASD group and did not show a significant relationship with AI scores (r = 0.10, p = 0.348), likely due to the restricted range. The AI, when implemented using Arabic-translated stimuli in a Qatari sample, showed good diagnostic differentiation and a strong correlation with parent-reported ASD symptoms. Thus, the AI appears to have cross-cultural validity and may be useful as a diagnostic aide to inform clinical judgment and track ASD symptom levels as part of the evaluation process.

Cognitive computing and 3D facial tracking method to explore the ethical implication associated with the detection of fraudulent system in online examination

Following the Covid-19 pandemic, the rapid spread of online education and tests demanded the implementation of cheating detection tools to ensure academic integrity. While advances in technology such as face recognition, face expression recognition, head posture analysis, eye gaze tracking, network data traffic analysis, and IP spoofing detection have shown promising results in detecting fraudulent behavior, their integration raises ethical concerns that must be carefully considered. This work presents a cognitive computing strategy for investigating the ethical implications of using cheating detection systems in online tests. This study attempts to examine the potential impact on students’ privacy, fairness, and trust in the examination process by employing cognitive computing, which models human cognitive capacities. A thorough literature review is used in the process to uncover existing ethical norms and regulatory frameworks linked to online assessments and cheating detection. Soft computing approaches are also used to evaluate the effectiveness and dependability of the aforementioned cheating detection strategies. The study looks into how far facial recognition and expression analysis can go in terms of privacy, as well as the possibility of bias in head posture analysis and eye gaze tracking algorithms. Furthermore, it investigates the ethical implications of monitoring network data traffic and detecting IP spoofing, with a focus on data security and user permission. The cognitive computing model, based on the analysis, presents a comprehensive framework for ethical decision-making when installing cheating detection technologies. The findings of this study contribute to the continuing discussion about the ethical concerns of using modern technologies to identify cheating in online exams. It provides educational institutions and policymakers with practical ideas for striking a balance between academic integrity and protecting students’ rights and dignity. By emphasizing ethical issues, this study aims to ensure that the implementation of cheating detection systems adheres to values of fairness, transparency, and privacy protection, promoting a trusting and supportive online learning environment for all parties involved.

Pupil Position by an Improved Technique of YOLO Network for Eye Tracking Application

This Eye gaze following is the real-time collection of information about a person's eye movements and the direction of their look. Eye gaze trackers are devices that measure the locations of the pupils to detect and track changes in the direction of the user's gaze. There are numerous applications for analyzing eye movements, from psychological studies to human-computer interaction-based systems and interactive robotics controls. Real-time eye gaze monitoring requires an accurate and reliable iris center localization technique. Deep learning technology is used to construct a pupil tracking approach for wearable eye trackers in this study. This pupil tracking method uses deep-learning You Only Look Once (YOLO) model to accurately estimate and anticipate the pupil's central location under conditions of bright, natural light (visible to the naked eye). Testing pupil tracking performance with the upgraded YOLOv7 results in an accuracy rate of 98.50% and a precision rate close to 96.34% using PyTorch.

Pupillometry in telerobotic surgery: A comparative evaluation of algorithms for cognitive effort estimation

Background: Eye gaze tracking and pupillometry are emerging topics in telerobotic surgery as it is believed that they will enable novel gaze-based interaction paradigms and provide insights into the user’s cognitive load (CL). Further, the successful integration of CL estimation into telerobotic systems is thought to catalyze the development of new human-computer interfaces for personalized assistance and training processes. However, this field is in its infancy, and identifying reliable gaze and pupil-tracking solutions in robotic surgery is still an area of ongoing research and high uncertainty. Methods: Considering the potential benefits of pupillometry-based CL assessments in telerobotic surgery, we seek to better understand the possibilities and limitations of contemporary pupillometry-based cognitive effort estimation algorithms in telerobotic surgery. To this end, we conducted a user study using the da Vinci Research Kit (dVRK) and performed two experiments where participants were asked to perform a series of N-Back tests, either while (i) idling or (ii) performing a peg transfer task. We then compare four contemporary CL estimation methods based on direct analysis of pupil diameter in the spatial and frequency domains. Results: We find that some methods can detect the presence of cognitive effort in simple scenarios (e.g., when the user is not performing any manual task), they fail to differentiate the different levels of CL. Similarly, when the manual peg transfer task is added, the reliability of all models is compromised, highlighting the necessity of more robust methods that consider different factors that complement the pupil diameter information. Conclusion: Our results offer a quantitative perspective of the limitations of the current solutions and highlight the necessity of developing tailored designs for the telerobotic surgery environment.

Studies of Vision in Cricket-A Narrative Review.

Vision is thought to play a substantial role in hitting and fielding in cricket. An understanding of which visual skills contribute during cricket play could inform future clinical training trials. This paper reviews what has been reported thus far regarding the relationship of visual skills to cricket performance and reviews the results of clinical trials in which the impact of visual skills training on cricket performance has been addressed. Fundamental or low-level visual skills, with the exception of color vision and perhaps near stereopsis and dynamic visual acuity, are similar between cricket players and the general population. Simple reaction time has been found to be shorter in cricket players in some but not all studies. While there is mixed or no evidence that the aforementioned visual skills are superior in cricket players compared to non-players, comparisons of eye and head movements and gaze tracking have revealed consistent differences between elite cricket batters and sub-elite batters. Future training studies could examine whether teaching sub-elite batters to emulate the gaze tracking patterns of elite batters is beneficial for batting. Lastly, clinical trials in which visual skills of cricket players have been trained have in many cases resulted in positive effects on visual skills, or judgments required in cricket, or cricket play. However, clinical trials with larger and more diverse groups of participants and correlations to on-field metrics and on-field performance (i.e., domain-specific assessments) are necessary before conclusions can be drawn regarding the efficacy of vision training.

Webcam-Based Eye-Tracking of Attentional Biases in Alzheimer’s Disease: A Proof-Of-Concept Study

ABSTRACT Objectives To measure home-based older adults’ attentional biases (AB) using webcam-based eye-tracking (WBET) and examine internal consistency. Methods Twelve participants with and without cognitive impairment completed online self-report anxiety and depression screens, and a 96-trial dot-probe task with eye-gaze tracking. For each trial, participants fixated on a cross, free-viewed sad-neutral, sad-angry, sad-happy, angry-neutral, angry-happy, and happy-neutral facial expression pairings, and then fixated on a dot. In emotional-neutral pairings, the time spent looking (dwell-time) at neutral was averaged and subtracted from the emotional average to indicate biases “away from” (negative score) and “toward” (positive score) each emotional face. Internal consistency was estimated for dwell-times and bias scores using Cronbach’s alpha and Spearman – Brown corrected split-half coefficients. Results The full-cohort and a comorbid anxious and depressed sub-group (n = 6) displayed AB away from sad faces, and toward angry and happy faces, with happy-face AB being more pronounced. AB indices demonstrated low reliability except sub-group happy-face indices. Happy-face AB demonstrated the highest reliability. Conclusions AB measures were in-line with lab-based eye-tracking literature, providing some support for WBET-based measurement. Clinical Implications Establishing the feasibility of WBET-based measures is a step toward an objective home-based clinical tool. Literature-based suggestions are provided to improve reliability.

Cognitive Load Estimation in VR Flight Simulator.

This paper discusses the design and development of a low-cost virtual reality (VR) based flight simulator with cognitive load estimation feature using ocular and EEG signals. Focus is on exploring methods to evaluate pilot's interactions with aircraft by means of quantifying pilot's perceived cognitive load under different task scenarios. Realistic target tracking and context of the battlefield is designed in VR. Head mounted eye gaze tracker and EEG headset are used for acquiring pupil diameter, gaze fixation, gaze direction and EEG theta, alpha, and beta band power data in real time. We developed an AI agent model in VR and created scenarios of interactions with the piloted aircraft. To estimate the pilot's cognitive load, we used low-frequency pupil diameter variations, fixation rate, gaze distribution pattern, EEG signal-based task load index and EEG task engagement index. We compared the physiological measures of workload with the standard user's inceptor control-based workload metrics. Results of the piloted simulation study indicate that the metrics discussed in the paper have strong association with pilot's perceived task difficulty.

Toward a see-through camera via AR lightguide.

As the foundation of virtual content generation, cameras are crucial for augmented reality (AR) applications, yet their integration with transparent displays has remained a challenge. Prior efforts to develop see-through cameras have struggled to achieve high resolution and seamless integration with AR displays. In this work, we present LightguideCam, a compact and flexible see-through camera based on an AR lightguide. To address the overlapping artifacts in measurement, we present a compressive sensing algorithm based on an equivalent imaging model that minimizes computational consumption and calibration complexity. We validate our design using a commercial AR lightguide and demonstrate a field of view of 23.1° and an angular resolution of 0.1° in the prototype. Our LightguideCam has great potential as a plug-and-play extensional imaging component in AR head-mounted displays, with promising applications for eye-gaze tracking, eye-position perspective photography, and improved human-computer interaction devices, such as full-screen mobile phones.