Gaze Classification Research Articles

Exploration of factors affecting webcam-based automated gaze coding

Online experiments have been transforming the field of behavioral research, enabling researchers to increase sample sizes, access diverse populations, lower the costs of data collection, and promote reproducibility. The field of developmental psychology increasingly exploits such online testing approaches. Since infants cannot give explicit behavioral responses, one key outcome measure is infants’ gaze behavior. In the absence of automated eyetrackers in participants’ homes, automatic gaze classification from webcam data would make it possible to avoid painstaking manual coding. However, the lack of a controlled experimental environment may lead to various noise factors impeding automatic face detection or gaze classification. We created an adult webcam dataset that systematically reproduced noise factors from infant webcam studies which might affect automated gaze coding accuracy. We varied participants’ left-right offset, distance to the camera, facial rotation, and the direction of the lighting source. Running two state-of-the-art classification algorithms (iCatcher+ and OWLET) revealed that facial detection performance was particularly affected by the lighting source, while gaze coding accuracy was consistently affected by the distance to the camera and lighting source. Morphing participants’ faces to be unidentifiable did not generally affect the results, suggesting facial anonymization could be used when making online video data publicly available, for purposes of further study and transparency. Our findings will guide improving study design for infant and adult participants during online experiments. Moreover, training algorithms using our dataset will allow researchers to improve robustness and allow developmental psychologists to leverage online testing more efficiently.

An Extensive Analysis of Different Approaches to Driver Gaze Classification

An Extensive Analysis of Different Approaches to Driver Gaze Classification

EM-Gaze: eye context correlation and metric learning for gaze estimation

In recent years, deep learning techniques have been used to estimate gaze—a significant task in computer vision and human-computer interaction. Previous studies have made significant achievements in predicting 2D or 3D gazes from monocular face images. This study presents a deep neural network for 2D gaze estimation on mobile devices. It achieves state-of-the-art 2D gaze point regression error, while significantly improving gaze classification error on quadrant divisions of the display. To this end, an efficient attention-based module that correlates and fuses the left and right eye contextual features is first proposed to improve gaze point regression performance. Subsequently, through a unified perspective for gaze estimation, metric learning for gaze classification on quadrant divisions is incorporated as additional supervision. Consequently, both gaze point regression and quadrant classification performances are improved. The experiments demonstrate that the proposed method outperforms existing gaze-estimation methods on the GazeCapture and MPIIFaceGaze datasets.

How to choose the size of facial areas of interest in interactive eye tracking.

Advances in eye tracking technology have enabled the development of interactive experimental setups to study social attention. Since these setups differ substantially from the eye tracker manufacturer's test conditions, validation is essential with regard to the quality of gaze data and other factors potentially threatening the validity of this signal. In this study, we evaluated the impact of accuracy and areas of interest (AOIs) size on the classification of simulated gaze (fixation) data. We defined AOIs of different sizes using the Limited-Radius Voronoi-Tessellation (LRVT) method, and simulated gaze data for facial target points with varying accuracy. As hypothesized, we found that accuracy and AOI size had strong effects on gaze classification. In addition, these effects were not independent and differed in falsely classified gaze inside AOIs (Type I errors; false alarms) and falsely classified gaze outside the predefined AOIs (Type II errors; misses). Our results indicate that smaller AOIs generally minimize false classifications as long as accuracy is good enough. For studies with lower accuracy, Type II errors can still be compensated to some extent by using larger AOIs, but at the cost of more probable Type I errors. Proper estimation of accuracy is therefore essential for making informed decisions regarding the size of AOIs in eye tracking research.

Convolutional Neural Networks(CNN) based Eye-Gaze Tracking System using Machine Learning Algorithm

To avoid the rising number of car crash deaths, which are mostly caused by drivers' inattentiveness, a paradigm shift is expected. The knowledge of a driver's look area may provide useful details about his or her point of attention. Cars with accurate and low-cost gaze classification systems can increase driver safety. When drivers shift their eyes without turning their heads to look at objects, the margin of error in gaze detection increases. For new consumer electronic applications such as driver tracking systems and novel user interfaces, accurate and effective eye gaze prediction is critical. Such systems must be able to run efficiently in difficult, unconstrained conditions while using reduced power and expense. A deep learning-based gaze estimation technique has been considered to solve this issue, with an emphasis on WSN based Convolutional Neural Networks (CNN) based system. The proposed study proposes the following architecture, which is focused on data science: The first is a novel neural network model that is programmed to manipulate any possible visual feature, such as the states of both eyes and head location, as well as many augmentations; the second is a data fusion approach that incorporates several gaze datasets. However, due to different factors such as environment light shifts, reflections on glasses surface, and motion and optical blurring of the captured eye signal, the accuracy of detecting and classifying the pupil centre and corneal reflection centre depends on a car environment. This work also includes pre-trained models, network structures, and datasets for designing and developing CNN-based deep learning models for Eye-Gaze Tracking and Classification.

MAD saccade: statistically robust saccade threshold estimation via the median absolute deviation.

Saccade detection is a critical step in the analysis of gaze data. A common method for saccade detection is to use a simple threshold for velocity or acceleration values, which can be estimated from the data using the mean and standard deviation. However, this method has the downside of being influenced by the very signal it is trying to detect, the outlying velocities or accelerations that occur during saccades. We propose instead to use the median absolute deviation (MAD), a robust estimator of dispersion that is not influenced by outliers. We modify an algorithm proposed by Nyström and colleagues, and quantify saccade detection performance in both simulated and human data. Our modified algorithm shows a significant and marked improvement in saccade detection - showing both more true positives and less false negatives – especially under higher noise levels. We conclude that robust estimators can be widely adopted in other common, automatic gaze classification algorithms due to their ease of implementation.

Automatic Gaze Classification for Aviators: Using Multi-task Convolutional Networks as a Proxy for Flight Instructor Observation

Automatic Gaze Classification for Aviators: Using Multi-task Convolutional Networks as a Proxy for Flight Instructor Observation

Towards End to End head-free gaze classification

Towards End to End head-free gaze classification

Driver Monitoring System Based on Gaze Classification by Using Eye Center Location Recognition

Driver Monitoring System Based on Gaze Classification by Using Eye Center Location Recognition

Investigating the correspondence between driver head position and glance location.

The relationship between a driver’s glance orientation and corresponding head rotation is highly complex due to its nonlinear dependence on the individual, task, and driving context. This paper presents expanded analytic detail and findings from an effort that explored the ability of head pose to serve as an estimator for driver gaze by connecting head rotation data with manually coded gaze region data using both a statistical analysis approach and a predictive (i.e., machine learning) approach. For the latter, classification accuracy increased as visual angles between two glance locations increased. In other words, the greater the shift in gaze, the higher the accuracy of classification. This is an intuitive but important concept that we make explicit through our analysis. The highest accuracy achieved was 83% using the method of Hidden Markov Models (HMM) for the binary gaze classification problem of (a) glances to the forward roadway versus (b) glances to the center stack. Results suggest that although there are individual differences in head-glance correspondence while driving, classifier models based on head-rotation data may be robust to these differences and therefore can serve as reasonable estimators for glance location. The results suggest that driver head pose can be used as a surrogate for eye gaze in several key conditions including the identification of high-eccentricity glances. Inexpensive driver head pose tracking may be a key element in detection systems developed to mitigate driver distraction and inattention.

Deep Learning-Based Gaze Detection System for Automobile Drivers Using a NIR Camera Sensor.

A paradigm shift is required to prevent the increasing automobile accident deaths that are mostly due to the inattentive behavior of drivers. Knowledge of gaze region can provide valuable information regarding a driver’s point of attention. Accurate and inexpensive gaze classification systems in cars can improve safe driving. However, monitoring real-time driving behaviors and conditions presents some challenges: dizziness due to long drives, extreme lighting variations, glasses reflections, and occlusions. Past studies on gaze detection in cars have been chiefly based on head movements. The margin of error in gaze detection increases when drivers gaze at objects by moving their eyes without moving their heads. To solve this problem, a pupil center corneal reflection (PCCR)-based method has been considered. However, the error of accurately detecting the pupil center and corneal reflection center is increased in a car environment due to various environment light changes, reflections on glasses surface, and motion and optical blurring of captured eye image. In addition, existing PCCR-based methods require initial user calibration, which is difficult to perform in a car environment. To address this issue, we propose a deep learning-based gaze detection method using a near-infrared (NIR) camera sensor considering driver head and eye movement that does not require any initial user calibration. The proposed system is evaluated on our self-constructed database as well as on open Columbia gaze dataset (CAVE-DB). The proposed method demonstrated greater accuracy than the previous gaze classification methods.

Gaze-in-World movement Classification for Unconstrained Head Motion during Natural Tasks.

Accurate classification of eye movements is an integral component of many psychophysical experiments. Most event classifiers are restricted to the analysis of eye-in-head (EIHv) vectors while the head is fixed, and detect two classes of gaze events, namely fixations and saccades. Classification of eye movements as Vestibular Ocular reflex (VOR) or Smooth pursuits (SP) require that we compensate gaze motion with a person's head movements. As the complexity of naturalistic behavioral paradigms increases, so does the need for gaze classification with head-free motion which could detect these complex eye movements. We propose the use of a supervised Machine Learning (ML) classifier for the automated analysis of head free gaze signal. As a preliminary test, we used a SVM classifier to label a 60 Hz test dataset from a virtual reality ball catching task into periods of fixations, saccades and pursuit. This preliminary test resulted in 94% accuracy in a sample to sample comparison with an artificially generated set of gaze movements reflective of measured gaze statistics, and visual inspection shows very high correspondence to subjective analysis of the gaze signal. This promising methodology has since been extended to the analysis of data collected during natural tasks. Five subjects' head and gaze movements were recorded using an IMU and a 120 Hz SMI Wireless ETG2 as they performed four different natural tasks for 10 minutes each. This yielded 120 minutes of angular velocity data of the gaze in world vector (GIWv) during natural tasks. Four experts manually annotated a subset of the data as periods of fixation, saccade, smooth pursuit, and blinks. Inter-labeler reliability was measured to generate a confidence metric and a cost function for misclassifications. We summarize the statistics of natural eye movements and compare various ML based temporal series classification schemes. Meeting abstract presented at VSS 2017

‘Owl’ and ‘Lizard’: patterns of head pose and eye pose in driver gaze classification

Accurate, robust, inexpensive gaze tracking in the car can help keep a driver safe by facilitating the more effective study of how to improve (i) vehicle interfaces and (ii) the design of future advanced driver assistance systems. In this study, the authors estimate head pose and eye pose from monocular video using methods developed extensively in prior work and ask two new interesting questions. First, how much better can they classify driver gaze using head and eye pose versus just using head pose? Second, are there individual‐specific gaze strategies that strongly correlate with how much gaze classification improves with the addition of eye pose information? The authors answer these questions by evaluating data drawn from an on‐road study of 40 drivers. The main insight of the study is conveyed through the analogy of an ‘owl’ and ‘lizard’ which describes the degree to which the eyes and the head move when shifting gaze. When the head moves a lot (‘owl’), not much classification improvement is attained by estimating eye pose on top of head pose. On the other hand, when the head stays still and only the eyes move (‘lizard’), classification accuracy increases significantly from adding in eye pose. The authors characterise how that accuracy varies between people, gaze strategies, and gaze regions.