Gaze Detection Method Research Articles

Deep Learning-Based Gaze Detection System for Automobile Drivers Using a NIR Camera Sensor.

A paradigm shift is required to prevent the increasing automobile accident deaths that are mostly due to the inattentive behavior of drivers. Knowledge of gaze region can provide valuable information regarding a driver’s point of attention. Accurate and inexpensive gaze classification systems in cars can improve safe driving. However, monitoring real-time driving behaviors and conditions presents some challenges: dizziness due to long drives, extreme lighting variations, glasses reflections, and occlusions. Past studies on gaze detection in cars have been chiefly based on head movements. The margin of error in gaze detection increases when drivers gaze at objects by moving their eyes without moving their heads. To solve this problem, a pupil center corneal reflection (PCCR)-based method has been considered. However, the error of accurately detecting the pupil center and corneal reflection center is increased in a car environment due to various environment light changes, reflections on glasses surface, and motion and optical blurring of captured eye image. In addition, existing PCCR-based methods require initial user calibration, which is difficult to perform in a car environment. To address this issue, we propose a deep learning-based gaze detection method using a near-infrared (NIR) camera sensor considering driver head and eye movement that does not require any initial user calibration. The proposed system is evaluated on our self-constructed database as well as on open Columbia gaze dataset (CAVE-DB). The proposed method demonstrated greater accuracy than the previous gaze classification methods.

Investigation of gaze detection method based on eyeball center and iris center estimation using CNN

Investigation of gaze detection method based on eyeball center and iris center estimation using CNN

Binocular gaze detection method using a fuzzy algorithm based on quality measurements

Due to the limitations of gaze detection based on one eye, binocular gaze detection using the gaze positions of both eyes has been researched. Most previous binocular gaze detection research calculated a gaze position as the simple average position of the detected gaze points of both eyes. To improve this approach, we propose a new binocular gaze detection method using a fuzzy algorithm with quality measurement of both eyes. The proposed method is used in the following three ways. First, in order to combine the gaze points of the left and right eyes, we measure four qualities on both eyes: distortion by an eyelid, distortion by the specular reflection (SR), the level of circularity of the pupil, and the distance between the pupil boundary and the SR center. Second, in order to obtain a more accurate pupil boundary, we compensate the distorted boundary of a pupil by an eyelid based on information from the lower half-circle of the pupil. Third, the final gaze position is calculated using a fuzzy algorithm based on four quality-measured scores. Experimental results show that the root-mean-square error of gaze estimation by the proposed method is approximately 0.67518 deg.

New method for face gaze detection in smart television

Recently, gaze detection-based interfaces have been regarded as the most natural user interface for use with smart televisions (TVs). Past research conducted on gaze detection primarily used near-infrared (NIR) cameras with NIR illuminators. However, these devices are difficult to use with smart TVs; therefore, there is an increasing need for gaze-detection technology that utilizes conventional (visible light) web cameras. Consequently, we propose a new gaze-detection method using a conventional (visible light) web camera. The proposed approach is innovative in the following three ways. First, using user-dependent facial information obtained in an initial calibration stage, an accurate head pose is calculated. Second, using theoretical and generalized models of changes in facial feature positions, horizontal and vertical head poses are calculated. Third, accurate gaze positions on a smart TV can be obtained based on the user-dependent calibration information and the calculated head poses by using a low-cost conventional web camera without an additional device for measuring the distance from the camera to the user. Experimental results indicate that the gaze-detection accuracy of our method on a 60-in. smart TV is 90.5%.

Gaze Tracking System for User Wearing Glasses

Conventional gaze tracking systems are limited in cases where the user is wearing glasses because the glasses usually produce noise due to reflections caused by the gaze tracker's lights. This makes it difficult to locate the pupil and the specular reflections (SRs) from the cornea of the user's eye. These difficulties increase the likelihood of gaze detection errors because the gaze position is estimated based on the location of the pupil center and the positions of the corneal SRs. In order to overcome these problems, we propose a new gaze tracking method that can be used by subjects who are wearing glasses. Our research is novel in the following four ways: first, we construct a new control device for the illuminator, which includes four illuminators that are positioned at the four corners of a monitor. Second, our system automatically determines whether a user is wearing glasses or not in the initial stage by counting the number of white pixels in an image that is captured using the low exposure setting on the camera. Third, if it is determined that the user is wearing glasses, the four illuminators are turned on and off sequentially in order to obtain an image that has a minimal amount of noise due to reflections from the glasses. As a result, it is possible to avoid the reflections and accurately locate the pupil center and the positions of the four corneal SRs. Fourth, by turning off one of the four illuminators, only three corneal SRs exist in the captured image. Since the proposed gaze detection method requires four corneal SRs for calculating the gaze position, the unseen SR position is estimated based on the parallelogram shape that is defined by the three SR positions and the gaze position is calculated. Experimental results showed that the average gaze detection error with 20 persons was about 0.70° and the processing time is 63.72 ms per each frame.

Support Vector Machines을 이용한 시선 방향 추정방법

A human gaze detection and tracing method is importantly required for HMI(Human-Machine-Interface) like a Human-Serving robot. This paper proposed a novel three-dimension (3D) human gaze estimation method by using a face recognition, an orientation estimation and SVMs (Support Vector Machines). 2,400 images with the pan orientation range of <TEX>$-90^{\circ}{\sim}90^{\circ}$</TEX> and tilt range of <TEX>$-40^{\circ}{\sim}70^{\circ}$</TEX> with intervals unit of <TEX>$10^{\circ}$</TEX> were used. A stereo camera was used to obtain the global coordinate of the center point between eyes and Gabor filter banks of horizontal and vertical orientation with 4 scales were used to extract the facial features. The experiment result shows that the error rate of proposed method is much improved than Liddell's.

A robust gaze detection method by compensating for facial movements based on corneal specularities

In this paper, we propose an analytical method for robust gaze detection by compensating for facial movements based on corneal specularities. Our method has following four novelties compared to previous works. (i) We used a USB (Universal Serial Bus) camera with four IR-LED (infra-red light emitting diodes) which were attached on each corner of a monitor, so we could simply calculate the user’s gaze position based on 2D relations between the center of pupil and four corneal reflections without considering the complex 3D relations among the camera, the monitor and the pupil coordinates, (ii) the detection accuracy of four corneal reflections was much enhanced by removing the imposter reflections based on the analysis of 3D structure of the human eye, (iii) by analyzing and compensating the shape change of quadrangle which was formed by four corneal reflections according to facial movement, we could reduce the gaze detection errors caused by facial movements, (iv) by compensating the error caused by angle Kappa based on user-dependent calibration, we could much enhance the accuracy of our gaze detection method. Experimental results showed that the gaze detection error was about 1.0° and we could know our method could track user’s gaze position at real-time speed.

A Real-Time Gaze Position Estimation Method Based on a 3-D Eye Model

This paper proposes a new gaze-detection method based on a 3-D eye position and the gaze vector of the human eyeball. Seven new developments compared to previous works are presented. First, a method of using three camera systems, i.e., one wide-view camera and two narrow-view cameras, is proposed. The narrow-view cameras use autozooming, focusing, panning, and tilting procedures (based on the detected 3-D eye feature position) for gaze detection. This allows for natural head and eye movement by users. Second, in previous conventional gaze-detection research, one or multiple illuminators were used. These studies did not consider specular reflection (SR) problems, which were caused by the illuminators when working with users who wore glasses. To solve this problem, a method based on dual illuminators is proposed in this paper. Third, the proposed method does not require user-dependent calibration, so all procedures for detecting gaze position operate automatically without human intervention. Fourth, the intrinsic characteristics of the human eye, such as the disparity between the pupillary and the visual axes in order to obtain accurate gaze positions, are considered. Fifth, all the coordinates obtained by the left and right narrow-view cameras, as well as the wide-view camera coordinates and the monitor coordinates, are unified. This simplifies the complex 3-D converting calculation and allows for calculation of the 3-D feature position and gaze position on the monitor. Sixth, to upgrade eye-detection performance when using a wide-view camera, the adaptive-selection method is used. This involves an IR-LED on/off scheme, an AdaBoost classifier, and a principle component analysis method based on the number of SR elements. Finally, the proposed method uses an eigenvector matrix (instead of simply averaging six gaze vectors) in order to obtain a more accurate final gaze vector that can compensate for noise. Experimental results show that the root mean square error of gaze detection was about 0.627 cm on a 19-in monitor. The processing speed of the proposed method (used to obtain the gaze position on the monitor) was 32 ms (using a Pentium IV 1.8-GHz PC). It was possible to detect the user's gaze position at real-time speed.

A Flexible Gaze Detection Method Using Single PTZ Camera

In this paper, we propose a flexible gaze detection method using single PTZ (Pan-Tilt-Zoom) camera. In this method, a PTZ camera traces user's face and measures position of their viewing point, so they do not need to fix their head in front of camera. Furthermore, to realize accurate gaze detection, we employ elliptical iris template matching. To reduce calculation amount of iris template matching, we get rough gaze direction by simple method on ahead to decide ellipse shape. In this paper, we also adapt to variation of facial orientations, which will affect to detect viewing position and gaze direction. After several experiments, we examine accuracy of gaze detection and head tracking ability of this system.