Pupil Center Corneal Reflection Research Articles

35‐3: Modeling Eye Movement and Reflection in Virtual Environments for Eye Tracking

Eye movement tracking plays a pivotal role in use interaction within Virtual Reality (VR) environments. Our research focuses on an advanced eye movement simulation using virtual environment for eye tracking training and eye tracking precision evaluating platform. We developed a real‐eye model, enhanced light refraction and corneal reflections under infrared LED light with eye rotation. Our study also customized screen,light source and cameras placement to capture eye images more faithfully. We also presents a noval testing methology and equipments of spatial accuracy testing of the eye‐tracking systems. The proposed proprietary eye‐tracking test system designed to recreate scenarios of human gaze and provide true values for eye gaze directions. Overall, the integration of Pupil Center Corneal Reflection (PCCR) based dataset generation and testing platform mark a significant advancement, enabling huge database, precise groudtruth training and accurate results testing.

S172. Eye-tracking in awake craniotomy

In awake craniotomy, cortical- and subcortical structures are electrically stimulated to localize neural tissue involved in speech processing and other high-order brain functions. The surgical goal is to achieve maximal resection without causing permanent neurological deficits. Gliomas may involve the visual cortex or optic radiations. Electrical stimulation of those structures produces phosphenes, visual blurring or scotomas, phenomena that the patient is instructed to report. It is also possible to perform visual testing in the awake patient to avoid post-operative visual field defects. Obviously, it is critical that the patient remains focused on a center target to avoid false negative stimulation results i.e., where visual defects are not detected because the patient changes focus towards the presented object. Here, I present the feasibility of using infrared eye tracking during awake craniotomy to monitor eye position during visual field testing. A custom-made system for intraoperative language, neuropsychological and visual testing has been used at our institution since 2013. Different testing protocols are available and chosen depending on the surgical area of interest. For visual testing, the patient is instructed to denominate objects in one of the four visual quadrants while focusing on a target in the center of a screen. Positive stimulus results may then be detected either as speech error (e.g. anomia) or visual phenomenon reported by the patient. Images are presented on 25-in. monitor approximately 20 in. (50 cm) from the patient. The patient is positioned in the lateral position. An eyetracker, using near infra-red pupil center corneal reflection is secured at the bottom of the screen. Eye position is represented by a graphical object superimposed on the presented image. Ten patients have been tested by eye-tracking pre- and intraoperatively. Preoperatively (one day before surgery), eye-tracking was successful in all tested individuals during visual field testing. Intraoperatively, it was possible to confirm that the patient was focusing on the center of the screen in 60% of the cases. The main reason for failed eye tracking was drowsiness and/or suboptimal positioning of the monitor (e.g., downward gazing) where the eyelids tend to cover too much of the cornea for successful detection of the corneal reflection. Although it is sufficient to only track one eye, drooping of the upper eye lid from released scalp tissue in frontal surgery may further preclude successful eye tracking. Eye tracking with near infrared technique is feasible during awake glioma surgery. We aim to further explore the technique to increase the validity of visual field testing and to some extent monitor vigilance during other types of tests.

Deep Learning-Based Gaze Detection System for Automobile Drivers Using a NIR Camera Sensor.

A paradigm shift is required to prevent the increasing automobile accident deaths that are mostly due to the inattentive behavior of drivers. Knowledge of gaze region can provide valuable information regarding a driver’s point of attention. Accurate and inexpensive gaze classification systems in cars can improve safe driving. However, monitoring real-time driving behaviors and conditions presents some challenges: dizziness due to long drives, extreme lighting variations, glasses reflections, and occlusions. Past studies on gaze detection in cars have been chiefly based on head movements. The margin of error in gaze detection increases when drivers gaze at objects by moving their eyes without moving their heads. To solve this problem, a pupil center corneal reflection (PCCR)-based method has been considered. However, the error of accurately detecting the pupil center and corneal reflection center is increased in a car environment due to various environment light changes, reflections on glasses surface, and motion and optical blurring of captured eye image. In addition, existing PCCR-based methods require initial user calibration, which is difficult to perform in a car environment. To address this issue, we propose a deep learning-based gaze detection method using a near-infrared (NIR) camera sensor considering driver head and eye movement that does not require any initial user calibration. The proposed system is evaluated on our self-constructed database as well as on open Columbia gaze dataset (CAVE-DB). The proposed method demonstrated greater accuracy than the previous gaze classification methods.

Compensation Method of Natural Head Movement for Gaze Tracking System Using an Ultrasonic Sensor for Distance Measurement.

Most gaze tracking systems are based on the pupil center corneal reflection (PCCR) method using near infrared (NIR) illuminators. One advantage of the PCCR method is the high accuracy it achieves in gaze tracking because it compensates for the pupil center position based on the relative position of corneal specular reflection (SR). However, the PCCR method only works for user head movements within a limited range, and its performance is degraded by the natural movement of the user’s head. To overcome this problem, we propose a gaze tracking method using an ultrasonic sensor that is robust to the natural head movement of users. Experimental results demonstrate that with our compensation method the gaze tracking system is more robust to natural head movements compared to other systems without our method and commercial systems.

A novel approach to the low cost real time eye mouse

The PCCR (Pupil Center Corneal Reflection) method became dominant for finding human's diverse eye gaze directions through the research on the eye tracking technology that has been done for a very long period of time. The initial studies on the eye tracking technology were related to the general human interface for operating equipment and devices, then it has been promoted to the field of various purposes such as a market research in a recent study analyzing customer's behaviors. In particular, a real time eye gaze tracking system is most important for many HCI applications including stereoscopic synthesis, intend extraction, behavior analysis and etc. In order to make an eye gaze tracking system to be real time, the system must have an efficient pupil detection algorithm and ambience-independent image processing as well as reduced complexity, small size and number of circuit components. This paper proposes a method for getting clean images compared to the previous systems to reduce image processing overhead. Because it also helps reducing the number of image frames to be dropped during the image processing, the proposed method can provide a sufficient performance even on a low cost hardware system by reducing the transmission traffic.

Long-Range Gaze Tracking System for Large Movements

In the vision-based remote gaze tracking systems, the most challenging topics are to allow natural movement of a user and to increase the working volume and distance of the system. Several eye gaze estimation methods considering the natural movement of a user have been proposed. However, their working volume and distance are narrow and close. In this paper, we propose a novel 2-D mapping-based gaze estimation method that allows large-movement of user. Conventional 2-D mapping-based methods utilize mapping function between calibration points on the screen and pupil center corneal reflection (PCCR) vectors obtained in user calibration step. However, PCCR vectors and their associated mapping function are only valid at or near to the position where the user calibration is performed. The proposed movement mapping function, complementing the user's movement, estimates scale factors between two PCCR vector sets: one obtained at the user calibration position and another obtained at the new user position. The proposed system targets a longer range gaze tracking which operates from 1.4 to 3 m. A narrow-view camera mounted on a pan and tilt unit is used by the proposed system to capture high-resolution eye image, providing a wide and long working volume of about 100 cm × 40 cm × 100 cm. The experimental results show that the proposed method successfully compensated the poor performance due to user's large movement. Average angular error was 0.8° and only 0.07° of angular error was increased while the user moved around 81 cm.

Long range eye gaze tracking system for a large screen

Eye gaze tracking system has been widely researched for the replacement of the conventional computer interfaces such as the mouse and keyboard. In this paper, we propose the long range binocular eye gaze tracking system that works from 1.5 m to 2.5 m with allowing a head displacement in depth. The 3D position of the user's eye is obtained from the two wide angle cameras. A high resolution image of the eye is captured using the pan, tilt, and focus controlled narrow angle camera. The angles for maneuvering the pan and tilt motor are calculated by the proposed calibration method based on virtual camera model. The performance of the proposed calibration method is verified in terms of speed and convenience through the experiment. The narrow angle camera keeps tracking the eye while the user moves his head freely. The point-of-gaze (POG) of each eye onto the screen is calculated by using a 2D mapping based gaze estimation technique and the pupil center corneal reflection (PCCR) vector. PCCR vector modification method is applied to overcome the degradation in accuracy with displacements of the head in depth. The final POG is obtained by the average of the two POGs. Experimental results show that the proposed system robustly works for a large screen TV from 1.5 m to 2.5 m distance with displacements of the head in depth (+20 cm) and the average angular error is 0.69°.

Iris Center Corneal Reflection Method for Gaze Tracking Using Visible Light

Very few attempts, if any, have been made to use visible light in corneal reflection approaches to the problem of gaze tracking. The reasons usually given to justify the limited application of this type of illumination are that the required image features are less accurately depicted, and that visible light may disturb the user. The aim of this paper is to show that it is possible to overcome these difficulties and build an accurate and robust gaze tracker under these circumstances. For this purpose, visible light is used to obtain the corneal reflection or glint in a way analogous to the well-known pupil center corneal reflection technique. Due to the lack of contrast, the center of the iris is tracked instead of the center of the pupil. The experiments performed in our laboratory have shown very satisfactory results, allowing free-head movement and no need of recalibration.

A novel method for gaze tracking by local pattern model and support vector regressor

This paper presents a novel eye gaze tracking method with allowable head movement based on a local pattern model (LPM) and support vector regressor (SVR). The LPM, a combination of improved pixel-pattern-based texture feature (PPBTF) and local-binary-pattern texture feature (LBP), is employed to calculate texture features from the characteristics of the eyes and a new binocular vision scheme is adopted to detect the spatial coordinates of the eyes. The texture features from LPM and the spatial coordinates together are fed into support vector regressor (SVR) to match a gaze mapping function, and subsequently to track gaze direction under allowable head movement. The experimental results show that the proposed approach results in better accuracy in estimating the gaze direction than the state-of-the-art pupil center corneal reflection (PCCR) method.