Eye Center Localization Research Articles

Accurate and Robust Eye Center Localization by Deep Voting

Eye Center Localization (ECL) is one of the most crucial technologies for various computer vision applications, such as eye gazing estimation and eye-tracking. Current conventional implementations consist of two phases, including locating the approximate eye regions and finding the eye center position by extracting the semantic features around the corresponding eye region. However, the combination pipeline results in the ECL accuracy being influenced by not only the environmental factors, such as the variability of photographing angles, illuminations, and the probable occlusions by eyelids or glasses, but also the quality of preceding procedures. Inspired by the ensemble mechanism in machine learning, we formulate ECL problem as a process of end-to-end voting, and the core is to select a set of local descriptors which can capture efficient independent information to vote for eye centers. With the help of deep convolutional neural networks, we are able to determine semantic descriptors around the eye regions. Each descriptor proposes a vote pointing to the corresponding eye center, and all the votes indicate the eye centers finally. The experimental results on the public databases, BioID and GI4E, show that our method achieves 80.3% and 95.2% accuracy, respectively, which outperforms the existing state-of-the-art methods, and the results based on our customized challenging database verify the robustness of our method.

Cascade Learning for Driver Facial Monitoring

As a non-invasive method, vision-based driver monitoring aims to identify risky maneuvers for intelligent vehicles and it has gained an increasing interest over recent years. However, most existing methods tend to design models for specific tasks, such as head pose or gaze estimation, which results in redundant models hampering real time applications. Besides, most driver facial monitoring methods ignore the correlation of different tasks. In this work, we propose a unified framework based on cascade learning for simultaneous facial landmark detection and head pose estimation, as well as simultaneous eye center detection and gaze estimation. In particular, built upon the key idea that facial landmark locations and 3D face model parameters are implicitly correlated, we introduce a cascade regression framework to achieve these two tasks simultaneously. After coarsely extracting the driver’s eye region from the detected facial landmarks, we perform cascade regression for simultaneous eye center detection and gaze estimation. Leveraging the power of cascade learning allows our method to alternatively optimize facial landmark detection, head pose estimation, eye center localization, and gaze prediction. The comparison experiments conducted on benchmark datasets of 300-W, GI4E, BU, MPIIGaze, and driving dataset of SHRP2 demonstrate that our proposed method can achieve state-of-the-art performance with robust effectiveness on the real driver monitoring applications.

Eye center localization using gradient and intensity information under uncontrolled environment

Eye center localization using gradient and intensity information under uncontrolled environment

A novel eye center localization method for multiview faces

Existing studies on eye center localization have mostly applied localization methods on faces viewed from frontal angles or faces with small yaw-rotation angles. This study proposed a novel eye center localization method that can effectively localize the eye centers under various situations on multiview faces with large yaw-rotation angles (from +67.5° to −67.5°). First, this study developed a multiview face detector that can be applied to large yaw angles and can flexibly detect and precisely capture the face region. The face detector can facilitate the generation of satisfactory results by a complete representation generative adversarial network (CR-GAN). Because a large yaw-rotation angle can substantially reduce the completeness of eye representation in an image or even cause the eyes to disappear within the face image, this study used a CR-GAN to produce frontal face images to solve the problem of incomplete representation in multiview face images. Furthermore, this study proposed a new iris-ripple filter to increase the accuracy and robustness of gradient localization. Finally, a new depth corresponding-points conversion method was proposed to automatically estimate the rotation variable between two faces and the conversion relationship between the eye centers. This method can effectively solve the problem of instability resulting from a CR-GAN during eye generation and can ensure localization accuracy for eyeballs with subtle changes. According to the experimental results, compared with other advanced methods, the proposed method exhibited higher accuracy and robustness in eye center localization when applied to images obtained from four databases that involved various challenging situations, including large yaw-rotation angle, illumination change, head pose change, gaze interaction, and complete occlusion of eyes.

A Novel Eye Center Localization Method for Head Poses With Large Rotations.

Eye localization is undoubtedly crucial to acquiring large amounts of information. It not only helps people improve their understanding of others but is also a technology that enables machines to better understand humans. Although studies have reported satisfactory accuracy for frontal faces or head poses at limited angles, large head rotations generate numerous defects (e.g., disappearance of the eye), and existing methods are not effective enough to accurately localize eye centers. Therefore, this study makes three contributions to address these limitations. First, we propose a novel complete representation (CR) pipeline that can flexibly learn and generate two complete representations, namely the CR-center and CR-region, of the same identity. We also propose two novel eye center localization methods. This first method employs geometric transformation to estimate the rotational difference between two faces and an unknown-localization strategy for accurate transformation of the CR-center. The second method is based on image translation learning and uses the CR-region to train the generative adversarial network, which can then accurately generate and localize eye centers. Five image databases are employed to verify the proposed methods, and tests reveal that compared with existing methods, the proposed method can more accurately and robustly localize eye centers in challenging images, such as those showing considerable head rotation (both yaw rotation of -67.5° to +67.5° and roll rotation of +120° to -120°), complete occlusion of both eyes, poor illumination in addition to head rotation, head pose changes in the dark, and various gaze interaction.

Real-time accurate eye center localization for low-resolution grayscale images

Eye center localization is considered a crucial step for many human–computer interaction (HCI) real-time applications. Detecting the center of eye (COE), accurately and in real time, is very challenging due to the wide variation of poses, eye appearance and specular reflection, especially in low-resolution images. In this paper, an accurate real-time detection algorithm of the COE is proposed. The proposed approach depends on the image gradient to detect the COE. The computational complexity is minimized and the accuracy is improved by down sampling the face resolution and applying a rough-to-fine algorithms, to reduce the search area, in accordance with the Eye Region Of Interest (EROI) and the number of COE candidates, tested by the proposed algorithm. Also, the detection algorithm is applied on a limited number of pixels that represent the iris boundary of the COE candidates. The Look Up Tables (LUTs) are implemented to, initially, store the invariant elements of the proposed image gradient-based algorithm, to reduce the detection time. Before applying the proposed COE detection approach, a modified specular reflection method is used to improve the detection accuracy. The performance of the proposed algorithm has been evaluated by applying it to three benchmark databases: the BIOID, GI4E and Talking Face video datasets, at different face resolutions. Experimental results revealed that the accuracy of the proposed algorithm is up to 91.68% and 96.7% for BIOID and GI4E datasets, respectively, while the minimum achieved average detection time is 2.7 ms. The promising results highlight the potential of the proposed algorithm to be used in some eye gaze-based real-time applications. Comparing the proposed method with the most state-of-the-art approaches showed that the system outperforms most of them and has a comparable performance with the others, in terms of the COE localization accuracy and detection speed.

Eye Blink Detection System for Paralyzed Patients

Paralysis of an human being is caused due to the degeneration of motor neurons which weakens the muscle, so that it does not allow patient to move, speak, breathe , and loss in the voluntary actions. It is an incurable disease. To understand the feelings of a paralyzed patients Brain wave technique and Electro-oculography techniques were used. These techniques are afflictive, discomfortable and leads to unconsciousness of the paralyzed patient. The real time video oculography system fills the communication gap between the patient and the world. Video Oculography (VOG) is video-based method of measuring the vertical, torsional and horizontal position components of both the eye blinks with the help of small cameras placed in the head-mounted mask .This paper presents different visual technologies, such as eye blink detection, eye center localization and conversion of the eye blink to speech. The video oculography could achieve accuracy of 0.968.

Hybrid regression and isophote curvature for accurate eye center localization

The eye center localization is a crucial requirement for various human-computer interaction applications such as eye gaze estimation and eye tracking. However, although significant progress has been made in the field of eye center localization in recent years, it is still very challenging for tasks under the significant variability situations caused by different illumination, shape, color and viewing angles. In this paper, we propose a hybrid regression and isophote curvature for accurate eye center localization under low resolution. The proposed method first applies the regression method, which is called Supervised Descent Method (SDM), to obtain the rough location of eye region and eye centers. SDM is robust against the appearance variations in the eye region. To make the center points more accurate, isophote curvature method is employed on the obtained eye region to obtain several candidate points of eye center. Finally, the proposed method selects several estimated eye center locations from the isophote curvature method and SDM as our candidates and a SDM-based means of gradient method further refine the candidate points. Therefore, we combine regression and isophote curvature method to achieve robustness and accuracy. In the experiment, we have extensively evaluated the proposed method on the two public databases which are very challenging and realistic for eye center localization and compared our method with existing state-of-the-art methods. The results of the experiment confirm that the proposed method outperforms the state-of-the-art methods with a significant improvement in accuracy and robustness and has less computational complexity.

Accurate and robust eye center localization via fully convolutional networks

Eye center localization is one of the most crucial and basic requirements for some human-computer interaction applications such as eye gaze estimation and eye tracking. There is a large body of works on this topic in recent years, but the accuracy still needs to be improved due to challenges in appearance such as the high variability of shapes, lighting conditions, viewing angles and possible occlusions. To address these problems and limitations, we propose a novel approach in this paper for the eye center localization with a fully convolutional network &#x0028 FCN &#x0029, which is an end-to-end and pixels-to-pixels network and can locate the eye center accurately. The key idea is to apply the FCN from the object semantic segmentation task to the eye center localization task since the problem of eye center localization can be regarded as a special semantic segmentation problem. We adapt contemporary FCN into a shallow structure with a large kernel convolutional block and transfer their performance from semantic segmentation to the eye center localization task by fine-tuning. Extensive experiments show that the proposed method outperforms the state-of-the-art methods in both accuracy and reliability of eye center localization. The proposed method has achieved a large performance improvement on the most challenging database and it thus provides a promising solution to some challenging applications.

Eye center localization in a facial image based on geometric shapes of iris and eyelid under natural variability

The estimation of the human eye centers is one of the important step in several computer vision applications such as driver drowsiness detection, eye tracking, face recognition etc. Most of the existing techniques are able to localize eyes in frontal faces only while they fail to localize eye pairs in complex scenarios such as changes in head pose, scale, and illumination. In this paper, an eye localization method has been proposed that can locate the eye centers more precisely in facial images captured under the above-mentioned complexities. The proposed method consists of three stages: eye candidate detection, eye candidate verification, and post-processing. In eye candidate detection, the possible eye candidates are extracted using two new features namely Semi-Circular Edge Shape (sCES) and Semi-Ellipse Edge Shape (sEES) features. These features take into consideration the semi-circular and semi-ellipse edges of iris and eyelid and hence are able to localize eye centers more precisely. In verification, the extracted eye candidates are verified using a Support Vector Machine (SVM) based classifier. A scale-space framework is also included in the verification stage to handle the scale variations of images. In post-processing, the eye centers are paired using some geometrical constraints and then a modified gradient-based method is proposed to detect the required eye pair. The proposed system is evaluated on different databases to check its robustness to changes in head pose, scale, illumination etc. The experimental results suggest that the proposed method shows better accuracy in challenging environments and also outperforms some state-of-the-art methods.

Segmentation of Human Eye Pupil with Novel Grid Based Localization Computing

This paper discuss about grid computing which increase the performance of vision image segmentation. The segmentation process with grid computing identifies the challenging in computation task. Novel grid based method supports image processing technique to circle eye pupil in normal and defective eyes. The procedure of the proposed system use real eye images as data collection for input to grid computing. Image corner and line edge detection is replaced with grid based technique to achieve good segmentation result. Grid with different sizes classified to reach neighbors to form grid segment. Adjacency graph merge the regions of highlighted eye pupil. This grid location points of eye image is noted to identify the eye center localization. The experimental task proves the efficiency of automate segmentation compared with other algorithms.

Eye tracker accuracy: quantitative evaluation of the invisible eye center location.

We present a new method to evaluate the accuracy of an eye tracker-based eye localization system. Measuring the accuracy of an eye tracker's primary intention, the estimated point of gaze, is usually done with volunteers and a set of fixation points used as ground truth. However, verifying the accuracy of the location estimate of a volunteer's eye center in 3D space is not easily possible. This is because the eye center, the center of corneal curvature, is an intangible point. We evaluate the eye location accuracy by using an eye phantom instead of eyes of volunteers. For this, we developed a testing stage with a realistic artificial eye and a corresponding kinematic model, which we trained with [Formula: see text] data. This enables us to precisely evaluate the eye location estimate of an eye tracker. We show that the proposed testing stage with the corresponding kinematic model is suitable for such a validation. Further, we evaluate a particular eye tracker-based navigation system and show that this system is able to successfully determine the eye center with a mean accuracy of 0.68 mm. We show the suitability of the evaluated eye tracker for eye interventions, using the proposed testing stage and the corresponding kinematic model. The results further enable specific enhancements of the navigation system to potentially get even better results.

Driver Monitoring System Based on Gaze Classification by Using Eye Center Location Recognition

Driver Monitoring System Based on Gaze Classification by Using Eye Center Location Recognition

Hybrid eye center localization using cascaded regression and hand-crafted model fitting

We propose a new cascaded regressor for eye center detection. Previous methods start from a face or an eye detector and use either advanced features or powerful regressors for eye center localization, but not both. Instead, we detect the eyes more accurately using an existing facial feature alignment method. We improve the robustness of localization by using both advanced features and powerful regression machinery. Unlike most other methods that do not refine the regression results, we make the localization more accurate by adding a robust circle fitting post-processing step. Finally, using a simple hand-crafted method for eye center localization, we show how to train the cascaded regressor without the need for manually annotated training data. We evaluate our new approach and show that it achieves state-of-the-art performance on the BioID, GI4E, and the TalkingFace datasets. At an average normalized error of e < 0.05, the regressor trained on manually annotated data yields an accuracy of 95.07% (BioID), 99.27% (GI4E), and 95.68% (TalkingFace). The automatically trained regressor is nearly as good, yielding an accuracy of 93.9% (BioID), 99.27% (GI4E), and 95.46% (TalkingFace).

Pupil localization algorithm combining convex area voting and model constraint

Locating the center of the eyes plays a significant role in many computer vision applications and research, such as face alignment, face recognition, human-computer interaction, control devices for disabled people, user attention and gaze estimation. The disturbances such as occlusions by eyelashes or eyelids, uneven spots and spectacle frames of glasses affect the accuracy and stability of eye center location. This paper presents a hybrid eye center locating methodology for infrared eye images. The pupil edge points are extracted by Starburst algorithm, and when we get the position and the gradient of the edge points, the approximate pupil boundary is determined by a convex region voting methods. After that, the boundary edge points are iteratively optimized by fitting an ellipses modeling constraint. Finally, the pupil is located correctly. Experiment shows that this algorithm has performance advantages compared with some state of the art approaches in pupil localization accuracy, iteration times and their performance. This algorithm combining convex area voting and model constraint has strong robustness, high accuracy and speed in real environments with occlusions and distortion pupil.

An Accurate and Simple Approach to Detect Eye Centers in Low Resolution Face Images

ABSTRACTThis paper introduces a new approach for accurate eye center localization based upon optimized image gradients algorithm. The proposed approach requires pre-fetched feature descriptors to detect the accurate iris centers amongst the possible eye center candidates computed by image gradients over the face image data-set. The results of the proposed algorithm for detecting eye, iris, and iris centers over the test set images of publicly available low resolution and challenging data-set show an accuracy percentage of 98.9, 95.7, and 89.2, respectively. The comparison results obtained by the proposed approach were at par with the state-of-the-art techniques involving complex calculations and training requirements. So, the superior performance and outcome of the proposed approach show the usefulness of optimizing the results of simple image gradients by pre-fetched Scale Invariant Feature Transform feature descriptors in detecting iris centers under unconstrained environments. The proposed approach may be useful for the development of real-time eye gaze tracking application with improved robustness and accuracy.

Eye center localization and gaze gesture recognition for human-computer interaction.

This paper introduces an unsupervised modular approach for accurate and real-time eye center localization in images and videos, thus allowing a coarse-to-fine, global-to-regional scheme. The trajectories of eye centers in consecutive frames, i.e., gaze gestures, are further analyzed, recognized, and employed to boost the human-computer interaction (HCI) experience. This modular approach makes use of isophote and gradient features to estimate the eye center locations. A selective oriented gradient filter has been specifically designed to remove strong gradients from eyebrows, eye corners, and shadows, which sabotage most eye center localization methods. A real-world implementation utilizing these algorithms has been designed in the form of an interactive advertising billboard to demonstrate the effectiveness of our method for HCI. The eye center localization algorithm has been compared with 10 other algorithms on the BioID database and six other algorithms on the GI4E database. It outperforms all the other algorithms in comparison in terms of localization accuracy. Further tests on the extended Yale Face Database b and self-collected data have proved this algorithm to be robust against moderate head poses and poor illumination conditions. The interactive advertising billboard has manifested outstanding usability and effectiveness in our tests and shows great potential for benefiting a wide range of real-world HCI applications.

A Real Time Robust Eye Center Localization using Geometric Eye Model and Edge Gradients in Unconstrained Visual Environment

eye localization is an essential and fundamental step in initialization for other eye tracking applications. A fast and accurate eye center localization method is proposed in this paper. A novel geometric eye model is derived based on face anthropometry parameters to identify coarse eye region correctly. A novel voting method using edge gradients on the iris boundary are used for fast and accurate eye center localization. A precise range around the boundary of an iris is derived to indicate the region within which the gradients are allowed to vote. Additionally the range between the pupil centers is derived to validate the search region. A weight map is generated for efficient computation, which is combined with edge gradients and the maximum of the multiplication between the dot products and the weight from the map is identified as the eye center. The proposed method is evaluated on challenging BioID database and found to be highly accurate for eye center localization task. The proposed method is efficient under natural lighting condition, has low computational complexity and excellent real-time ability.

Robust Eye Center Localization through Face Alignment and Invariant Isocentric Patterns.

The localization of eye centers is a very useful cue for numerous applications like face recognition, facial expression recognition, and the early screening of neurological pathologies. Several methods relying on available light for accurate eye-center localization have been exploited. However, despite the considerable improvements that eye-center localization systems have undergone in recent years, only few of these developments deal with the challenges posed by the profile (non-frontal face). In this paper, we first use the explicit shape regression method to obtain the rough location of the eye centers. Because this method extracts global information from the human face, it is robust against any changes in the eye region. We exploit this robustness and utilize it as a constraint. To locate the eye centers accurately, we employ isophote curvature features, the accuracy of which has been demonstrated in a previous study. By applying these features, we obtain a series of eye-center locations which are candidates for the actual position of the eye-center. Among these locations, the estimated locations which minimize the reconstruction error between the two methods mentioned above are taken as the closest approximation for the eye centers locations. Therefore, we combine explicit shape regression and isophote curvature feature analysis to achieve robustness and accuracy, respectively. In practical experiments, we use BioID and FERET datasets to test our approach to obtaining an accurate eye-center location while retaining robustness against changes in scale and pose. In addition, we apply our method to non-frontal faces to test its robustness and accuracy, which are essential in gaze estimation but have seldom been mentioned in previous works. Through extensive experimentation, we show that the proposed method can achieve a significant improvement in accuracy and robustness over state-of-the-art techniques, with our method ranking second in terms of accuracy. According to our implementation on a PC with a Xeon 2.5Ghz CPU, the frame rate of the eye tracking process can achieve 38 Hz.

Precise localization of eye centers in low resolution color images

The localization of eye centers and tracking of gaze constitutes an integral component of many human–computer interaction applications. A number of constraints including intrusiveness, mobility, robustness and high-price of eye tracking systems have hindered the way of the integration of eye trackers in everyday applications. Several ‘passive’ systems based on a single camera have been lately proposed in the literature, exhibiting though subordinate precision compared to the commercial, hardware-based eye tracking devices. In this paper we introduce an automatic, non-intrusive method for precise eye center localization in low resolution images, acquired from single low-cost cameras. To this end, the proposed system uses color information to derive a novel eye map that emphasizes the iris area and a radial symmetry transform which operates both on the original eye images and the eye map. The performance of the proposed method is extensively evaluated on four publicly available databases containing low resolution images and videos. Experimental results demonstrate great accuracy in challenging cases and resilience to pose and illumination variations, achieving significant improvement over existing methods.