Iris Tracking Research Articles

Finding Stars From Fireworks: Improving Non-Cooperative Iris Tracking

We revisit the problem of iris tracking with RGB cameras, aiming to obtain iris contours from captured images of eyes. We find the reason that limits the performance of the state-of-the-art method in more general non-cooperative environments, which prohibits a wider adoption of this useful technique in practice. We believe that because the iris boundary could be inherently unclear and blocked, as its pixels occupy only an extremely limited percentage of those on the entire image of the eye, similar to the stars hidden in fireworks, we should not treat the boundary pixels as one class to conduct end-to-end recognition directly. Thus, we propose to learn features from iris and sclera regions first, and then leverage entropy to sketch the thin and sharp iris boundary pixels, where we can trace more precise parameterized iris contours. In this work, we also collect a new dataset by smartphone with 22 K images of eyes from video clips. We annotate a subset of 2 K images, so that label propagation can be applied to further enhance the system performance. Extensive experiments over both public and our own datasets show that our method outperforms the state-of-the-art method. The results also indicate that our method can improve the coarsely labeled data to enhance the iris contour’s accuracy and support the downstream application better than the prior method.

Irissometry: Effects of Pupil Size on Iris Elasticity Measured With Video-Based Feature Tracking.

PurposeIt is unclear how the iris deforms during changes in pupil size. Here, we report an application of a multi-feature iris tracking method, which we call irissometry, to investigate how the iris deforms and affects the eye position signal as a function of pupil size.MethodsTo evoke pupillary responses, we repeatedly presented visual and auditory stimuli to healthy participants while we additionally recorded their right eye with a macro lens–equipped camera. We tracked changes in iris surface structure between the pupil and sclera border (limbus) by calculating local densities (distance between feature points) across evenly spaced annular iris regions.ResultsThe time analysis of densities showed that the inner regions of the iris stretched more strongly as compared with the outer regions of the iris during pupil constrictions. The pattern of iris densities across eccentricities and pupil size showed highly similar patterns across participants, highlighting the robustness of this elastic property. Importantly, iris-based eye position detection led to more stable signals than pupil-based detection.ConclusionsThe iris regions near the pupil appear to be more elastic than the outer regions near the sclera. This elastic property explains the instability of the pupil border and the related position errors induced by eye movement and pupil size in pupil-based eye-tracking. Tracking features in the iris produce more robust eye position signals. We expect that irissometry may pave the way to novel eye trackers and diagnostic tools in ophthalmology.

Eye Tracking Interaction on Unmodified Mobile VR Headsets Using the Selfie Camera

Input methods for interaction in smartphone-based virtual and mixed reality (VR/MR) are currently based on uncomfortable head tracking controlling a pointer on the screen. User fixations are a fast and natural input method for VR/MR interaction. Previously, eye tracking in mobile VR suffered from low accuracy, long processing time, and the need for hardware add-ons such as anti-reflective lens coating and infrared emitters. We present an innovative mobile VR eye tracking methodology utilizing only the eye images from the front-facing (selfie) camera through the headset’s lens, without any modifications. Our system first enhances the low-contrast, poorly lit eye images by applying a pipeline of customised low-level image enhancements suppressing obtrusive lens reflections. We then propose an iris region-of-interest detection algorithm that is run only once. This increases the iris tracking speed by reducing the iris search space in mobile devices. We iteratively fit a customised geometric model to the iris to refine its coordinates. We display a thin bezel of light at the top edge of the screen for constant illumination. A confidence metric calculates the probability of successful iris detection. Calibration and linear gaze mapping between the estimated iris centroid and physical pixels on the screen results in low latency, real-time iris tracking. A formal study confirmed that our system’s accuracy is similar to eye trackers in commercial VR headsets in the central part of the headset’s field-of-view. In a VR game, gaze-driven user completion time was as fast as with head-tracked interaction, without the need for consecutive head motions. In a VR panorama viewer, users could successfully switch between panoramas using gaze.

EllSeg: An Ellipse Segmentation Framework for Robust Gaze Tracking.

Ellipse fitting, an essential component in pupil or iris tracking based video oculography, is performed on previously segmented eye parts generated using various computer vision techniques. Several factors, such as occlusions due to eyelid shape, camera position or eyelashes, frequently break ellipse fitting algorithms that rely on well-defined pupil or iris edge segments. In this work, we propose training a convolutional neural network to directly segment entire elliptical structures and demonstrate that such a framework is robust to occlusions and offers superior pupil and iris tracking performance (at least 10% and 24% increase in pupil and iris center detection rate respectively within a two-pixel error margin) compared to using standard eye parts segmentation for multiple publicly available synthetic segmentation datasets.

A Novel Gaze Input System Based on Iris Tracking With Webcam Mounted Eyeglasses

AbstractDue to the high cost of eye-tracking systems based on pupillary corneal reflections, efforts to develop a webcam-based eye-tracking system have increased to provide an affordable alternative to disabled people in recent years. However, due to the camera specification and location, ambient light changes and positional changes of the users, the gazing point of the eyes has not yet been determined precisely by such a system. Therefore, only 8 different gaze directions or up to 10 gaze regions could be detected in the previous webcam-based human–computer interaction studies. In this study, a novel gaze input system has been proposed to make the best use of the limited performance of webcam-based eye tracking and offer an economical alternative for disabled people. To reduce the impact of head movements, the webcam has been mounted to an ordinary glasses frame and positioned in front of the eye. For estimation of the gaze regions, a feature-based method (Hough transformation) was used by considering the circular shape of the iris and the contrast between the iris and sclera. The central coordinates of the iris image captured by the webcam were given to the k-nearest neighbor classifier. We performed a series of experiments with 20 subjects to determine the performance of the system and to investigate the effect of ambient light on the system’s accuracy. The 23 regions that were gazed at by subjects were determined with an average accuracy of 99.54%. When the ambient light level was reduced by half, the accuracy decreased to 94.74%. As a result, it has been found that the proposed prototype allows more accurate recognition of a larger number of regions on the screen than previous webcam-based systems. It has been observed that system performance decreases if the ambient light is reduced byhalf.

Sistem Komunikasi Augmentatif dan Alternatif Berbasis Tracking Realtime Mata

Communication focuses on understanding of a person to another person, a simple communication can occur if there are similarities between the delivery of messages and people who receive messages. However, there are some people who lose communication skills or language and speech disorder which is a side effect of some diseases. One think to facilitate the problem is building a Augmentative and Alternative Communication (AAC) Systems. AAC is a communication method to support speaking or writing abilities for someone who have oral and writing disorders. In this research, eye-gaze boards used to implant AAC method. The computer programs created to implement that methods, with centroid iris tracking as input variable. The tests were carried out on various respondents with various eye shapes. The result of this research show that iris centroid could represent eyes direction, with six classifications (left – middle – right and upper – lower).

An Enhanced Eye-Tracking Approach Using Pipeline Computation

Tracking and authentication at a low cost and in a convenient way is an important issue nowadays as opposed to using high cost for specified hardware and complex calibration process. In the paper, the prime concentration is divided into two: (i) eye tracking and (ii) parallelism in execution. First part has been subdivided as: Firstly, human face is taken from video sequence, and this is divided into frames. Eye regions are recognized from image frames. Secondly, iris region and pupils (central point) are being identified. In identifying pupil, the energy intensity and edge strength are taken into account together. Iris and eye corners are considered as tracking points. The famous sinusoidal head model is used for 3-D head shape, and adaptive probability density function (APDF) is proposed for estimating pose in facial features extraction. Thirdly, iris (pupil) tracking is completed by integrating eye vector and head movement information gained from APDF. The second part focuses on reducing execution time (ET) by applying pipeline architecture (PA). So the entire process has been subdivided into three phases. Phase-I is tracking eyes and pupils and preparing an eye vector. Phase-II is completing the calibration process. Phase-III is for matching and feature extraction. PA is integrated to improve performance in terms of ET because three phases are running in parallel. The experiment is done based on CK + image sets. Indeed, by using PA, the ET is reduced by (66.68%) two-thirds. The average accuracy in tracking (91% in left eye pupil (LEP)) and 93.6% in right eye pupil (REP)) is a robust one.

Real-Time Iris Tracking Using Deep Regression Networks for Robotic Ophthalmic Surgery

Robotic-assisted platforms are expected to guarantee the accuracy of surgical operation and accelerate its learning curve. Iris tracking can guide the robotic manipulator during the operation. However, few researches focused on it during surgery. It is a big challenge due to the deformation of the iris and occlusion caused by instruments. A novel real-time iris tracking method based on a regression network are proposed to meet the speed and accuracy requirements of the ophthalmic robotic system. It utilizes the low-level visual features and high-level semantic meanings from different layers to capture the discriminative representation of the iris target. Then the bottleneck layers are added to improve computation efficiency. Furthermore, a multi-loss function is designed by jointly learning Absolute loss and Euclidean loss. Finally, the experimental results under the typical surgical scene demonstrate that iris tracker achieves an accuracy of 89.16% and a real-time speed of 134fps with GPU, which is suitable for the ophthalmic robotic system to perform real-time robotic manipulation.

A Hybrid Pupil and Iris Tracking Using RGB-IR Camera

A Hybrid Pupil and Iris Tracking Using RGB-IR Camera

Unconstrained and Calibration-Free Gaze Estimation in a Room-Scale Area Using a Monocular Camera

Gaze estimation using monocular cameras has significant commercial applicability, and many studies have been undertaken on head pose-invariant and calibration-free gaze estimation. The head positions in existing data sets used in these studies are, however, limited to the vicinity of the camera, and methods trained on such data sets are not applicable when subjects are at greater distances from the camera. In this paper, we create a room-scale gaze data set with large variations in head poses to achieve robust gaze estimation across a broader range of widths and depths. The head positions are much farther from the camera, and the resolution of the eye image is lower than in conventional data sets. To address this issue, we propose a likelihood evaluation method based on edge gradients with dense particles for iris tracking, which achieves robust tracking at low-resolution eye images. Cross-validation experiments show that our proposed method is more accurate than conventional methods on all the individuals in our data set.

The comparison of accommodative response and ocular movements in viewing 3D and 2D displays

Some people often appear asthenopia symptoms of eye fatigue, double vision, nausea, and dizziness while viewing 3D movies. By testing the changes of accommodation function and ocular movements during watching 3D and 2D movies, the factors in visual discomfort are confirmed in this study.20 subjects with normal visual acuity and binocular vision function view 3D and 2D movies with the same content for 30min, and the amplitude of accommodation, binocular vergence ability, stereo-acuity, and tear break-up time of the subjects before and after viewing the films are measured. Furthermore, an open-field auto-refractor is utilized for synchronously testing the change of accommodative response while viewing 3D and 2D films, and Skalar IRIS tracking system is applied to record ocular movements through infrared positioning.In comparison with viewing 2D movies, the accommodative response and ocular movements reveal obvious changes while viewing 3D movies. The accommodation ability and binocular vergence ability obviously drop after viewing 3D movies; moreover, the stability of stereo-acuity and tear film also gets worse. The changes of such physiological factors might be the major cause of asthenopia.

Real-Time Iris Tracking-based on a Generalized Probabilistic Particle Filter

Objectives: We have proposed a new approach for tracking the iris, based on a generalized probabilistic particle filter. Accurate iris tracking plays an important role in many advanced applications, such as human-computer interactions and driver fatigue detection. Methods/Statistical Analysis: This paper illustrates a new method for iris tracking, based on a generalized probabilistic particle filter. This approach utilizes a sample set for the tracked iris, which is constructed at the beginning of the tracking process. The prior representation and position of the tracked iris were then predicted, based on minimization of the parameters of the proposed generalized probabilistic distribution. Findings: The computation of the likelihood of generalized particle filters for each distribution function is hen estimated. Our approach had provided reliable results with a low error rate and low levels of computation. This technique aimed to improve the speed of the eye tracking algorithm so that can be used for real-time applications. Application/Improvements: Accurate iris tracking plays an important role in many advanced applications, such as human-computer interactions and driver fatigue detection.

Head Pose‐Invariant Eyelid and Iris Tracking Method

SUMMARYThese days, there is more demand for a camera‐based gaze estimation method for new interfaces and new marketing measurement tools. Considering these applications, the system should track a new user without any operation such as calibration. It should also admit user's natural head pose changes. Previous methods, however, need a calibration procedure before execution and have less accuracy in a head moving situation. In this paper, we propose a method which tracks the user's eyelids and iris automatically and accurately. Our method is a pretreatment of gaze estimation without any calibration and head pose restraint. First of all, we track the facial feature points from an input face image and estimate its head pose, extracting the eye region image. On the eye region image, we track the eyelid shape based on an eyelid shape model generated beforehand from PCA. Finally we track the iris inside the eyelid based on the eyeball model. The eyelid and iris tracking is processed by Particle Filter. An evaluation against a database including head pose changes confirmed that the accuracy of eyelid and iris tracking was improved compared with previous methods.

Corneal-Imaging Calibration for Optical See-Through Head-Mounted Displays.

In recent years optical see-through head-mounted displays (OST-HMDs) have moved from conceptual research to a market of mass-produced devices with new models and applications being released continuously. It remains challenging to deploy augmented reality (AR) applications that require consistent spatial visualization. Examples include maintenance, training and medical tasks, as the view of the attached scene camera is shifted from the user's view. A calibration step can compute the relationship between the HMD-screen and the user's eye to align the digital content. However, this alignment is only viable as long as the display does not move, an assumption that rarely holds for an extended period of time. As a consequence, continuous recalibration is necessary. Manual calibration methods are tedious and rarely support practical applications. Existing automated methods do not account for user-specific parameters and are error prone. We propose the combination of a pre-calibrated display with a per-frame estimation of the user's cornea position to estimate the individual eye center and continuously recalibrate the system. With this, we also obtain the gaze direction, which allows for instantaneous uncalibrated eye gaze tracking, without the need for additional hardware and complex illumination. Contrary to existing methods, we use simple image processing and do not rely on iris tracking, which is typically noisy and can be ambiguous. Evaluation with simulated and real data shows that our approach achieves a more accurate and stable eye pose estimation, which results in an improved and practical calibration with a largely improved distribution of projection error.

A robust perception based method for iris tracking

The paper presents an application of the modified kernel based object tracking to iris tracking. Human perception rules are used for defining a proper feature space for iris that mainly accounts for the fact that eyes represent instinctive visual fixation regions. In addition, a similarity metric that is close to the way human vision compares and perceives the difference between scene components has been employed for finding the exact iris location in subsequent frames. As a result, just one iteration of the mean shift algorithm allows us to get a faithful estimation of iris location in subsequent frames. This property makes the proposed algorithm implementable on mobile devices and useful for real time applications. Experimental results performed on MICHE database show that the proposed method is robust to changes in illumination or scale, iris partial or total occlusion for some subsequent frames, blinks.

Head-pose Invariant Eyelid and Iris Tracking Method

These days, there is more demand for a camera-based gaze estimation method for new interfaces and new marketing measurement tools. Considering these applications, the system should track a new user without any operation such as calibration. It should also admit user's natural head pose changes. Previous methods, however, need a calibration procedure before execution and have less accuracy in a head moving situation. In this paper, we propose a method which tracks the user's eyelids and iris automatically and accurately. Our method is a pretreatment of gaze estimation without any calibration and head pose restraint. First of all, we track the facial feature points from an input face image and estimate its head pose, extracting the eye region image. On the eye region image, we track the eyelid shape based on an eyelid shape model generated beforehand from PCA. Finally we track the iris inside the eyelid based on the eyeball model. The eyelid and iris tracking is processed by Particle Filter. An evaluation against a database including head pose changes confirmed that the accuracy of eyelid and iris tracking was improved compared with previous methods.

Hierarchical On-line Appearance-Based Tracking for 3D head pose, eyebrows, lips, eyelids and irises

In this paper, we propose an On-line Appearance-Based Tracker (OABT) for simultaneous tracking of 3D head pose, lips, eyebrows, eyelids and irises in monocular video sequences. In contrast to previously proposed tracking approaches, which deal with face and gaze tracking separately, our OABT can also be used for eyelid and iris tracking, as well as 3D head pose, lips and eyebrows facial actions tracking. Furthermore, our approach applies an on-line learning of changes in the appearance of the tracked target. Hence, the prior training of appearance models, which usually requires a large amount of labeled facial images, is avoided. Moreover, the proposed method is built upon a hierarchical combination of three OABTs, which are optimized using a Levenberg–Marquardt Algorithm (LMA) enhanced with line-search procedures. This, in turn, makes the proposed method robust to changes in lighting conditions, occlusions and translucent textures, as evidenced by our experiments. Finally, the proposed method achieves head and facial actions tracking in real-time.

Iris-Eyelid Separability Filter for Irises Tracking

This paper describes a sophisticated method to track irises in a monocular video sequence with a particle filter that uses a newly proposed Iris-Eyelid separability filter (IESF). In order to reduce the influence of eyelids and to cope with the variance of the appearance of eyes of different people, we propose an Iris-Eyelid separability filter (IESF), and use it to estimate the likelihood of hypotheses in the particle filter. We confirmed that our method has the ability of tracking various eyes for different people whose head motion including translation, rotation and zoom, even when people are wearing glasses. Through the comparative experiments of the IESF and the conventional circular separability filter (CCSF), we confirmed that our iris tracking algorithm is more robustly.

Eye movement analysis for activity recognition based on one web camera

Eye movement information is the key clue for recognizing the vision-dominated tasks, such as browsing the web, or watching a video. However, traditional wearable sensors are invasive and the vision-based eye trackers are very expensive and need time consuming calibration. Therefore, an activity recognition method based on eye movement analysis under one web camera is first proposed and the feasibility is assessed. First, an iris tracking method for the low quality image is proposed to acquire eye movement information. Then, five ten novel features are extracted from the horizontal and the vertical eye movement signals for activity recognition, and the optimal feature subset is selected. Finally, the support vector machine is used to assess the feasibility of the proposed method. Three experiments are designed for different applications: leave-one-out cross-validation, k-fold cross-validation, and validation after respective calibration. Experimental results show that their accuracies are 68.4%, 79.3% and 84.1%, respectively, which demonstrate the promise of eye based activity recognition using one web camera.

Controlling Devices Using Biological Signals

Knowing that the driving task of a conventional wheelchair could be difficult or even impossible for impairment people, this work presents an overview of some strategies developed to aid these people. Within this context, a myoelectrical eye-blink and an iris tracking system to guide a robotic wheelchair are briefly described. Futhermore, some comments about EEG-based systems are also presented. Finally, it is presented a robotic wheelchair navigation system capable to reach a desired pose in a planar environment while avoiding static and dynamic obstacles.