Pupil Detection Research Articles

The importance of monitoring vergence eye movements for solutions using virtual technologies

Eyesight monitoring systems are of particular interest to virtual technology research teams and concern mainly three key aspects: physiological, technical and functional. The first one is to solve the so-called conflict of vergence and accommodation. In contrast to typical gaming applications, e.g. vision therapy requires the ability to smoothly transfer attention between the points of near and far. The technical aspect is primarily the search for new measurement methods, which is particularly important in the case of mobile goggles with limited computing capabilities of the processor. The last aspect concerns the possibility of creating effective HMI (Human Machine Interface) solutions. Full Text: PDF ReferencesJ. D. Bayliss et. al., "Lazy Eye Shooter: a Novel Game Therapy for Visual Recovery in Adult Amblyopia", 2012 IEEE International Games Innovation Conference CrossRef Y. L. Chen., L. Shi, J. W. L. Lewis, M. Wang, "Infrared Retinoscopy", Photonics 2014, 1, 303-322. CrossRef W. Fuhl, M. Tonsen, A. Bulling, and E. Kasneci, "Pupil detection for head-mounted eye tracking in the wild: an evaluation of the state of the art," Mach. Vis. Appl., vol. 27, no. 8, pp. 1275-1288, 2016 CrossRef N. Sarkar, D. Strathearn, G. Lee, M. Olfat, A. Rohani and R. R. Mansour, "A large angle, low voltage, small footprint micromirror for eye tracking and near-eye display applications", 2015 Transducers - 2015 18th International Conference on Solid-State Sensors, Actuators and Microsystems (TRANSDUCERS), Anchorage, AK, 2015, pp. 855-858, CrossRef

Towards Accurate Pupil Detection Based on Morphology and Hough Transform

Automatic recognition of individuals is very important in modern eras. Biometric techniques have emerged as an answer to the matter of automatic individual recognition. This paper tends to give a technique to detect pupil which is a mixture of easy morphological operations and Hough Transform (HT) is presented in this paper. The circular area of the eye and pupil is divided by the morphological filter as well as the Hough Transform (HT) where the local Iris area has been converted into a rectangular block for the purpose of calculating inconsistencies in the image. This method is implemented and tested on the Chinese Academy of Sciences (CASIA V4) iris image database 249 person and the IIT Delhi (IITD) iris database v1 using MATLAB 2017a. This method has high accuracy in the center and radius finding reaches 97% for 2268 iris on CASIA V4 image and 99.77% for 2240 iris images on IITD, the speed is acceptable compared to the real-time detection speed and stable performance.

Görüntü İşleme Teknikleri Kullanarak Anizokori Hastalığını Teşhis Eden Uygulama

Human pupil abnormalities can be an indicator of many diseases. Anisocoria is a common condition that has been estimated at 20% of the average population. It is specified by inequality in the size of the pupils of the eyes. This paper proposes an anisocoria determining algorithm from a digital image by using the MATLAB computing environment that involves the usage of MATLAB Computer Vision and Image Processing Toolbox. The image used in this work as input data is an image that has been fetched from Siblings Image Database. An input image where anisocoria is present has been downloaded from the Internet. The paper gives an idea of understanding how pupil detection and measurement can be used in medical and psychology diagnostics by using a simple algorithm.

PDIF: Pupil Detection After Isolation and Fitting

Pupil detection plays a key role in eye and gaze video-based tracking algorithms. Various algorithms have been proposed through the years in order to improve the performances or the robustness in real-world scenarios. However, the development of an algorithm which excels in both execution time and pupil detection precision is still an open challenge. This paper presents a novel, feature-based eye-tracking algorithm for pupil detection. Morphological operators are used to remove corneal reflections and to reduce noise in the pupil area prior to the pupil detection step: this solution allows to significantly reduce the computational overhead without lowering the tracking precision. Moreover, a shape validation step is performed after the elliptical fitting and, if the elliptical shape is not detected properly, a set of additional steps is performed to improve the pupil estimation. The proposed solution, Pupil Detection after Isolation and Fitting (PDIF), has been compared with other state-of-the-art tracking algorithms that use morphological operations such as ElSe (Ellipse Selection) and ExCuSe (Exclusive Curve Selector) to evaluate both speed and robustness; the proposed algorithm has been tested over numerous datasets offering different pupil detection challenges. Obtained results show how PDIF provides comparable tracking precision at a significantly lower computational cost compared to ElSe and ExCuSe.

Robust Drowsiness Detection for Vehicle Driver using Deep Convolutional Neural Network

Drowsiness detection during driving is still an unsolved research problem which needs to be addressed to reduce road accidents. Researchers have been trying to solve this problem using various methods where most of these solution lacks behind in accuracy, real-time performance, costly, complex to build, and has a higher computational cost with low frame rate. This research proposes robust method for drowsiness detection of vehicle drivers based on head pose estimation and pupil detection by extracting facial region initially. Proposed method used frame aggregation strategy in case of face region cannot be extracted in any frame due to shortcomings, i.e. light reflection, shadow. In order to improve identification under highly varying lighting conditions, proposed research used cascade of regressors cutting edge method where each regression refers estimation of facial landmarks. Proposed method used deep convolutional neural network (DCNN) for accurate pupil detection to learn non linear data pattern. In this context, challenges of varying illumination, blurring and reflections for robust pupil detection are overcome by using batch normalization for stabilizing distributions of internal activations during training phase which makes overall methodology less influenced by parameter initialization. Proposed research performed extensive experimentation where accuracy rate of 98.97% was achieved using frame rate of 35 fps which is higher comparing with previous research results. Experimental results reveal the effectiveness of the proposed methodology.

Pupil Detection Based on Oblique Projection Using a Binocular Camera

The real visual attention areas of students during the learning process are vital data. They represent the students’ visual concentration and provide strong support for the analysis of learning effects. It is necessary to use professional equipment to collect the visual attention areas of students on a screen but this equipment is expensive, so it cannot be solely depended upon. A binocular camera can directly obtain the depth information under static conditions; thus, detected pupil and iris sizes do not change as the user-to-screen distance changes. Robust pupil detection is an important prerequisite for gaze detection in a real-world setting. Hence, an accurate algorithm to calculate the position of pupils and the size of irises under natural light is proposed in this paper. It is based on the positive and negative oblique projection of relative linear density. The MPIIGaze dataset was used to test the algorithm, and the experimental results show that the algorithm is resistant to changes in illumination and the presence of glasses. Moreover, compared with a system based on a monocular camera, the error in the distance between label points and gaze location is decreased by 5 pixels and dispersion is decreased by 3 pixels. Finally, the line of sight is concentrated and the dispersion is low. With the ‘user-screen’ distance between 60cm and 80cm, the accuracy can reach up to 1.5° to 2.2°.

A Robust and Accurate Particle Filter-Based Pupil Detection Method for Big Datasets of Eye Video

Accurate detection of pupil position in successive frames of eye videos is finding applications in many areas including assistive systems and E-learning. Processing the big datasets of eye videos in such systems requires robust and fast eye-tracking algorithms that can predict the position of eye pupil in consecutive video frames. As a major technique, particle filters provide adequate speed but have a low detection rate. To solve this problem, the present paper suggests the use of genetic algorithms in the sampling step of the particle filter technique. As a result, in each frame, the variety of particles required for predicting the pupil position in the next video frame is maintained and their uniformity is reduced. Finally, the speed and detection rate of the proposed method, as well as the basic particle filter method in predicting the pupil position in video frames are calculated and compared for various populations. The experimental results indicate that, in comparison with the basic particle filter algorithm, the proposed algorithm detects the pupil more accurately and in a shorter time. Also, by achieving an average detection rate of 79.89% in estimation of the pupil center with an error of five pixels on a variety of eye videos with different situations of occlusion and illumination, not only the robustness of the proposed method is assessed but also its superiority to the state-of-the-art methods is evinced.

POnline: An Online Pupil Annotation Tool Employing Crowd-sourcing and Engagement Mechanisms

Pupil center and pupil contour are two of the most important features in the eye-image used for video-based eye-tracking. Well annotated databases are needed in order to allow benchmarking of the available- and new pupil detection and gaze estimation algorithms. Unfortunately, creation of such a data set is costly and requires a lot of efforts, including manual work of the annotators. In addition, reliability of manual annotations is hard to establish with a low number of annotators. In order to facilitate progress of the gaze tracking algorithm research, we created an online pupil annotation tool that engages many users to interact through gamification and allows utilization of the crowd power to create reliable annotations \cite{artstein2005bias}. We describe the tool and the mechanisms employed, and report results on the annotation of a publicly available data set. Finally, we demonstrate an example utilization of the new high-quality annotation on a comparison of two state-of-the-art pupil center algorithms.

POnline: An Online Pupil Annotation Tool Employing Crowd-sourcing and Engagement Mechanisms

Pupil center and pupil contour are two of the most important features in the eye-image used for video-based eye-tracking. Well annotated databases are needed in order to allow benchmarking of the available- and new pupil detection and gaze estimation algorithms. Unfortunately, creation of such a data set is costly and requires a lot of efforts, including manual work of the annotators. In addition, reliability of manual annotations is hard to establish with a low number of annotators. In order to facilitate progress of the gaze tracking algorithm research, we created an online pupil annotation tool that engages many users to interact through gamification and allows utilization of the crowd power to create reliable annotations \cite{artstein2005bias}. We describe the tool and the mechanisms employed, and report results on the annotation of a publicly available data set. Finally, we demonstrate an example utilization of the new high-quality annotation on a comparison of two state-of-the-art pupil center algorithms.

Optical Mouse Sensor for Eye Blink Detection and Pupil Tracking: Application in a Low-Cost Eye-Controlled Pointing Device

In this paper, a new application of the optical mouse sensor is presented. The optical mouse is used as a main low-cost infrared vision system of a new proposal of a head-mounted human-computer interaction (HCI) device controlled by eye movements. The default optical mouse sensor lens and illumination source are replaced in order to improve its field of view and capture entire eye images. A complementary 8-bit microcontroller is used to acquire and process these images with two optimized algorithms to detect forced eye blinks and pupil displacements which are translated to computer pointer actions. This proposal introduces an inexpensive and approachable plug and play (PnP) device for people with severe disability in the upper extremities, neck, and head. The presented pointing device performs standard computer mouse actions with no extra software required. It uses the human interface device (HID) standard class of the universal serial bus (USB) increasing its compatibility for most computer platforms. This new device approach is aimed at improving comfortability and portability of the current commercial devices with simple installation and calibration. Several performance tests were done with different volunteer users obtaining an average pupil detection error of 0.34 pixels with a successful detection in 82.6% of all mouse events requested by means of pupil tracking.

Automating, Analyzing and Improving Pupillometry with Machine Learning Algorithms

The investigation of the pupillary light reflex (PLR) is a well-known method to provide information about the functionality of the autonomic nervous system. Pupillometry, a non-invasive technique, was applied to study the PLR alterations in a new, schizophrenia-like rat substrain, named WISKET. The pupil responses to light impulses were recorded with an infrared camera; the videos were automatically processed and features were extracted. Besides the classical statistical analysis (ANOVA), feature selection and classification were applied to reveal the significant differences in the PLR parameters between the control and WISKET animals. Based on these results, the disadvantages of this method were analyzed and the measurement process was redesigned and improved. The automated pupil detection method has also been adapted to the new videos. 2564 images were annotated manually and used to train a fully-convolutional neural network to produce pupil mask images. The method was evaluated on 329 test images and achieved 4% median relative error. With the new setup, the pupil detection became reliable and the new data acquisition offers robustness to the experiments.

Eyelid and Pupil Landmark Detection and Blink Estimation Based on Deformable Shape Models for Near-Field Infrared Video

The eyelid contour, pupil contour and blink event are important features of eye activity, and their estimation is a crucial research area for emerging wearable camera-based eyewear in a wide range of applications e.g. mental state estimation. Current approaches often estimate a single eye activity, such as blink or pupil center, from far-field and non-infrared (IR) eye images, and often depend on the knowledge of other eye components. This paper presents a unified approach to simultaneously estimate the landmarks for the eyelids, the iris and the pupil, and detect blink from near-field IR eye images based on a statistically learned deformable shape model and local appearance. Unlike the facial landmark estimation problem, by comparison, different shape models are applied to all eye states – closed eye, open eye with iris visible, and open eye with iris and pupil visible – to deal with the self-occluding interactions among the eye components. The most likely eye state is determined based on the learned local appearance. Evaluation on three different realistic datasets demonstrates that the proposed three-state deformable shape model achieves state-of-the-art performance for the open eye with iris and pupil state, where the normalized error was lower than 0.04. Blink detection can be as high as 90% in recall performance, without direct use of pupil detection. Cross-corpus evaluation results show that the proposed method improves on the state-of-the-art eyelid detection algorithm. This unified approach greatly facilitates eye activity analysis for research and practice when different types of eye activity are required rather than employ different techniques for each type. Our work is the first study proposing a unified approach for eye activity estimation from near-field IR eye images and achieved the state-of-the-art eyelid estimation and blink detection performance.

Pupil Detection Algorithm Based on Feature Extraction for Eye Gaze

Exact real-time pupil tracking is an essential step in a live eye gaze. Since pupil centre is a base point’s reference, eye centre localization is essential for many applications. In this research, we extract pupil eye features exactly within different intensity levels of eye images, mostly with localization of determined interest objects and where the human is looking for. Since it’s a digital world and digital transformation, everything is becoming virtual. Hence this concept has a huge scope in e-learning, class room training and analyzing human behaviour. This research covers eye tracking technology to track and analyze the learners' behavior and emotion on e-learning platform like level of attention and tiredness. Harr’s cascade classifier was used to first locate the eye’s area, and once found and support vector machine (SVM) for classification with the trained datasets. We also include the state of emotions, facial landmarks of the salient patches on face image using automated learning-free facial landmark detection technique.Experimental results help in developing learner eye gaze detection in system using Pycharm and hardware output using Raspberry Pi. In Raspberry Pi is given with the input image captured using external webcam and based on the engagement level of the learner content 1 or 2 would be displayed in the Raspbian OS environment

Development of an Eye Tracking-Based Human-Computer Interface for Real-Time Applications.

In this paper, the development of an eye-tracking-based human–computer interface for real-time applications is presented. To identify the most appropriate pupil detection algorithm for the proposed interface, we analyzed the performance of eight algorithms, six of which we developed based on the most representative pupil center detection techniques. The accuracy of each algorithm was evaluated for different eye images from four representative databases and for video eye images using a new testing protocol for a scene image. For all video recordings, we determined the detection rate within a circular target 50-pixel area placed in different positions in the scene image, cursor controllability and stability on the user screen, and running time. The experimental results for a set of 30 subjects show a detection rate over 84% at 50 pixels for all proposed algorithms, and the best result (91.39%) was obtained with the circular Hough transform approach. Finally, this algorithm was implemented in the proposed interface to develop an eye typing application based on a virtual keyboard. The mean typing speed of the subjects who tested the system was higher than 20 characters per minute.

Relative Afferent Pupillary Defect Screening Through Transfer Learning.

Abnormalities in pupillary light reflex can indicate optic nerve disorders that may lead to permanent visual loss if not diagnosed in an early stage. In this study, we focus on relative afferent pupillary defect (RAPD), which is based on the difference between the reactions of the eyes when they are exposed to light stimuli. Incumbent RAPD assessment methods are based on subjective practices that can lead to unreliable measurements. To eliminate subjectivity and obtain reliable measurements, we introduced an automated framework to detect RAPD. For validation, we conducted a clinical study with lab-on-a-headset, which can perform automated light reflex test. In addition to benchmarking handcrafted algorithms, we proposed a transfer learning-based approach that transformed a deep learning-based generic object recognition algorithm into a pupil detector. Based on the conducted experiments, proposed algorithm RAPDNet can achieve a sensitivity and a specificity of 90.6% over 64 test cases in a balanced set, which corresponds to an AUC of 0.929 in ROC analysis. According to our benchmark with three handcrafted algorithms and nine performance metrics, RAPDNet outperforms all other algorithms in every performance category.

Automatic Detection of Driver Impairment Based on Pupillary Light Reflex

The main objective of this paper is to determine the feasibility of designing a driver drunkenness detection system based on the dynamic analysis of a subject’s pupillary light reflex (PLR). This involuntary reaction is widely utilized in the medical field to diagnose a variety of diseases, and in this paper, the effectiveness of such a method to reveal an impairment condition due to alcohol abuse is evaluated. The test method consists in applying a light stimulus to one eye of the subject and to capture the dynamics of constriction of both eyes; for extracting the pupil size profiles from the video sequences, a two-step methodology is described, where in the first phase, the iris/pupil search within the image is performed, and in the second stage, the image is cropped to perform pupil detection on a smaller image to improve time efficiency. The undesired pupil dynamics arising in the PLR are defined and evaluated; a spontaneous oscillation of the pupil diameter is observed in the range [0, 2] Hz and the accommodation reflex causes pupil constriction of about 10% of the iris diameter. A database of pupillary light responses is acquired on different subjects in baseline condition and after alcohol consumption, and for each one, a first-order model is identified. A set of features is introduced to compare the two populations of responses and is used to design a support vector machine classifier to discriminate between “Sober” and “Drunk” states.

XSDL: stroboscopic differential lighting eye tracker with extended temporal support

Eye tracking (ET) for gaze interaction in wearable computing imposes harder constraints on computational efficiency and illumination conditions than remote ET. In this paper we present xSDL, an extended temporal support computer vision algorithm for accurate, robust, and efficient pupil detection and gaze estimation. The robustness and efficiency of xSDL partly come from the use of stroboscopic differential lighting (SDL), an extension of the differential lighting pupil detection technique developed in the 90’s. Due to the erratic behavior of eye movements, traditional computer vision tracking techniques (such as Kalman filters) do not perform well, so most ET techniques simply detect some eye feature (such as the pupil center) at every frame. Extended temporal support uses keyframes selected during eye fixations and a simple translation model of the pupil to further improve the computational performance of SDL. A prototype composed of two independent acquisition systems was developed to evaluate the performance of xSDL and other four state-of-the-art ET techniques under similar conditions. Our results show that xSDL outperforms those four algorithms, both in speed (close to 2000 Hz using 240 line frames) and accuracy.

Deep Residual CNN-Based Ocular Recognition Based on Rough Pupil Detection in the Images by NIR Camera Sensor.

Accurate segmentation of the iris area in input images has a significant effect on the accuracy of iris recognition and is a very important preprocessing step in the overall iris recognition process. In previous studies on iris recognition, however, the accuracy of iris segmentation was reduced when the images of captured irises were of low quality due to problems such as optical and motion blurring, thick eyelashes, and light reflected from eyeglasses. Deep learning-based iris segmentation has been proposed to improve accuracy, but its disadvantage is that it requires a long processing time. To resolve this problem, this study proposes a new method that quickly finds a rough iris box area without accurately segmenting the iris region in the input images and performs ocular recognition based on this. To address this problem of reduced accuracy, the recognition is performed using the ocular area, which is a little larger than the iris area, and a deep residual network (ResNet) is used to resolve the problem of reduced recognition rates due to misalignment between the enrolled and recognition iris images. Experiments were performed using three databases: Institute of Automation Chinese Academy of Sciences (CASIA)-Iris-Distance, CASIA-Iris-Lamp, and CASIA-Iris-Thousand. They confirmed that the method proposed in this study had a higher recognition accuracy than existing methods.

An FPGA-Based Real-Time Human Eye Pupil Detection System Using E2V Smart Camera

Threshold based pupil detection algorithm was found tobe most efficient method to detect human eye. An implementation of a real-time system on an FPGA board to detect and track a human's eye is the main motive to obtain from proposed work. The Pupil detection algorithm involved thresholding and image filtering. The Pupil location was identified by computing the center value of the detected region. The proposed hardware architecture is designed using Verilog HDL and implemented on aAltera DE2 cyclone II FPGA for prototyping and logic utilizations are compared with Existing work. The overall setup included Cyclone II FPGA, a E2V camera, SDRAM and a VGA monitor. Experimental results proved the accuracy and effectiveness of the hardware realtime implementation as the algorithm was able to manage various types of input video frame. All calculation was performed in real time. Although the system can be furthered improved to obtain better results, overall the project was a success as it enabled any inputted eye to be accurately detected and tracked.

Pupil detection and tracking for AR 3D under various circumstances

Pupil detection and tracking for AR 3D under various circumstances