Pupil Detection Research Articles

Cascaded learning with transformer for simultaneous eye landmark, eye state and gaze estimation

Eye tracking have garnered attention in human–machine interaction, disease monitoring, biometrics, etc. Existing investigations for eye tracking have predominantly concentrated on individual task for pupil detection or gaze estimation, overlooking the implicit relationships that exist among different tasks for eye tracking. In this work, we introduce a cascaded framework with transformer to collaboratively realize eye landmark detection, eye state detection and gaze estimation. Within our framework, we leverage Transformer to capture long dependencies with explicit eye-related structural information and implicit correlation among different tasks. Furthermore, the proposed cascade iteration framework alternatively optimize each task and boost the overall performance for pupil center, eye state and gaze estimation simultaneously. To address the problem of manual annotation, we further introduce the Control-Eye Diffusion Model (CEDM), a controllable eye image generation method conditioned on a simple contour with structure information. The proposed methods are evaluated on challenging datasets such as GI4E, BioID and MPIIGaze, and the results show that our methods outperform state-of-the-art methods in several tasks.

Cognitive load in tele-robotic surgery: a comparison of eye tracker designs.

Eye gaze tracking and pupillometry are evolving areas within the field of tele-robotic surgery, particularly in the context of estimating cognitive load (CL). However, this is a recent field, and current solutions for gaze and pupil tracking in robotic surgery require assessment. Considering the necessity of stable pupillometry signals for reliable cognitive load estimation, we compare the accuracy of three eye trackers, including head and console-mounted designs. We conducted a user study with the da Vinci Research Kit (dVRK), to compare the three designs. We collected eye tracking and dVRK video data while participants observed nine markers distributed over the dVRK screen. We compute and analyze pupil detection stability and gaze prediction accuracy for the three designs. Head-worn devices present better stability and accuracy of gaze prediction and pupil detection compared to console-mounted systems. Tracking stability along the field of view varies between trackers, with gaze predictions detected at invalid zones of the image with high confidence. While head-worn solutions show benefits in confidence and stability, our results demonstrate the need to improve eye tacker performance regarding pupil detection, stability, and gaze accuracy in tele-robotic scenarios.

Implementation of a High-Accuracy Neural Network-Based Pupil Detection System for Real-Time and Real-World Applications.

In this paper, the implementation of a new pupil detection system based on artificial intelligence techniques suitable for real-time and real-word applications is presented. The proposed AI-based pupil detection system uses a classifier implemented with slim-type neural networks, with its classes being defined according to the possible positions of the pupil within the eye image. In order to reduce the complexity of the neural network, a new parallel architecture is used in which two independent classifiers deliver the pupil center coordinates. The training, testing, and validation of the proposed system were performed using almost 40,000 eye images with a resolution of 320 × 240 pixels and coming from 20 different databases, with a high degree of generality. The experimental results show a detection rate of 96.29% at five pixels with a standard deviation of 3.38 pixels for all eye images from all databases and a processing speed of 100 frames/s. These results indicate both high accuracy and high processing speed, and they allow us to use the proposed solution for different real-time applications in variable and non-uniform lighting conditions, in fields such as assistive technology to communicate with neuromotor-disabled patients by using eye typing, in computer gaming, and in the automotive industry for increasing traffic safety by monitoring the driver's cognitive state.

EyeMo: A Solution for Individuals with Disabilities to Use a Computer Through Eye Movements

The maintenance of an upright posture and the execution of desired movements by individuals necessitate the seamless and harmonious functioning of the muscular and skeletal systems. Neuromuscular diseases, spinal cord injuries, and related conditions can adversely impact individuals' motor functions, leading to a loss of muscle strength and, in severe cases, paralysis. Such health challenges may result in social isolation and detachment from one's social life. This study is focused on the development of a computer control application utilizing eye movements to aid users in navigating and interacting with computers. The system utilizes a lightweight eyeglass frame and a mini-USB camera to accurately capture images of the user's pupil. Pupil detection is achieved through a combination of traditional image processing and deep learning techniques, ensuring high accuracy under diverse conditions. Users have the ability to interactively calibrate the application to accommodate varying screen sizes, thereby enhancing system responsiveness. The user interface incorporates functionalities such as left-click, right-click, double-click, and scrolling, empowering users to perform tasks ranging from internet browsing to video watching. 
 After the development of the application, a request for research permission was submitted to the local institution to allow volunteers to test the application, adhering to ethical standards. Following the approval of ethical clearance, the application was made available for use by volunteers, and feedback was systematically collected. Volunteers generally reported that the application was beneficial, indicating its potential use by disadvantaged individuals. The upcoming versions of the application have the potential for integration with smart home systems. Additionally, the developed application can be used in games and for educational purposes.

Robust residual convolutional neural network based pupil tracking for low-computational power applications

Existing Deep Fully Convolutional Network (DFCN) based pupil segmentation models have been shown to perform robust and accurate pupil detection. However, they lack high computational resources when real-time predictions for low computational wearable devices such as Raspberry Pi are of interest. Simple FCN (Fully Convolutional Network) models, on the other hand, can provide real-time yet not as accurate predictions with low computational power devices. In this study, we address this dilemma by proposing a CNN (Convolutional Neural Network)-based model, Residual CNN, with several advanced operations such as residual connections, Squeeze and Excitation (SE) attention, and Atrous Spatial Pyramid Pooling (ASPP) to improve the prediction performance without complicating the model. Moreover, we leverage transfer learning by training synthetic images, then fine-tuning them with authentic eye images. We also fully quantize our model parameters to speed up the predictions and apply the Quantization Aware Training (QAT) strategy to obtain accurate predictions. Our experiments show that our quantized Res-CNN model trained by QAT strategy with 40x30 resolution images provides robust yet real-time predictions with an average of 1.351 Root Mean Square Error (RMSE) for the pupil center predictions and 8.317 ms response time per image on Raspberry Pi.

High-speed pupil dynamic tracking algorithm for RAPD measurement equipment utilizing gray-level features

Relative pupillary afferent disorder (RAPD) plays a crucial role in diagnosing optic nerve dysfunction. This paper introduces an innovative equipment design with a high-speed pupil detection algorithm and a binocular independent stimulation optical path. The proposed algorithm utilizes the grayscale characteristics of the pupil region to achieve rapid and accurate pupil detection and tracking. Initially, a pupil threshold is estimated using eigenvalues, enabling the calculation of the pupil centroid. Subsequently, leveraging the unique characteristics of the pupil region, a dynamic tracking algorithm, a second-order partial derivative threshold algorithm, and a pupil diameter extraction algorithm are employed to precisely locate the centroid. By incorporating a binocular independent stimulus light path design, the algorithm overcomes limitations associated with the current measurement equipment. The experimental results demonstrate the algorithm’s high robustness and fast detection speed, meeting the tracking speed requirement of 1250 frames per second for a single eye. These advancements have the potential to significantly enhance the diagnosis and assessment of optic nerve dysfunction.

Journey tracker: driver alerting system with a deep learning approach.

Negligence of public transport drivers due to drowsiness poses risks not only to their own lives but also to the lives of passengers. The designed journey tracker system alerts the drivers and activates potential penalties. A custom EfficientNet model architecture, based on EfficientNet design principles, is built and trained using the Media Research Lab (MRL) eye dataset. Reflections in frames are filtered out to ensure accurate detections. A 10 min initial period is utilized to understand the driver's baseline behavior, enhancing the reliability of drowsiness detections. Input from drivers is considered to determine the frame rate for more precise real-time monitoring. Only the eye regions of individual drivers are captured to maintain privacy and ethical standards, fostering driver comfort. Hyperparameter tuning and testing of different activation functions during model training aim to strike a balance between model complexity, performance and computational cost. Obtained an accuracy rate of 95% and results demonstrate that the "swish" activation function outperforms ReLU, sigmoid and tanh activation functions in extracting hierarchical features. Additionally, models trained from scratch exhibit superior performance compared to pretrained models. This system promotes safer public transportation and enhances professionalism by monitoring driver alertness. The system detects closed eyes and performs a cross-reference using personalization data and pupil detection to trigger appropriate alerts and impose penalties.

Eye detection and coarse localization of pupil for video-based eye tracking systems

A video-based eye tracking system generally captures NIR images, each of which contains one or two eyes of a subject. The subject’s point of gaze is then determined using 3D eye model and pupil centre corneal reflection technique. Eye detection and pupil localization play significant roles in video-based eye tracking systems. However, face rotation, wearing glasses, eye-shape variation and illumination variation make it difficult to detect an eye and localize a pupil accurately in the images captured by video-based eye tracking systems. In this paper, we proposed an eye detector that adopts a coarse-to-fine strategy. The eye detector consists of three classifiers: an ATLBP-THACs feature-based cascade classifier, a branch CNN and a multi-task CNN. We also proposed a method for coarse pupil localization. Coarse localization is an important step for pupil localization since it can provide initial pupil coordinates for a fine pupil localization method. Given a downscaled eye image, a shallow CNN is used to estimate the location of seven landmarks. On this basis, pupil center and radius are estimated. A method for small dim target enhancement is used to increase the contrast between pupil and background. The main goal of pupil enhancement is to make it easier to binarize an eye image. At last, component filtering is made by utilizing the estimated pupil center and radius. We collected a dataset named neepuEYE dataset that consists of 5500 NIR eye images from 109 people. The images can be used to generate augmented samples for training an eye detector since they contain eyes with different shape, orientation and pupil localization. Experimental results show that our eye detector is a fast and robust detector. Furthermore, our method for coarse pupil localization can obtain not only high detection rate but also high localization speed.

Pupil Detection for Automatic Diagnosis of Eye Diseases Using Optimized Color Mapping

Introduction: Pupil and iris disorders form an important category of eye diseases. Accurate segmentation of the pupil is the first and most important step in the automatic diagnosis of diseases related to the pupil and iris. Most of the existing methods do not have enough accuracy and are sensitive to the effects of noise and specular spot reflection. In addition, the images used in these methods usually have limitations, such as the viewing angle . Method: In the proposed algorithm, a stable method is offered to remove the effects of specular spot reflection in the pupil, and necessary preprocessing is done to detect the exact location of the pupil. An optimized color mapping algorithm is proposed and the mapping is calculated with the help of the LM algorithm to accurately determine the pupil boundary. This method does not impose any restrictions on the eye image and shape, and the angle of the pupil in the image can be in any shape and direction . Results: The proposed method does not assume any specific model as the final pupil boundary (circle or oval) and is robust to noise and specular reflection factors as well. This method has been able to accurately detect the pupil boundary with the accuracy of 98.8% using UBIRIS dataset and 98% using the collected data by authors . Conclusion: The method presented in this paper can be used to increase the accuracy in determining the internal and external border of the iris to diagnose diseases related to the pupil and iris, as well as identity identification based on iris tissue.

EYE-DIRECTION-BASED SAFETY NAVIGATION SYSTEM FOR ELDERLY AND PHYSICALLY CHALLENGED PERSONS

This research paper includes the eye-direction-based safety automated navigation system that was implemented for the elderly and physically challenged people. The purpose of this navigation system is to avoid the assistance required for physically challenged people. This system, which controls the motorised wheelchair navigation, depends on pupil detection. The sequential images were captured via Bluetooth specs glass using an image processing technique. The system navigates in the directions specified by the user, such as "move to the left," "move to the right," "move forward," and "stop." Additionally, a sensor is fixed in front of the wheelchair to detect objects and avoid faulty navigation. A centralised wireless detector device is also available in a wheelchair for emergency purposes. A Raspberry Pi Model B is a high-speed detection kit controlled by the whole system.

The Human Eye Pupil Detection System Using BAT Optimized Deep Learning Architecture

The pupil recognition method is helpful in many real-time systems, including ophthalmology testing devices, wheelchair assistance, and so on. The pupil detection system is a very difficult process in a wide range of datasets due to problems caused by varying pupil size, occlusion of eyelids, and eyelashes. Deep Convolutional Neural Networks (DCNN) are being used in pupil recognition systems and have shown promising results in terms of accuracy. To improve accuracy and cope with larger datasets, this research work proposes BOC (BAT Optimized CNN)-IrisNet, which consists of optimizing input weights and hidden layers of DCNN using the evolutionary BAT algorithm to efficiently find the human eye pupil region. The proposed method is based on very deep architecture and many tricks from recently developed popular CNNs. Experiment results show that the BOC-IrisNet proposal can efficiently model iris microstructures and provides a stable discriminating iris representation that is lightweight, easy to implement, and of cutting-edge accuracy. Finally, the region-based black box method for determining pupil center coordinates was introduced. The proposed architecture was tested using various IRIS databases, including the CASIA (Chinese academy of the scientific research institute of automation) Iris V4 dataset, which has 99.5% sensitivity and 99.75% accuracy, and the IIT (Indian Institute of Technology) Delhi dataset, which has 99.35% specificity and MMU (Multimedia University) 99.45% accuracy, which is higher than the existing architectures.

3D Gaze Estimation Using RGB-IR Cameras.

In this paper, we present a framework for 3D gaze estimation intended to identify the user's focus of attention in a corneal imaging system. The framework uses a headset that consists of three cameras, a scene camera and two eye cameras: an IR camera and an RGB camera. The IR camera is used to continuously and reliably track the pupil and the RGB camera is used to acquire corneal images of the same eye. Deep learning algorithms are trained to detect the pupil in IR and RGB images and to compute a per user 3D model of the eye in real time. Once the 3D model is built, the 3D gaze direction is computed starting from the eyeball center and passing through the pupil center to the outside world. This model can also be used to transform the pupil position detected in the IR image into its corresponding position in the RGB image and to detect the gaze direction in the corneal image. This technique circumvents the problem of pupil detection in RGB images, which is especially difficult and unreliable when the scene is reflected in the corneal images. In our approach, the auto-calibration process is transparent and unobtrusive. Users do not have to be instructed to look at specific objects to calibrate the eye tracker. They need only to act and gaze normally. The framework was evaluated in a user study in realistic settings and the results are promising. It achieved a very low 3D gaze error (2.12°) and very high accuracy in acquiring corneal images (intersection over union-IoU = 0.71). The framework may be used in a variety of real-world mobile scenarios (indoors, indoors near windows and outdoors) with high accuracy.

Sistem Pakar Deteksi Covid-19 Dengan Lie Detection Menggunakan Metode Circle Hough Transform

The number of health workers infected with Covid-19 in Indonesia continues to grow amid the coronavirus pandemic. Not only doctors, nurses and other medical support staff have been exposed to Covid-19. To date, IDI has recorded more than 180 health workers who have died from Corona. Health workers are dying because of a lack of Personal Protective Equipment (PPE) and fatigue. Prevention of direct contact between asymptomatic patients and health workers is a way to prevent health workers from contracting COVID-19. An expert system for diagnosing COVID-19 with lie detection is proposed to be used for patients who wish to seek treatment at a health center before they meet face-to-face with health workers. Several previous studies have proven that the certainty method can be used to diagnose COVID-19 with an accuracy of up to 90%, provided that the patient answers questions honestly. In this study, control questions and pupil detection were added using the circle hard transform to find out whether patients who wanted treatment did not lie when answering questions about symptoms of exposure to Covid, travel history and family history of exposure to Covid. The combination of an expert system and lie detection is expected to be the first protective alternative for health workers from asymptomatic patients. Based on the results of the application testing carried out, it can be seen that the movement of the patient's pupils when answering questions.

Noisy iris smoothing and segmentation scheme based on improved Wildes method.

In an automated iris recognition system, in order to get higher accuracy, we should have an efficient iris segmentation process. The reliability of accurate “iris recognition” system largely depends on the accuracy of segmentation process. Traditional “iris segmentation” methods are unable to detect the exact boundaries of iris and pupil, which is time consuming and also highly sensitive to noise. To overcome these problems, we have proposed an improved Wildes method (IWM) for segmentation in iris recognition system. The proposed algorithm consists of two major steps before applying Wildes method for segmentation: edge detection of iris and pupil from a noisy eye image with improved Canny with fuzzy logic (ICWFL) and removal of unwanted noise from above step with a hybrid restoration fusion filter (HRFF). A comparative study of various edge detection techniques is performed to prove the efficiency of ICWFL method. Similarly, the proposed method is tested with various noise densities from 10 to 95 dB. Also the working of the proposed HRFF is compared with some existing smoothing filters. Various experiments have been performed with the help of iris database of IIT_Delhi. Both visual and numerical results prove the efficiency of the proposed algorithm.

Comparison of EOG and VOG obtained eye movements during horizontal head impulse testing.

IntroductionVideo head impulse testing is frequently used to evaluate the vestibular function. During this test, eye movement responses are recorded with video-oculography (VOG). However, the use of VOG can sometimes be challenging, especially due to pupil detection problems (e.g., blinking, droopy eyelids, etc.). Therefore, this study investigated whether electro-oculography (EOG), a technique that does not depend on pupil tracking but on the orientation of the corneoretinal potential, might be an alternative to VOG for quantifying eye movement responses during head impulse testing.Subjects and methodsHead impulse testing was performed in 19 healthy subjects without a prior history of vestibular symptoms. Horizontal eye movements were recorded simultaneously with EOG (using an EOG system) and VOG (using a VHIT system: ICS Impulse). The eye movement responses to each side of both techniques were compared using a concordance correlation coefficient (rc), t-testing, and Bayes Factor (BF) paired t-testing.ResultsEOG and VOG obtained eye movement traces that correlated well with each other during head impulse testing (average rc = 0.89). Average VOR gains obtained with EOG and VOG were not significantly different from each other for all subjects during left head impulses. However, VOG gains differed between both techniques regarding right head impulses. VOG showed significant VOR gain asymmetry (5% to the right), whereas EOG showed no significant asymmetry (1% to the right).ConclusionThis study demonstrated the use of EOG to record eye movements during head impulse testing for the first time. EOG and VOG obtained eye movement traces that correlated well with each other during horizontal head impulse testing. In addition, EOG showed smaller VOR gain asymmetry in healthy individuals, in contrast to VOG. These findings indicate that EOG might potentially be applicable as an alternative to VOG for collecting eye movement responses during head impulse testing.Trial registration number10192021-38 dated 19.10.21.

Pupillometry examinations of the human eye with the eye diagnostic device PEP-2000 – First results

Abstract Pupillometry forms the diagnostic basis for numerous pathologies of the eye. For this reason, fast and accurate diagnostics in the field of ophthalmology are essential. Two examination techniques, full-field ERG and pupillometry were combined in a diagnostic device developed by ICM e.V. to reduce the examination process for both examiners and patients. In this paper, the device is examined for the quality of the pupillometry measurements. A study with 12 healthy subjects (3 f, 9 m, 36.33 ± 11.94 years) was conducted to evaluate the device. The results showed that the minimal pupil diameter is 40 % higher than the literature values. The main reason for the differences is the low light intensity of 15 cd/m2. However, the maximum pupil diameter is within the range of the researched values. The results of the pupillary reaction measurements show that the values obtained (amplitude, contraction time and peak time) are within the range of literature values. The latency time of 690 ms is 40 % too high. The reason for this could be the moderate pupil detection rate of 50-70 %. Nevertheless, plausible and comparable analysis values could be obtained with the eye diagnostic device PEP-2000. Further work will look at improving pupil detection rates.

Pupil Detection Using Hybrid Vision Transformer

Pupil detection is an indispensable part of the process of eye-tracking. Due of the limitation of existing methods on pupil image quality, we propose a pupil detection method using vision transformer with a hybrid structure. We first extract the local features of the image with CNN, and then obtain the global dependence through the encoder of the transformer, to excavate more accurate information on pupil position. We trained and tested the proposed model on 10 600 images from three publicly available datasets and compared with other pupil detection models. The analysis of the outcomes demonstrated that the hybrid vision transformer was superior to these comparison approaches in terms of accuracy and robustness in locating the pupil position. It achieved a detection rate of more than 90% for pupils within a 5-pixel error in all evaluated datasets.

Keras Deep Learning for Pupil Detection Method

Nowadays, deep learning is the most common field in artificial intelligence, deep learning is a technique invented by man in order to try to imitate the way the human mind works. Deep learning tries to simulate the human mind in all its capabilities, which include; Seeing, understanding speech, composing it, hearing, and other powerful abilities that our human mind possesses and is not rivaled by anything else, or so it was. The matter did not stop at that only, but that scientists have studied the human brain and how it works in order to design algorithms and programs capable of simulating it, and for this reason we find that these algorithms are inspired by medical and neurological studies of humans and try as much as possible to imitate them, but by computer methods. Not biological. In this paper studied and practiced the multi-task deep convolution neural network(MTCNN) for face detection after detecting the faces its try to extract eyes from image, finally pupil detector works by using Keras model in python. The result of this paper shows the power of deep learning field biologically comparative with human thinks.

Robust and High Accuracy Algorithm for Detection of Pupil Images

Recently, many researchers have tried to develop a robust, fast, and accurate algorithm. This algorithm is for eye-tracking and detecting pupil position in many applications such as head-mounted eye tracking, gaze-based human-computer interaction, medical applications (such as deaf and diabetes patients), and attention analysis. Many real-world conditions challenge the eye appearance, such as illumination, reflections, and occasions. On the other hand, individual differences in eye physiology and other sources of noise, such as contact lenses or make-up. The present work introduces a robust pupil detection algorithm with and higher accuracy than the previous attempts for real-time analytics applications. The proposed circular hough transform with morphing canny edge detection for Pupillometery (CHMCEP) algorithm can detect even the blurred or noisy images by using different filtering methods in the pre-processing or start phase to remove the blur and noise and finally the second filtering process before the circular Hough transform for the center fitting to make sure better accuracy. The performance of the proposed CHMCEP algorithm was tested against recent pupil detection methods. Simulations and results show that the proposed CHMCEP algorithm achieved detection rates of 87.11, 78.54, 58, and 78 according to Świrski, ExCuSe, Else, and labeled pupils in the wild (LPW) data sets, respectively. These results show that the proposed approach performs better than the other pupil detection methods by a large margin by providing exact and robust pupil positions on challenging ordinary eye pictures.

LAD: A Hybrid Deep Learning System for Benign Paroxysmal Positional Vertigo Disorders Diagnostic

Herein, we introduce “Look and Diagnose” (LAD), a hybrid deep learning-based system that aims to support doctors in the medical field for diagnosing effectively the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Benign Paroxysmal Positional Vertigo</i> (BPPV) disorder. Given the body postures of the patient in the Dix-Hallpike and lateral head turns test, the visual information of both eyes is captured and fed into LAD for analyzing and classifying into one of six possible disorders which the patient might be suffering from. The proposed system consists of two streams: (1) an RNN-based stream that takes raw RGB images of both eyes to extract visual features and optical flow of each eye followed by ternary classification to determine left/right posterior canal (PC) or other; and (2) pupil detector stream that detects the pupil when it is classified as Non-PC and classifies the direction and strength of the beating to categorize the Non-PC types into the remaining four classes: <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Geotropic</i> BPPV (left and right) and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Apogeotropic</i> BPPV (left and right). Experimental results show that with the given body postures of the patient, the system is capable of accurately classifying given BPPV disorder into the six types of disorder with an accuracy of 91% on the validation set. The proposed method can successfully classify disorders with an accuracy of 93% for the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Posterior Canal</i> disorder and 95% for the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Geotropic</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Apogeotropic</i> disorder, paving a potential direction for research with the medical data.