Pupil Location Research Articles

An Irregular Pupil Localization Network Driven by ResNet Architecture

An Irregular Pupil Localization Network Driven by ResNet Architecture

Methods for Detecting the Patient’s Pupils’ Coordinates and Head Rotation Angle for the Video Head Impulse Test (vHIT), Applicable for the Diagnosis of Vestibular Neuritis and Pre-Stroke Conditions

In the contemporary era, dizziness is a prevalent ailment among patients. It can be caused by either vestibular neuritis or a stroke. Given the lack of diagnostic utility of instrumental methods in acute isolated vertigo, the differentiation of vestibular neuritis and stroke is primarily clinical. As a part of the initial differential diagnosis, the physician focuses on the characteristics of nystagmus and the results of the video head impulse test (vHIT). Instruments for accurate vHIT are costly and are often utilized exclusively in healthcare settings. The objective of this paper is to review contemporary methodologies for accurately detecting the position of pupil centers in both eyes of a patient and for precisely extracting their coordinates. Additionally, the paper describes methods for accurately determining the head rotation angle under diverse imaging and lighting conditions. Furthermore, the suitability of these methods for vHIT is being evaluated. We assume the maximum allowable error is 0.005 radians per frame to detect pupils’ coordinates or 0.3 degrees per frame while detecting the head position. We found that for such conditions, the most suitable approaches for head posture detection are deep learning (including LSTM networks), search by template matching, linear regression of EMG sensor data, and optical fiber sensor usage. The most relevant approaches for pupil localization for our medical tasks are deep learning, geometric transformations, decision trees, and RASNAC. This study might assist in the identification of a number of approaches that can be employed in the future to construct a high-accuracy system for vHIT based on a smartphone or a home computer, with subsequent signal processing and initial diagnosis.

Research on Pupil Center Localization Detection Algorithm with Improved YOLOv8

Addressing issues such as low localization accuracy, poor robustness, and long average localization time in pupil center localization algorithms, an improved YOLOv8 network-based pupil center localization algorithm is proposed. This algorithm incorporates a dual attention mechanism into the YOLOv8n backbone network, which simultaneously attends to global contextual information of input data while reducing dependence on specific regions. This improves the problem of difficult pupil localization detection due to occlusions such as eyelashes and eyelids, enhancing the model’s robustness. Additionally, atrous convolutions are introduced in the encoding section, which reduce the network model while improving the model’s detection speed. The use of the Focaler-IoU loss function, by focusing on different regression samples, can improve the performance of detectors in various detection tasks. The performance of the improved Yolov8n algorithm was 0.99971, 1, 0.99611, and 0.96495 in precision, recall, MAP50, and mAP50-95, respectively. Moreover, the improved YOLOv8n algorithm reduced the model parameters by 7.18% and the computational complexity by 10.06%, while enhancing the environmental anti-interference ability and robustness, and shortening the localization time, improving real-time detection.

Constraining first-order pupil locations of rotationally symmetric imaging systems

Constraining first-order pupil locations of rotationally symmetric imaging systems

Variation in gaze following across the life span: A process-level perspective.

Following eye gaze is fundamental for many social-cognitive abilities, for example, when judging what another agent can or cannot know. While the emergence of gaze following has been thoroughly studied on a group level, we know little about (a) the developmental trajectory beyond infancy and (b) the sources of individual differences. In Study 1, we examined gaze following across the lifespan (N=478 3- to 19-year-olds from Leipzig, Germany; and N=240 20- to 80-year-old international, remotely tested adults). We found a steep performance improvement during preschool years, in which children became more precise in locating the attentional focus of an agent. Precision levels then stayed comparably stable throughout adulthood with a minor decline toward old age. In Study 2, we formalized the process of gaze following in a computational cognitive model that allowed us to conceptualize individual differences in a psychologically meaningful way (N=60 3- to 5-year-olds, 50 adults). According to our model, participants estimate pupil angles with varying levels of precision based on observing the pupil location within the agent's eyes. In Study 3, we empirically tested how gaze following relates to vector following in non-social settings and perspective-taking abilities (N=102 4- to 5-year-olds). We found that gaze following is associated with both of these abilities but less so with other Theory of Mind tasks. This work illustrates how the combination of reliable measurement instruments and formal theoretical models allows us to explore the in(ter)dependence of core social-cognitive processes in greater detail. RESEARCH HIGHLIGHTS: Gaze following develops beyond infancy. The highest precision levels in localizing attentional foci are reached in young adulthood with a slight decrease towards old age. We present a computational model that describes gaze following as a process of estimating pupil angles and the corresponding gaze vectors. The model explains individual differences and recovers signature patterns in the data. To estimate the relation between gaze- and vector following, we designed a non-social vector following task. We found substantial correlations between gaze following and vector following, as well as Level 2 perspective-taking. Other Theory of Mind tasks did not correlate.

Fast and Accurate Pupil Localization in Natural Scenes

Fast and Accurate Pupil Localization in Natural Scenes

Pupil localization algorithm based on lightweight convolutional neural network

Pupil localization algorithm based on lightweight convolutional neural network

Visible light pupil localization and alignment method for fundus imaging

Visible light pupil localization and alignment method for fundus imaging

Head Pose-Aware Regression for Pupil Localization From a-Pillar Cameras

Head Pose-Aware Regression for Pupil Localization From a-Pillar Cameras

Eye detection and coarse localization of pupil for video-based eye tracking systems

A video-based eye tracking system generally captures NIR images, each of which contains one or two eyes of a subject. The subject’s point of gaze is then determined using 3D eye model and pupil centre corneal reflection technique. Eye detection and pupil localization play significant roles in video-based eye tracking systems. However, face rotation, wearing glasses, eye-shape variation and illumination variation make it difficult to detect an eye and localize a pupil accurately in the images captured by video-based eye tracking systems. In this paper, we proposed an eye detector that adopts a coarse-to-fine strategy. The eye detector consists of three classifiers: an ATLBP-THACs feature-based cascade classifier, a branch CNN and a multi-task CNN. We also proposed a method for coarse pupil localization. Coarse localization is an important step for pupil localization since it can provide initial pupil coordinates for a fine pupil localization method. Given a downscaled eye image, a shallow CNN is used to estimate the location of seven landmarks. On this basis, pupil center and radius are estimated. A method for small dim target enhancement is used to increase the contrast between pupil and background. The main goal of pupil enhancement is to make it easier to binarize an eye image. At last, component filtering is made by utilizing the estimated pupil center and radius. We collected a dataset named neepuEYE dataset that consists of 5500 NIR eye images from 109 people. The images can be used to generate augmented samples for training an eye detector since they contain eyes with different shape, orientation and pupil localization. Experimental results show that our eye detector is a fast and robust detector. Furthermore, our method for coarse pupil localization can obtain not only high detection rate but also high localization speed.

Pupil Localization Algorithm Based on Improved U-Net Network

Accurately localizing the pupil is an essential requirement of some new human–computer interaction methods. In the past, a lot of work has been done to solve the pupil localization problem based on the appearance characteristics of the eye, but these methods are often specific to the scenario. In this paper, we propose an improved U-net network to solve the pupil location problem. This network uses the attention mechanism to automatically select the contribution of coded and uncoded features in the model during the skip connection stage of the U-net network in the channel and spatial axis. It can make full use of the two features of the model in the decoding stage, which is beneficial for improving the performance of the model. By comparing the sequential channel attention module and spatial attention module, average pooling and maximum pooling operations, and different attention mechanisms, the model was finally determined and validated on two public data sets, which proves the validity of the proposed model.

Radially variant contrast measurement in virtual reality headsets using circular concentric ring patterns

AbstractImage quality on virtual reality (VR) head‐mounted displays (HMDs) is radially variant on each eyepiece from the visual center (optical axis of the HMD) towards the periphery. The virtual image is blurred at the periphery of display field of view (FOV) resulting in contrast and resolution degradation. Conventional measurement methods adopted from flat‐panel display evaluation do not consider the spatially dependent image artifacts by additional optical components such as HMD lenses and thus may not be adequate to assess the VR image quality. We develop a new method using circular concentric ring patterns to measure the radially variant Michelson contrast on multiple VR HMDs. The results show that measured image contrast reduces towards the periphery of display FOV especially at high‐spatial frequencies. In addition, image quality on VR HMDs can be affected by the 3D position of the eye (or the camera) within the eyebox. Specifically, contrast is further degraded if the interpupillary distance (IPD) setting of the HMD does not match that of the user. It also varies with the entrance pupil location of the light‐measuring device. We propose an efficient method for image quality evaluation that can facilitate the regulatory evaluation of VR headsets.

Pupil Dynamics-derived Sleep Stage Classification of a Head-fixed Mouse Using a Recurrent Neural Network.

The standard method for sleep state classification is thresholding the amplitudes of electroencephalography (EEG) and electromyography (EMG) data, followed by manual correction by an expert. Although popular, this method has some shortcomings: (1) the time-consuming manual correction by human experts is sometimes a bottleneck hindering sleep studies, (2) EEG electrodes on the skull interfere with wide-field imaging of the cortical activity of a head-fixed mouse under a microscope, (3) invasive surgery to fix the electrodes on the thin mouse skull risks brain tissue injury, and (4) metal electrodes for EEG and EMG recording are difficult to apply to some experimental apparatus such as that for functional magnetic resonance imaging. To overcome these shortcomings, we propose a pupil dynamics-based vigilance state classification method for a head-fixed mouse using a long short-term memory (LSTM) model, a variant of a recurrent neural network, for multi-class labeling of NREM, REM, and WAKE states. For supervisory hypnography, EEG and EMG recording were performed on head-fixed mice. This setup was combined with left eye pupillometry using a USB camera and a markerless tracking toolbox, DeepLabCut. Our open-source LSTM model with feature inputs of pupil diameter, pupil location, pupil velocity, and eyelid opening for 10 s at a 10 Hz sampling rate achieved vigilance state estimation with a higher classification performance (macro F1 score, 0.77; accuracy, 86%) than a feed-forward neural network. Findings from a diverse range of pupillary dynamics implied possible subdivision of the vigilance states defined by EEG and EMG. Pupil dynamics-based hypnography can expand the scope of alternatives for sleep stage scoring of head-fixed mice.

Current Trends in Human Pupil Localization: A Review

Current Trends in Human Pupil Localization: A Review

Event Camera-Based Pupil Localization: Facilitating Training With Event-Style Translation of RGB Faces

Event Camera-Based Pupil Localization: Facilitating Training With Event-Style Translation of RGB Faces

Study on Servo Control Method of Eye Movement Intention Based on Reinforcement Learning

Eye movement servo control has the advantage of non-contact and the eye movement is not affected by other limbs. By applying eye-tracking technology to rehabilitation research of people with movement disorders, the interaction between eye-tracking technology and computers will be gradually put into practice, which is equivalent to giving patients a pair of “invisible hands” that can give instructions freely. In this paper, a servo control method based on eye-movement coding technology is proposed to solve the problems of low accuracy, poor reliability, and insufficient human intention initiative in the field of eye-movement servo control of human-computer visual interaction. The human eye movement instruction is processed by coding and integrated with human eye movement intention to realize intelligent servo control of human eye intention. In order to improve the accuracy of eye movement servo control, this paper proposes an eye movement instruction coding processing method, which uses the trained AdaBoost fusion strong classifier algorithm and the landmark detector to detect the real-time algorithm in the video sequence to complete the eye location and blink determination. In this paper, an improved pupil location method based on multi-angle joint detection is proposed. The pupil location is completed by using the information of eye image points, and the digital coding model of eye command action is constructed, which provides an instruction base for eye movement servo control. In order to improve the reliability and initiative of eye-movement servo control, a reinforcement learning method combining eye-movement coded instruction and intention was proposed. By integrating the improved DDPG continuous control strategy reinforcement learning algorithm with eye movement intention instruction, the environmental status can be sensed in the human-computer interaction process and the optimal strategy of eye movement instruction selection can be learned.

Pupil-Aware Holography

Holographic displays promise to deliver unprecedented display capabilities in augmented reality applications, featuring a wide field of view, wide color gamut, spatial resolution, and depth cues all in a compact form factor. While emerging holographic display approaches have been successful in achieving large étendue and high image quality as seen by a camera, the large étendue also reveals a problem that makes existing displays impractical: the sampling of the holographic field by the eye pupil. Existing methods have not investigated this issue due to the lack of displays with large enough étendue, and, as such, they suffer from severe artifacts with varying eye pupil size and location. We show that the holographic field as sampled by the eye pupil is highly varying for existing display setups, and we propose pupil-aware holography that maximizes the perceptual image quality irrespective of the size, location, and orientation of the eye pupil in a near-eye holographic display. We validate the proposed approach both in simulations and on a prototype holographic display and show that our method eliminates severe artifacts and significantly outperforms existing approaches.

Pupil trajectory tracing from video-oculography with a new definition of pupil location

Pupil localization is the critical technology of eye tracking, which plays a significant role in many eye movement analysis applications, such as pupil trajectory tracing from video-oculography(VOG). However, when the pupil is occluded, the existing representative pupil center based localization methods are imprecise or even unable to find the pupil. Unfortunately, in actual VOG, the pupil is inevitably covered by the upper eyelid, resulting in a poor effect of these approaches in practice. In order to alleviate this drawback, we initiate a new way to determine the pupil position by locating the lower pole of pupil (LLPP), which can accurately locate the pupil with about 5% of the exposed area. As a result, the proposed method can reduce locating mistakes, increase data utilization, and maintain pupil trajectories continuity. The experimental results in real VOG of the benign paroxysmal positional vertigo (BPPV) patients demonstrate the effectiveness and superiority of the proposed LLPP.

Asymmetric sparse telescope

We designed and built a novel model of a deployed space telescope which can reliably align its segments to achieve the finest possible resolution. An asymmetric design of both the segment shapes and their pupil locations were tested in simulation and experiment. We optimised the sparse aperture for better spatial frequency coverage and for smoother images with less artifacts. The unique segment shapes allow for an easier identification and alignment, and the feedback is based only upon the focal image. The autonomous alignment and fine tuning are governed by mechanical simplicity and reliability.

ORB Features and Isophotes Curvature Information for Eye Center Accurate Localization

Pupil center recognition and location is an essential branch of ergonomics. It can be applied to emotion analysis and attention judgment. How to get the position of the pupil center from eye photos is the core of this field. Previous studies provided a helpful method, using scale-invariant feature transform (SIFT) to extract relevant features and combine them with the K-Nearest Neighbor (KNN) classifier. However, this method’s accuracy is not satisfying, and under some conditions, it will be position drift and other problems. We put forward a new idea to solve it by using Oriented FAST and Rotated BRIEF (ORB) features and Random Forest (RF) classifies. It is proved by experiment that our method improves the robustness of localization and the use of isophotes yields low computational cost, allowing for real-time processing. Meanwhile, we found that the ORB and RF are nearly as good, yielding an accuracy of 92.88% (BioID database).