Pupil Tracking System Research Articles

P‐27: ML‐Based Eye Tracking for Augmented Reality Heads‐Up Displays (AR HUDs)

3D Augmented Reality (AR) Heads‐up Displays (HUDs) have the potential of overlaying virtual objects at the correct locations with accurate motion parallax. Accurate overlays require tracking the pupils of the driver's eyes. We developed an ML‐based pupil tracking system based on a convolutional neural network (CNN) to find the precise location of the pupils.

Review on Centration, Astigmatic Axis Alignment, Pupil Size and Optical Zone in SMILE.

The advent of “flapless” small-incision lenticule extraction (SMILE), employing all-in-one technology, has resulted in a revolutionary breakthrough in refractive surgeries. SMILE has been gaining popularity due to fewer potential complications, such as postoperative dry eyes and greater biomechanical stability, etc. However, attention must be given to 1) the centration on the corneal vertex, 2) the proper alignment of the astigmatic axis, and 3) the relationship between pupil size and treatment diameter, to achieve good SMILE results. There is no pupil-tracking system to ascertain the accuracy of centration during the SMILE surgery. To improve the centration accuracy, our center uses two corneal topographers (Pentacam and Sirius) to measure and determine corneal vertex. Proper predicted optical zone diameter is not clearly defined yet in SMILE. Some scholars insist that mesopic pupil size should be taken into consideration when setting the predicted optical zone. Meanwhile, the issue of “functional optical zone” still has many unresolved issues and warrants further studies.

Design of an accurate and high-speed binocular pupil tracking system based on GPGPUs

An efficient and robust pupil tracking system is an important tool in visual optics and ophthalmology. It is also central to techniques for gaze tracking, of use in psychological and medical research, marketing, human–computer interaction, virtual reality and other areas. A typical setup for pupil tracking includes a camera linked to infrared LED illumination. In this work, we evaluate and parallelize several pupil tracking algorithms with the aim of accurately estimating the pupil position and size in both eyes simultaneously, to be applied in a high-speed binocular pupil tracking system. To achieve high processing speed, the original non-parallel algorithms have been parallelized by using CUDA and OpenMP. Modern graphics processors are designed to process images at high temporal frequencies and spatial resolution, and CUDA enables them to be used for general-purpose computing. Our implementation allows for efficient binocular pupil tracking at high speeds for high-resolution images (up to 988 fps with images of 1280times 1024 pixels) using a state-of-the-art GPU.

A Novel Method to Measure Artificial Eye Motility.

To measure objectively, reproducibly, and noninvasively artificial eye motility, a fundamental aspect in the (cosmetic) outcome of enucleation. A gaze- and pupil-tracking system, the "iView X," was implemented to measure horizontal eye motility. The system, with adjusted software, was tested with patients wearing 1 prosthetic eye after enucleation for retinoblastoma. Measurements were repeated 5 times in every patient and analyses were performed twice by 2 independent observers. Reproducibility was tested via linear mixed models. After the implementation of the method, more data were obtained, including more patients with a history of enucleation for the treatment of retinoblastoma for eye tracker measurement, and differences in motility percentages between implant types and sizes were analyzed via Mann-Whitney U tests. The intraclass correlation coefficient of the interobserver variable in the patient test group (n = 27, 6-53 years) was 0.98 and 0.96 for measurement of left gaze and right gaze, respectively. Intraobserver variation was <0.001. In the total of 58 included patients for comparative analysis, mean difference of prosthesis motility compared with the contralateral eye in abduction was 57.1% (range 3.2-91.5%); in adduction 65.8% (range 24.0-92.0%). No statistical differences were found between the different implant types and sizes. Motility measurements of the prosthetic eye in comparison to the contralateral eye using the iView X system are reproducible and reliable. This is, to the authors' knowledge, the first easy applicable, noninvasive, reproducible, and commercially available instrument to evaluate prosthesis motility. With the adjusted software program (freely available on request), a similar objective measurement can be performed worldwide. The implant size or type did not influence the outcome of the motility measurement; this finding deserves additional study.

Focus and perspective adaptive digital surgical microscope: optomechanical design and experimental implementation.

This paper relates to the improvement of conventional surgical stereo microscopy via the application of digital recording devices and adaptive optics. The research is aimed at improving the working conditions of the surgeon during the operation, such that free head movement is possible. The depth clues known from conventional stereo microscopy in interaction with the human eye’s functionality, such as convergence, disparity, angular elevation, parallax, and accommodation, are implemented in a digital recording system via adaptive optomechanical components. Two laterally moving pupil apertures have been used mimicking the digital implementation of the eye’s vergence and head motion. The natural eye’s accommodation is mimicked via the application of a tunable lens. Additionally, another system has been built, which enables tracking the surgeon’s eye pupil through a digital displaying stereoscopic microscope to supply the necessary information for steering the recording system. The optomechanical design and experimental results for both systems, digital recording stereoscopic microscope and pupil tracking system, are shown.

Pupil Tracking for Real-Time Motion Corrected Anterior Segment Optical Coherence Tomography.

Volumetric acquisition with anterior segment optical coherence tomography (ASOCT) is necessary to obtain accurate representations of the tissue structure and to account for asymmetries of the anterior eye anatomy. Additionally, recent interest in imaging of anterior segment vasculature and aqueous humor flow resulted in application of OCT angiography techniques to generate en face and 3D micro-vasculature maps of the anterior segment. Unfortunately, ASOCT structural and vasculature imaging systems do not capture volumes instantaneously and are subject to motion artifacts due to involuntary eye motion that may hinder their accuracy and repeatability. Several groups have demonstrated real-time tracking for motion-compensated in vivo OCT retinal imaging, but these techniques are not applicable in the anterior segment. In this work, we demonstrate a simple and low-cost pupil tracking system integrated into a custom swept-source OCT system for real-time motion-compensated anterior segment volumetric imaging. Pupil oculography hardware coaxial with the swept-source OCT system enabled fast detection and tracking of the pupil centroid. The pupil tracking ASOCT system with a field of view of 15 x 15 mm achieved diffraction-limited imaging over a lateral tracking range of +/- 2.5 mm and was able to correct eye motion at up to 22 Hz. Pupil tracking ASOCT offers a novel real-time motion compensation approach that may facilitate accurate and reproducible anterior segment imaging.

Harman demonstrates pupil-based driver monitoring system

Harman International Industries has unveiled a system that measures increases in pupil dilation as an indication of a driver's mental workload. Harman's eye and pupil tracking system measures high cognitive load and mental multitasking in the driver's seat, and signals the car's other safety systems to adapt to the driver's state.

Abnormalities of fixation, saccade and pursuit in posterior cortical atrophy.

The clinico-neuroradiological syndrome posterior cortical atrophy is the cardinal 'visual dementia' and most common atypical Alzheimer's disease phenotype, offering insights into mechanisms underlying clinical heterogeneity, pathological propagation and basic visual phenomena (e.g. visual crowding). Given the extensive attention paid to patients' (higher order) perceptual function, it is surprising that there have been no systematic analyses of basic oculomotor function in this population. Here 20 patients with posterior cortical atrophy, 17 patients with typical Alzheimer's disease and 22 healthy controls completed tests of fixation, saccade (including fixation/target gap and overlap conditions) and smooth pursuit eye movements using an infrared pupil-tracking system. Participants underwent detailed neuropsychological and neurological examinations, with a proportion also undertaking brain imaging and analysis of molecular pathology. In contrast to informal clinical evaluations of oculomotor dysfunction frequency (previous studies: 38%, current clinical examination: 33%), detailed eyetracking investigations revealed eye movement abnormalities in 80% of patients with posterior cortical atrophy (compared to 17% typical Alzheimer's disease, 5% controls). The greatest differences between posterior cortical atrophy and typical Alzheimer's disease were seen in saccadic performance. Patients with posterior cortical atrophy made significantly shorter saccades especially for distant targets. They also exhibited a significant exacerbation of the normal gap/overlap effect, consistent with 'sticky fixation'. Time to reach saccadic targets was significantly associated with parietal and occipital cortical thickness measures. On fixation stability tasks, patients with typical Alzheimer's disease showed more square wave jerks whose frequency was associated with lower cerebellar grey matter volume, while patients with posterior cortical atrophy showed large saccadic intrusions whose frequency correlated significantly with generalized reductions in cortical thickness. Patients with both posterior cortical atrophy and typical Alzheimer's disease showed lower gain in smooth pursuit compared to controls. The current study establishes that eye movement abnormalities are near-ubiquitous in posterior cortical atrophy, and highlights multiple aspects of saccadic performance which distinguish posterior cortical atrophy from typical Alzheimer's disease. We suggest the posterior cortical atrophy oculomotor profile (e.g. exacerbation of the saccadic gap/overlap effect, preserved saccadic velocity) reflects weak input from degraded occipito-parietal spatial representations of stimulus location into a superior collicular spatial map for eye movement regulation. This may indicate greater impairment of identification of oculomotor targets rather than generation of oculomotor movements. The results highlight the critical role of spatial attention and object identification but also precise stimulus localization in explaining the complex real world perception deficits observed in posterior cortical atrophy and many other patients with dementia-related visual impairment.

EYE TRACKING IN POSTERIOR CORTICAL ATROPHY: A NEW INSIGHT INTO THE VISUAL SYNDROME

Visual disorientation is a common and disabling symptom in Posterior Cortical Atrophy (PCA). However few studies of point localisation skills have been conducted, and it remains unclear to what extent...

Particle Filtering in the Design of an Accurate Pupil Tracking System

Accurate pupil tracking plays an important role in many sophisticated applications such as HCI, behavioral research, driver's fatigue detection and marketing. In this study particle filter helps us to track pupil in video frames under active IR illumination. The proposed method can be categorized as follow; The eye video is captured by a HMD camera, then precise location of the pupil is determined by thresholding and applying Snake algorithm, next the contour of the detected pupil is modeled with an ellipse, finally the parameters of the ellipse such as center coordinates and major & minor axiss are tracked by particle filter. Experimental results of the proposed method which applied to 20 different subjects, demonstrate the accuracy of the proposed method. General Terms Human-Computer Interaction

Detection of eye movements from fMRI data.

Awake humans make eye movements with amplitudes and frequencies that depend on behavioral state and task. This poses two problems for functional magnetic resonance imaging (fMRI) studies that compare brain activity across tasks. First, motion of the eye in the orbit increases the variance of the MR signal in adjacent regions of the orbitofrontal cortex, hampering activation detection. Second, eye movements are associated with activity in a distributed network of brain areas, confounding comparisons of task activation. Direct measurement of eye movements in the scanner bore is possible with expensive and technically demanding equipment. A method is described that detects eye movements directly from MR data without the use of additional equipment. Changes in the MR time series from the vitreous of the eye were observed that correlated with eye movements, as measured directly with an infrared pupil tracking system. In each of 10 subjects, the variance of the MR time series from the eye vitreous was greater when the subject made eye movements than when the subject fixated centrally (average standard deviation (SD) 99.7 vs. 75.6, P = 0.001). The assessment of eye movements directly from fMRI data may be especially useful for retrospective and meta-analyses.

Development of a noninvasive diabetes screening device using the ratio of fluorescence to Rayleigh scattered light

We have developed a new lens measurement system (LMS) that simultaneously measures the intensities of fluorescence and Rayleigh components at various distances into the lens along the optical axis. The noninvasive measurement is performed through an undilated pupil, and with the assistance of a pupil tracking system that facilitates maintaining the x and y positions of the sample volume (ca. 300 mm in length, 600 mm high, and 80 mm wide) to within 6100 mm of any programmed lock position. The intensity of the Rayleigh component that is used to normalize the measured fluorescent signal serves to correct the attenuation effects due to absorption and lens light scatter (Mie–Tyndall type). This report, resulting from a SpectRx Site L clinical study using a refined instrumentation (SpecRx Scan V), presents analysis of fluorescence and Rayleigh data from the lenses of 923 controls and 239 diabetic subjects (45 Type I and 194 Type II) ranging from 23 to 75 years old. Fluorescence and Rayleigh data have been obtained via confocal mode from various locations nominally along the lens optical axis (the true trajectory of a sample volume could be shown in a figure) for controls and diabetics, at different ages, using three pairs of excitation and collection wavelengths: 364/495 nm, 434/495 nm, and 485/515 nm. For control subjects, there exists a strong, almost linear relationship between age and fluorescence, while diabetic subjects tend to deviate from this age fluorescence relationship. Our data show that the lenses of diabetic patients are subject to an accelerated aging process, presumably due to an elevated level of brown and fluorescent protein adducts and crosslinks from nonenzymatic glycosylation (Maillard-Amadori reactions). We have also shown that by using the measured Rayleigh profiles to normalize the measured fluorescence, most of the absorption effects are removed and therefore the separation between the fluorescence of diabetics and controls is greatly improved.