Point Of Regard Research Articles

The effects of post‐saccadic oscillations on visual processing time

Aims/Purpose: Saccadic eye movements enable us to search for a target of interest in a crowded scene, or in the case of goal‐directed saccades, to bring the peripheral target to the fovea. Visual information is processed quickly at the end of each saccade, yet precise measurements of the times involved remain controversial. This research investigates the time needed to process visual information at the end of each saccade.Methods: A new psychophysical test makes it possible to measure eye movement response latencies and visual processing times. When the test is used in conjunction with an eye tracker, each component that makes up the ‘integrated saccade latency’ (ISL) time, from the onset of the peripheral stimulus to the correct interpretation of the information carried by the stimulus, can be measured and the discrete components delineated. We measured ISL times in six subjects in response to Landolt ring optotypes of varying contrast presented peripherally at an eccentricity of 8° and also at the point of regard (0°).Results: We show that the time required to process and encode the stimulus attribute of interest at the end of a saccade is longer than that needed to carry out the same task in the absence of an eye movement. Depending on the stimulus contrast, these time durations ranged between 101–120 ms (15%) and 34–48 ms (75%) when eye movements were involved and 57–35 (15%) and less than 25 ms (75%) in the absence of eye movements.Conclusions: The extended visual processing times measured when saccades are involved may be the result of transient loss of spatial resolution caused by eyeball instability during post‐saccadic oscillations. These results explain why stimuli of high luminance contrast yield best performance in visual tasks when eye movements are involved.Acknowledgements: We acknowledge the UK Civil Aviation Authority for their support.

The use of colour and rapid flicker sensitivity tests for early detection and monitoring of vision changes in diabetes

Aims/Purpose: There is increasing evidence of impaired visual function prior to the presence of diabetic retinopathy (DR), yet current diagnostic criteria and treatment onset rely mostly on retinal vascular changes. In this study we measured changes in chromatic and rapid flicker sensitivity in subjects with prediabetes and diabetes to establish the value of such tests as a potential biomarker in the detecting early diabetes.Methods: Three groups of subjects were recruited: 1) normal controls, 2) subjects with prediabetes (HbA1c between 42‐47 mmol/mol), and 3) diabetics (with or without DR). Monocular cone‐ and rod‐enhanced flicker modulation thresholds (FMTs) were measured at the point of regard and in four parafoveal (5°) locations using the Flicker‐Plus test. Monocular red/green (RG) and yellow/blue (YB) colour thresholds were also measured at the fovea using the Colour Assessment and Diagnosis (CAD) test.Results: Both the RG and YB thresholds, as well as rod‐ and cone‐mediated flicker thresholds were significantly higher in both diabetic and prediabetic groups (when compared with healthy controls). When examined individually, ~50% and ~ 63% of the prediabetics and ~ 58% and ~ 70% of the diabetics revealed abnormal RG and YB thresholds, respectively (when compared with the upper normal limits for the corresponding age). HBA1c levels also revealed a positive trend with increasing RG and YB thresholds in the prediabetic group.Conclusions: In agreement with previous findings, both colour and FMTs are affected in patients diagnosed with diabetes. Unexpectedly, those deemed prediabetic that do not meet the clinical criteria for diabetes exhibit considerable loss of both colour and flicker sensitivity. The results suggest that colour and rapid flicker tests can provide useful information when screening for diabetes.

A Dual-Camera Eye-Tracking Platform for Rapid Real-Time Diagnosis of Acute Delirium: A Pilot Study.

Delirium, an acute confusional state, affects 20-80% of patients in Intensive Care Units (ICUs), one in three medically hospitalized patients, and up to 50% of all patients who have had surgery. Its development is associated with short- and long-term morbidity, and increased risk of death. Yet, we lack any rapid, objective, and automated method to diagnose delirium. Here, we detail the prospective deployment of a novel dual-camera contextual eye-tracking platform. We then use the data from this platform to contemporaneously classify delirium. We recruited 42 patients, resulting in 210 (114 with delirium, 96 without) recordings of hospitalized patients in ICU across two centers, as part of a prospective multi-center feasibility pilot study. All recordings made with our platform were usable for analysis. We divided the collected data into training and validation cohorts based on the data originating center. We trained two Temporal Convolutional Network (TCN) models that can classify delirium using a pre-existing manual scoring system (Confusion Assessment Method in ICU (CAM-ICU)) as the training target. The first model uses eye movements only which achieves an Area Under the Receiver Operator Curve (AUROC) of 0.67 and a mean Average Precision (mAP) of 0.68. The second model uses the point of regard, the part of the scene the patient is looking at, and increases the AUROC to 0.76 and the mAP to 0.81. These models are the first to classify delirium using continuous non-invasive eye-tracking but will require further clinical prospective validation prior to use as a decision-support tool. Eye-tracking is a biological signal that can be used to identify delirium in patients in ICU. The platform, alongside the trained neural networks, can automatically, objectively, and continuously classify delirium aiding in the early detection of the deteriorating patient. Future work is aimed at prospective evaluation and clinical translation.

Effect of monocular sensitivity on binocular summation of luminance-modulated flicker.

Binocular summation of luminance contrast signals in the spatial domain has been investigated in many studies, but less attention has been paid to the analogous interactions in the temporal domain. The present study determined the impact of monocular sensitivity on the binocular detection of luminance-modulated flickering stimuli. Binocular summation ratios (BSRs) were determined in 13 visually-normal adults for a range of monocular flicker modulation thresholds (FMTs), generated by changing stimulus size (7'- 60') and luminance (mesopic and photopic). Monocular and binocular FMTs were measured at the point of regard and in each of the four quadrants at 5° eccentricity for each target size and luminance using the Flicker-Plus test. Monocular and binocular FMT's increased with decreasing target size for all retinal locations (p<0.001), and were overall larger for mesopic than for photopic condition (p<0.001). BSRs for mesopic (mean±SD: 1.50±0.21) and photopic (1.60±0.24) stimuli were greater than unity (p<0.001), with the latter showing larger estimates than former (p<0.001). BSRs showed no significant trend across target sizes for both luminance conditions (p>0.12). The results demonstrate that the visual system successfully summates inputs from the two eyes to enhance flicker detection, independent of their absolute monocular detection thresholds. These findings may serve as a predictive baseline for further experiments designed to determine how other stimulus properties and interocular differences in monocular thresholds may affect the binocular perception of flicker.

An Improved Cross-Ratio Based Gaze Estimation Method Using Weighted Average and Polynomial Compensation

The cross-ratio (CR)-based method exploits the invariance property of CRs in projective transformation to determine a screen point corresponding to the pupil center. However, this point is essentially the intersection of the eyeball optical axis (OA) and the screen, rather than the actual point-of-regard (POR). In addition, the premise of CR calculation is that the corneal reflection points of four on-screen light sources are coplanar with the 3D pupil center, but they are only assumed to be coplanar. To solve these issues, this paper proposes an improved CR-based gaze estimation method using weighted average and polynomial compensation. Under the configuration of a single camera and two light sources, the 3D corneal center and the normal vector of virtual pupil plane are first estimated using the eyeball imaging model, and then four reference planes parallel to the virtual pupil plane are determined based on the geometric model of pupil and screen corner points. The screen point corresponding to the intersection of each reference plane and the line connecting the camera optical center and the imaging pupil center is calculated using the conventional CR-based method. Thus, the point where the OA intersects the screen is determined by the weighted average of these four points. Finally, a polynomial is learned to compensate it to the POR. The experimental results show that the gaze accuracy can reach 1.33° when the more accurate eye is selected, and it would be improved by 24% by calculating the joint POR, which is competitive with the state-of-the-art methods using more complex systems. On the basis of simplifying the system configuration of CR-based methods, the proposed method avoids the non-coplanarity of 3D pupil center and corneal reflection plane, and improves the gaze estimation performance.

Estimating 3D spatiotemporal point of regard: a device evaluation.

This paper presents and evaluates a system and method that record spatiotemporal scene information and location of the center of visual attention, i.e., spatiotemporal point of regard (PoR) in ecological environments. A primary research application of the proposed system and method is for enhancing current 2D visual attention models. Current eye-tracking approaches collapse a scene's depth structures to a 2D image, omitting visual cues that trigger important functions of the human visual system (e.g., accommodation and vergence). We combined head-mounted eye-tracking with a miniature time-of-flight camera to produce a system that could be used to estimate the spatiotemporal location of the PoR-the point of highest visual attention-within 3D scene layouts. Maintaining calibration accuracy is a primary challenge for gaze mapping; hence, we measured accuracy repeatedly by matching the PoR to fixated targets arranged within a range of working distances in depth. Accuracy was estimated as the deviation from estimated PoR relative to known locations of scene targets. We found that estimates of 3D PoR had an overall accuracy of approximately 2° omnidirectional mean average error (OMAE) with variation over a 1 h recording maintained within 3.6° OMAE. This method can be used to determine accommodation and vergence cues of the human visual system continuously within habitual environments, including everyday applications (e.g., use of hand-held devices).

INTERACTION AND VISUALIZATION DESIGN CONSIDERATIONS FOR GAZE-GUIDED COMMUNICATION IN COLLABORATIVE EXTENDED REALITY

Abstract. There is evidence in literature that collaborative work while using digital tools could benefit from visualizing the real time eye movements of a selected participant, or possibly, several participants. In this study, we examine alternative gaze interaction and visualization design prototypes in a digital collaboration scenario, in which assumed collaboration environment is a co-located mixed reality environment. Specifically, we implemented a virtual pointer as a baseline, and representations of gaze as a line, a cursor, and an ‘automated line’ where the line and cursor are automatically alternated based on occlusion detection. These prototypes are then evaluated in a series of usability studies with additional exploratory observations for a spatial communication scenario. In the scenario participants either describe routes to someone else or learn them from someone else for navigational planning. In this paper we describe the alternative interaction design prototypes, as well as various visualization designs for the gaze itself (continuous line and dashed line) and the point of regard (donut, dashed donut, sphere, rectangle) to guide collaboration and report our findings from several usability studies (n=6). We also interviewed our participants which allows us to make some qualitative observations on the potential function and usefulness of these visualization and interaction prototypes. Overall, the outcomes suggest that gaze visualization solutions in general are promising approaches to assist communication in collaborative XR, although, not surprisingly, how they are designed is important.

Robust 3-D Gaze Estimation via Data Optimization and Saliency Aggregation for Mobile Eye-Tracking Systems

In order to precisely predict 3-D gaze points, calibration is needed for each subject prior to first use the mobile gaze tracking system. However, traditional calibration methods normally expect the user to stare at predefined targets in the scene, which is troublesome and time-consuming. In this study, we proposed a novel method to remove the explicit user calibration and achieve robust 3-D gaze estimation in the room-scale area. Our proposed framework treats salient regions in the scene as possible 3-D locations of gaze points. To improve the efficiency of predicting 3-D gaze from visual saliency, the bag-of-word algorithm is adopted for eliminating redundant scene image data based on their similarities. After the elimination, saliency maps are generated from those scene images, and the geometrical relationship among the scene and eye cameras is obtained through aggregating 3-D salient targets with eye visual directions. Finally, we calculate the 3-D point of regard (PoR) by utilizing 3-D structures of the scene. The experimental results indicate that our method enhances the reliability of saliency maps and achieves promising performances on 3-D gaze estimation with different subjects.

Improving the robustness of gaze tracking under unconstrained illumination conditions

In human-computer interaction (HCI) applications, the performance degradation of gaze trackers in real-world environments is a critical issue. Typically, gaze trackers utilize the pupil center and corneal reflection (CR) obtained from an infrared (IR) light source to estimate the point of regard (POR). However, false CRs are often generated due to extraneous light sources such as sunlight or lamps. In this study, we propose a method of improving the robustness of gaze tracking under unconstrained illumination conditions. First, the proposed method generates a coded CR pattern by utilizing time-multiplexed IR light sources. Next, the CR candidates are detected in eye images, and their coordinates are compensated based on the head and eye movements of the user. Finally, true CRs are selected from the motion-compensated CR candidates by utilizing a novel cost function. Experimental results indicate that the gaze-tracking performance of the proposed method under various light conditions is considerably better than those of the conventional methods.

Shared Gaze Fails to Improve Team Visual Monitoring

The aim was to test the value of shared gaze as a way to improve team performance in a visual monitoring task. Teams outperform individuals in monitoring tasks, but fall short of achievable levels. Shared-gaze displays offer a potential method of improving team efficiency. Within a shared-gaze arrangement, operators collaborate on a visual task, and each team member's display includes a cursor to represent the other teammates' point of regard. Past work has suggested that shared gaze allows operators to better communicate and coordinate their attentional scanning in a visual search task. The current experiments sought to replicate and extend earlier findings of inefficient team performance in a visual monitoring task, and asked whether shared gaze would improve team efficiency. Participants performed a visual monitoring task framed as a sonar operation. Displays were matrices of luminance patches varying in intensity. The participants' task was to monitor for occasional critical signals, patches of high luminance. In Experiment 1, pairs of participants performed the task independently, or working as teams. In Experiment 2, teams of two participants performed the task with or without shared-gaze displays. In Experiment 1, teams detected more critical signals than individuals, but were statistically inefficient; detection rates were lower than predicted by a control model that assumed pairs of operators searching in isolation. In Experiment 2, shared gaze failed to increase target detection rates. Operators collaborate inefficiently in visual monitoring tasks, and shared gaze does not improve their performance.

Demonstration of a Virtual Reality Driving Simulation Platform

A virtual reality (VR) driving simulation platform has been built for use in addressing multiple research interests. This platform is a VR 3D engine (Unity © ) that provides an immersive driving experience viewed in an HTC Vive © head-mounted display (HMD). To test this platform, we designed a virtual driving scenario based on a real tunnel used by Törnros to perform onroad tests [1] . Data from the platform, including driving speed and lateral lane position, was compared the published on-road tests. The correspondence between the driving simulation and onroad tests is assessed to demonstrate the ability of our platform as a research tool. In addition, the drivers’ eye movement data, such as 3D gaze point of regard (POR), will be collected during the test with an Tobii © eye-tracker integrated in the HMD. The data set will be analyzed offline and examined for correlations with driving behaviors in future study.

视觉正常的自闭症儿童双眼注视点间距的特点及其意义

视觉正常的自闭症儿童双眼注视点间距的特点及其意义

A Novel Multi-Camera Global Calibration Method for Gaze Tracking System

In a 3-D gaze tracking system, the system calibration is used to determine the positions of the light sources and the screen in the camera coordinate system, which plays an important role in the estimation of the 3-D gaze and the point of regard (PoR). The traditional way for the system calibration is the mirror-based method, in which a mirror is used to make the light sources and the screen can be captured by the camera through specular reflection. However, the mirror-based method is imprecise and complex. In this paper, a novel system-calibration method for the 3-D gaze tracking system is proposed. On the basis of the system camera, a calibration camera is placed on the opposite of the screen to constitute a multi-camera system. The images are captured by the two cameras to estimate the light sources and the screen of the system camera coordinate frame. Simulations and experiments were carried out to illustrate the validation of the method. This method solves the problem that the light sources and the screen are not in view of the system camera. In addition, it is easy to implement while ensuring high accuracy and providing a new method for other similar calibration systems.

A Robust Extrinsic Calibration Method for Non-Contact Gaze Tracking in the 3-D Space

In general, 3-D gaze tracking methods employ both a frontal-viewing camera and an eye-capturing camera facing the opposite direction to precisely estimate the point-of-regard (POR) in the 3-D space. The extrinsic calibration of these two cameras for accurate 3-D gaze tracking is a challenging task. This paper presents a robust extrinsic calibration method for non-contact gaze tracking in the 3-D space. Even in a noisy environment, the extrinsic calibration parameters are precisely estimated by minimizing the proposed cost function consisting of both the angular and Euclidean errors. Furthermore, using the estimated parameters, the 3-D POR is exactly determined based on the two-view geometry. Compared with the conventional methods, the proposed method provides superior results in experiments considering various factors such as the noise level, head movement, and camera configuration. In real experiments, we achieved an average Euclidean error of 12.6 cm and the average angular error of 0.98° when estimating the 3-D coordinates of PORs that were 4–8 m away from the user.

Adjusting stereoscopic parameters by evaluating the point of regard in a virtual environment

Despite the growth in research and development in the area of virtual reality over the past few years, virtual worlds do not yet convey a feeling of presence that matches reality. This is particularly due to the difference in visual perception of flat images as compared to actual 3D. We studied the impact of two parameters of the stereoscopic configuration, namely, the inter-camera distance (ICD) and the presence of a depth of field blur (DOF blur). We conducted an experiment involving 18 participants in order to evaluate this impact, based on both subjective and objective criteria. We examined six configurations which differed in the presence or absence of DOF blur and the value of the ICD: fixed and equal to the anatomical interpupillary distance, fixed and chosen by the participant, or variable, depending on the depth of the viewer’s point of regard (POR). The DOF blur and the variable ICD require the use of an eye tracking system in order to be adjusted with respect to the POR. To our knowledge, no previously published research has tested a gaze-contingent variable ICD along with dynamic DOF blur in a Cave Automatic Virtual Environment. Our results show that the anatomical and variable ICD performed similarly regarding each criterion of the experiment, both being more efficient than the fixed ICD. Besides, as with earlier similar attempts, the configurations with DOF blur obtained lower subjective evaluations. Although mainly not significant, the results obtained by the variable ICD and DOF blur are likely due to a noticeable delay in the parameters update. We also designed a new methodology to objectively compare the geometry and depth rendering, based on the reproduction of the same scene in the real and virtual setups, and then on the study of resulting ocular convergence and angular deviation from a target. This leads to a new comparative criterion for the perceptual realism of immersive virtual environments based on the visual behavior similarity between real and virtual setups.

Comparing eye trackers by correlating their eye-metric data

Up to now, the potential of eye tracking in science as well as in everyday life has not been fully realized because of the high acquisition cost of trackers. Recently, manufacturers have introduced low-cost devices, preparing the way for wider use of this underutilized technology. As soon as scientists show independently of the manufacturers that low-cost devices are accurate enough for application and research, the real advent of eye trackers will have arrived. To facilitate this development, we propose a simple approach for comparing two eye trackers by adopting a method that psychologists have been practicing in diagnostics for decades: correlating constructs to show reliability and validity. In a laboratory study, we ran the newer, low-cost EyeTribe eye tracker and an established SensoMotoric Instruments eye tracker at the same time, positioning one above the other. This design allowed us to directly correlate the eye-tracking metrics of the two devices over time. The experiment was embedded in a research project on memory where 26 participants viewed pictures or words and had to make cognitive judgments afterwards. The outputs of both trackers, that is, the pupil size and point of regard, were highly correlated, as estimated in a mixed effects model. Furthermore, calibration quality explained a substantial amount of individual differences for gaze, but not pupil size. Since data quality is not compromised, we conclude that low-cost eye trackers, in many cases, may be reliable alternatives to established devices.

Impact of depth of field simulation on visual fatigue: Who are impacted? and how?

While stereoscopic content can be compelling, it is not always comfortable for users to interact with on a regular basis. This is because the stereoscopic content on displays viewed at a short distance has been associated with different symptoms such as eye-strain, visual discomfort, and even nausea. Many of these symptoms have been attributed to cue conflict, for example between vergence and accommodation. To resolve those conflicts, volumetric and other displays have been proposed to improve the user's experience. However, these displays are expensive, unduly restrict viewing position, or provide poor image quality. As a result, commercial solutions are not readily available. We hypothesized that some of the discomfort and fatigue symptoms exhibited from viewing in stereoscopic displays may result from a mismatch between stereopsis and blur, rather than between sensed accommodation and vergence. To find factors that may support or disprove this claim, we built a real-time gaze-contingent system that simulates depth of field (DOF) that is associated with accommodation at the virtual depth of the point of regard (POR). Subsequently, a series of experiments evaluated the impact of DOF on people of different age groups (younger versus older adults). The difference between short duration discomfort and fatigue due to prolonged viewing was also examined. Results indicated that age may be a determining factor for a user's experience of DOF. There was also a major difference in a user's perception of viewing comfort during short-term exposure and prolonged viewing. Primarily, people did not find that the presence of DOF enhanced short-term viewing comfort, while DOF alleviated some symptoms of visual fatigue but not all.

Gaze dependant prefetching of web content to increase speed and comfort of web browsing

Video-oculography gaze tracking permits the monitoring of a computer user׳s point of regard (PoR) on the screen by tracking with a video camera the centre of the pupil and patterns of infrared reflections on the cornea. Gaze tracking has been used as an alternative to traditional pointing mechanisms during human computer interaction. Gaze direction provides also an implicit but imprecise hint about intention. This work uses gaze as a source of information about user׳s intention. We explore how to benefit from behind the scenes gaze tracking in a web browsing scenario by employing the user׳s PoR to prefetch URLs of potential future interest. Our working hypothesis is that before clicking on a URL of interest, the user would have previously fixated at the URL and/or the surrounding area. Hence, using gaze data to prefetch URLs can speed up web navigation and increase user comfort. The results of our experiments show that browsing the web using prefetching of web content based on gaze activity is in fact faster than traditional browsing. The caveat of our approach is that the network traffic volume increases as a result of the prefetching activity occurring in the browser׳s background. We have also carried out additional experiments to find out the optimal trade-off between speed gain and network traffic increase in a prefetching scenario. Leveraging upon the positive results of our experiments, we postulate the idea of an attentive computer system that monitors users gaze and takes preventive or assertive action in terms of what the user is likely to do next based on the direction of gaze. To our knowledge, this is the first in-depth study on the concept of prefetching content of potential future interest in advance using the information gathered by monitoring a user׳s gaze.

Autocalibration-based partioning relationship and parallax relation for head-mounted eye trackers

This paper presents new methods to calibrate a head-mounted Eye Tracker (ET) automatically, as well as a new way to obtain an estimated point of regard (POR), taking account of the parallax. Calibration is performed in real time; it is easy for the user who just needs to look at one calibration pattern for a few seconds before starting. This method provides a very important couple of points which helps to use a local relationship to compute the POR instead of a global one. This approach significantly improves the precision of the points of regard when the scene camera is mounted with a short focal lens. An estimation of POR when the user looks somewhere outside the calibration distance is also proposed. This estimation is based on an ET modelling such as a stereovision system, to take account of the parallax effect. The aim of this study is to simplify the use of ET techniques for “non-initiated” people, especially here learner drivers.

A Multi-Gesture Interaction System Using a 3-D Iris Disk Model for Gaze Estimation and an Active Appearance Model for 3-D Hand Pointing

In this paper, we present a vision-based human-computer interaction system, which integrates control components using multiple gestures, including eye gaze, head pose, hand pointing, and mouth motions. To track head, eye, and mouth movements, we present a two-camera system that detects the face from a fixed, wide-angle camera, estimates a rough location for the eye region using an eye detector based on topographic features, and directs another active pan-tilt-zoom camera to focus in on this eye region. We also propose a novel eye gaze estimation approach for point-of-regard (POR) tracking on a viewing screen. To allow for greater head pose freedom, we developed a new calibration approach to find the 3-D eyeball location, eyeball radius, and fovea position. Moreover, in order to get the optical axis, we create a 3-D iris disk by mapping both the iris center and iris contour points to the eyeball sphere. We then rotate the fovea accordingly and compute the final, visual axis gaze direction. This part of the system permits natural, non-intrusive, pose-invariant POR estimation from a distance without resorting to infrared or complex hardware setups. We also propose and integrate a two-camera hand pointing estimation algorithm for hand gesture tracking in 3-D from a distance. The algorithms of gaze pointing and hand finger pointing are evaluated individually, and the feasibility of the entire system is validated through two interactive information visualization applications.