Gaze Depth Research Articles

For higher accuracy: 3D eye movement gaze depth estimation method based on Mix-TCN

Currently, the biggest obstacle in applying eye tracking technology in Virtual Reality (VR) and Augmented Reality (AR) scenes is the particular difficulty in choosing distance and object distance in 3D scenes. In previous research, geometric calculation methods using vestibular ocular reflex (VOR) and research on binocular visual angle have been studied to some extent, but unfortunately, their effects have not reached a practical level. The paper proposes a new research idea to estimate the depth of binocular gaze using a method of time series eye movement data analysis based on depth learning, and proposes a Mix-Temporal Convolutional Network (TCN) optical time series network. By combining VOR and deep learning theory, the paper has realized the current state-of-the-art technology for estimating the depth of gaze from the movement of the gaze.

Characterizing information access needs in gaze-adaptive augmented reality interfaces: implications for fast-paced and dynamic usage contexts

ABSTRACT Gaze-adaptive interfaces can enable intuitive hands-free augmented reality (AR) interaction but unintentional selection (i.e. “Midas Touch”) can have serious consequences during high-stakes real-world AR use. In the present study, we assessed how simulated gaze-adaptive AR interfaces, implementing single and dual gaze inputs, influence Soldiers’ human performance and user experience (UX) in a fast-paced virtual reality marksmanship task. In Experiment 1, we investigated 1- and 2-stage dwell-based interfaces, finding confirmatory dual gaze dwell input effectively reduced Midas Touch but also reduced task performance and UX compared to an always-on (AO) interface. In Experiment 2, we investigated gaze depth-based interfaces, finding similar negative impacts of confirmatory dwell on Midas Touch, task performance, and UX. Overall, compared to the AO interface, single gaze input interfaces (e.g. single dwell or gaze depth threshold) reduced viewing of task-irrelevant information and yielded similar task performance and UX despite being prone to Midas Touch. Broadly, our findings demonstrate that AR users performing fast-paced dynamic tasks can tolerate some unintentional activation of AR displays if reliable and rapid information access is maintained and point to the need to develop and refine gaze depth estimation algorithms and novel gaze depth-based interfaces that provide rapid access to AR display content.

Gaze-Vergence-Controlled See-Through Vision in Augmented Reality.

Augmented Reality (AR) see-through vision is an interesting research topic since it enables users to see through a wall and see the occluded objects. Most existing research focuses on the visual effects of see-through vision, while the interaction method is less studied. However, we argue that using common interaction modalities, e.g., midair click and speech, may not be the optimal way to control see-through vision. This is because when we want to see through something, it is physically related to our gaze depth/vergence and thus should be naturally controlled by the eyes. Following this idea, this paper proposes a novel gaze-vergence-controlled (GVC) see-through vision technique in AR. Since gaze depth is needed, we build a gaze tracking module with two infrared cameras and the corresponding algorithm and assemble it into the Microsoft HoloLens 2 to achieve gaze depth estimation. We then propose two different GVC modes for see-through vision to fit different scenarios. Extensive experimental results demonstrate that our gaze depth estimation is efficient and accurate. By comparing with conventional interaction modalities, our GVC techniques are also shown to be superior in terms of efficiency and more preferred by users. Finally, we present four example applications of gaze-vergence-controlled see-through vision.

Eye-Tracking Beyond Peripersonal Space in Virtual Reality: Validation and Best Practices

Recent developments in commercial virtual reality (VR) hardware with embedded eye-tracking create tremendous opportunities for human subjects researchers. Accessible eye-tracking in VR opens new opportunities for highly controlled experimental setups in which participants can engage novel 3D digital environments. However, because VR embedded eye-tracking differs from the majority of historical eye-tracking research, in both providing for relatively unconstrained movement and stimulus presentation distances, there is a need for greater discussion around methods for implementation and validation of VR based eye-tracking tools. The aim of this paper is to provide a practical introduction to the challenges of, and methods for, 3D gaze-tracking in VR with a focus on best practices for results validation and reporting. Specifically, first, we identify and define challenges and methods for collecting and analyzing 3D eye-tracking data in VR. Then, we introduce a validation pilot study with a focus on factors related to 3D gaze tracking. The pilot study provides both a reference data point for a common commercial hardware/software platform (HTC Vive Pro Eye) and illustrates the proposed methods. One outcome of this study was the observation that accuracy and precision of collected data may depend on stimulus distance, which has consequences for studies where stimuli is presented on varying distances. We also conclude that vergence is a potentially problematic basis for estimating gaze depth in VR and should be used with caution as the field move towards a more established method for 3D eye-tracking.

3D Gaze in Virtual Reality: Vergence, Calibration, Event Detection

Eye movement analysis in modern 3D rendering systems is reviewed and three new techniques are derived inspired by work developed in early Virtual Reality so-called 2.5D implementations, namely (a) gaze depth (i.e., vergence) estimation, (b) vergence calibration, and (c) real-time 3D event detection that considers eye- and head-coupling. The new 3D calibration shows excellent error reduction in terms of Mean Squared Error (MSE).

A Novel Integrated Eye-Tracking System with Stereo Stimuli for 3D Gaze Estimation

Eye-tracking technology is increasingly used in applications such as 3D displays and human-computer interaction (HCI). However, most current eye trackers focus on 2D point-of-gaze (PoG) estimation and cannot provide accurate gaze depth. Concerning future applications such as HCI with 3D displays, we propose an integrated binocular eye-tracking device with stereo stimuli to provide highly accurate 3D PoG estimation. In our device, the 3D stereo imaging system can provide users with a friendly and immersive 3D visual experience without wearing any accessories. The eye-capturing system can directly record the users’ eye movements under 3D stimuli without disturbance, which can realize front views of eye images while ensuring the user’s wide field of view (FOV). A regression-based 3D eye-tracking model is built based on collected eye movement data under stereo stimuli. Our model estimates users’ 2D gaze with features defined by eye region landmarks and further estimates 3D gaze with a multi-source feature set constructed by comprehensive eye movement features and disparity features from stereo stimuli. Two test stereo scenes with different depths of field are designed to verify the model’s effectiveness. Experimental results of four-fold cross-validation show that the average error for 2D gaze estimation was 0.65°. For 3D gaze estimation, the average errors are 1.69 cm over the workspace volume 50 × 30 × 75 cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> and 0.14 m over the workspace volume 2.4 × 4.0 × 7.9 m <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> . Two preliminary validation experiments also demonstrate that our system has broad application prospects, such as 3D vision technologies and medical applications related to neurological diseases.

Method for Automatic Switching Screen of OST-HMD using Gaze Depth Estimation

본 논문에서는 시선 깊이 추정 기술을 이용한 OST-HMD의 자동화면 on/off 기능을 제안한다. 제안하는 방법은 MLP (Multi-layer Perceptron)을 이용하여 사용자의 시선 정보와 보는 물체의 거리를 학습 한 후, 시선 정보만 입력하여 거리를 추정한다. 학습 단계에서는 착용 할 수 있는 양안 추적기를 사용하여 시선 관련 특징을 얻는다. 그런 다음 이 특징을 다층 퍼셉 트론 (MLP: Multi-layer Perceptron)에 입력하여 학습하고 모델을 생성한다. 추론 단계에서는 안구 추적기로부터 실시간으로 시선 관련 특징을 얻고 이를 MLP에 입력하여 추정 깊이 값을 얻는다. 마지막으로 HMD의 화면을 켜거나 끌 것인지 여부를 결정하기 위해 이 계산결과를 활용한다. 제안된 방법의 가능성을 평가하기 위해 프로토타입을 구현하고 실험을 수행하였다.In this paper, we propose automatic screen on / off method of OST-HMD screen using gaze depth estimation technique. The proposed method uses MLP (Multi-layer Perceptron) to learn the user s gaze information and the corresponding distance of the object, and inputs the gaze information to estimate the distance. In the learning phase, eye-related features obtained using a wearable eye-tracker. These features are then entered into the Multi-layer Perceptron (MLP) for learning and model generation. In the inference step, eye - related features obtained from the eye tracker in real time input to the MLP to obtain the estimated depth value. Finally, we use the results of this calculation to determine whether to turn the display of the HMD on or off. A prototype was implemented and experiments were conducted to evaluate the feasibility of the proposed method.

Online 3D Gaze Localization on Stereoscopic Displays

This article summarizes our previous work on developing an online system to allow the estimation of 3D gaze depth using eye tracking in a stereoscopic environment. We report on recent extensions allowing us to report the full 3D gaze position. Our system employs a 3D calibration process that determines the parameters of a mapping from a naive depth estimate, based simply on triangulation, to a refined 3D gaze point estimate tuned to a particular user. We show that our system is an improvement on the geometry-based 3D gaze estimation returned by a proprietary algorithm provided with our tracker. We also compare our approach with that of the Parameterized Self-Organizing Map (PSOM) method, due to Essig and colleagues, which also individually calibrates to each user. We argue that our method is superior in speed and ease of calibration, is easier to implement, and does not require an iterative solver to produce a gaze position, thus guaranteeing computation at the rate of tracker acquisition. In addition, we report on a user study that indicates that, compared with PSOM, our method more accurately estimates gaze depth, and is nearly as accurate in estimating horizontal and vertical position. Results are verified on two different 4D eye tracking systems, a high accuracy Wheatstone haploscope and a medium accuracy active stereo display. Thus, it is the recommended method for applications that primarily require gaze depth information, while its ease of use makes it suitable for many applications requiring full 3D gaze position.

Proposal for a 3‐D display with accommodative compensation: 3DDAC

Abstract— This paper proposes a novel binocular‐disparity three‐dimensional (3‐D) display with accommodative compensation (3DDAC) to achieve fatigue‐free viewing. The display principle is based both on detecting gaze points and moving the display screen to the gaze depth in real time. Gaze detection is discussed with respect to whether eyeball rotation occurs or not, and screen motion is discussed with respect to whether the screen is real or a virtual image. In an experimental setup, a 3‐D computer‐graphics image was produced to evaluate the accommodation effect. The gaze position was fixed at the center of the field of view, and the display depth was controlled by a z‐buffer value of the pixel focused on. The display screen was then moved, as a virtual image, by back‐and‐forth movements of a relay lens. It was determined that the perceived sharpness of the image increased for the proposed display mode.

Seeing Her Looking at You: Acquaintance and Variation in the Judgment of Gaze Depth

The acquaintance of subject and looker as well as the depth of gaze affected male subjects' judgments of a female assistant's looking behavior. In a situation ruling out visual interaction, eye-contact gazes were located no more accurately than other gazes. Nevertheless, although gaze depths were not accurately discriminated, gazes deviating vertically and horizontally, to the edge of the head and just beyond the head, were located with some accuracy. The pattern of errors was toward the head and away from the body. Acquaintance produces a stronger bias away from the body, and may produce other interaction-facilitating biases.