Gaze Zone Research Articles

Driver Gaze Zone Estimation Based on Three-Channel Convolution-Optimized Vision Transformer With Transfer Learning

Driver Gaze Zone Estimation Based on Three-Channel Convolution-Optimized Vision Transformer With Transfer Learning

Gaze Zone Classification for Driving Studies Using YOLOv8 Image Classification.

Gaze zone detection involves estimating where drivers look in terms of broad categories (e.g., left mirror, speedometer, rear mirror). We here specifically focus on the automatic annotation of gaze zones in the context of road safety research, where the system can be tuned to specific drivers and driving conditions, so that an easy to use but accurate system may be obtained. We show with an existing dataset of eye region crops (nine gaze zones) and two newly collected datasets (12 and 10 gaze zones) that image classification with YOLOv8, which has a simple command line interface, achieves near-perfect accuracy without any pre-processing of the images, as long as a model is trained on the driver and conditions for which annotation is required (such as whether the drivers wear glasses or sunglasses). We also present two apps to collect the training images and to train and apply the YOLOv8 models. Future research will need to explore how well the method extends to real driving conditions, which may be more variable and more difficult to annotate for ground truth labels.

Calibration-free and deep-learning-based customer gaze direction detection technology based on the YOLOv3-tiny model for smart advertising displays

ABSTRACT Because of the COVID-19 pandemic, gaze tracking for nontouch user interface designs used in advertising displays or automatic vending machines has become an emerging research topic. In this study, a cost-effective deep-learning-based customer gaze direction detection technology was developed for a smart advertising display. To achieve calibration-free interactions between customers and displays, the You-Only-Look-Once (YOLO)-v3-tiny-based deep learning model was used for determining the bounding boxes of eyes and pupils. Next, postprocessing was conducted using a voting mechanism and difference vectors between the central coordinates of the bounding boxes for effectively predicting customer gaze directions. Product images were separated into two or four gaze zones. For cross-person testing, the Recall, Precision, Accuracy, and F1-score for two gaze zones were approximately 77%, 99%, 88%, and 87%, respectively, and those for four gaze zones were approximately 72%, 91%, 91%, and 79%, respectively. Software implementations on NVIDIA graphics-processing-unit-accelerated embedded platforms exhibited a frame rate of nearly 30 frames per second. The proposed design achieved real-time gaze direction detection for a smart advertising platform.

Driver's Head Pose and Gaze Zone Estimation Based on Multi-Zone Templates Registration and Multi-Frame Point Cloud Fusion.

Head pose and eye gaze are vital clues for analysing a driver’s visual attention. Previous approaches achieve promising results from point clouds in constrained conditions. However, these approaches face challenges in the complex naturalistic driving scene. One of the challenges is that the collected point cloud data under non-uniform illumination and large head rotation is prone to partial facial occlusion. It causes bad transformation during failed template matching or incorrect feature extraction. In this paper, a novel estimation method is proposed for predicting accurate driver head pose and gaze zone using an RGB-D camera, with an effective point cloud fusion and registration strategy. In the fusion step, to reduce bad transformation, continuous multi-frame point clouds are registered and fused to generate a stable point cloud. In the registration step, to reduce reliance on template registration, multiple point clouds in the nearest neighbor gaze zone are utilized as a template point cloud. A coarse transformation computed by the normal distributions transform is used as the initial transformation, and updated with particle filter. A gaze zone estimator is trained by combining the head pose and eye image features, in which the head pose is predicted by point cloud registration, and the eye image features are extracted via multi-scale spare coding. Extensive experiments demonstrate that the proposed strategy achieves better results on head pose tracking, and also has a low error on gaze zone classification.

Dual-Cameras-Based Driver's Eye Gaze Tracking System with Non-Linear Gaze Point Refinement.

The human eye gaze plays a vital role in monitoring people’s attention, and various efforts have been made to improve in-vehicle driver gaze tracking systems. Most of them build the specific gaze estimation model by pre-annotated data training in an offline way. These systems usually tend to have poor generalization performance during the online gaze prediction, which is caused by the estimation bias between the training domain and the deployment domain, making the predicted gaze points shift from their correct location. To solve this problem, a novel driver’s eye gaze tracking method with non-linear gaze point refinement is proposed in a monitoring system using two cameras, which eliminates the estimation bias and implicitly fine-tunes the gaze points. Supported by the two-stage gaze point clustering algorithm, the non-linear gaze point refinement method can gradually extract the representative gaze points of the forward and mirror gaze zone and establish the non-linear gaze point re-mapping relationship. In addition, the Unscented Kalman filter is utilized to track the driver’s continuous status features. Experimental results show that the non-linear gaze point refinement method outperforms several previous gaze calibration and gaze mapping methods, and improves the gaze estimation accuracy even on the cross-subject evaluation. The system can be used for predicting the driver’s attention.

Driver Gaze Zone Estimation via Head Pose Fusion Assisted Supervision and Eye Region Weighted Encoding

Driver gaze zone estimation is an important task in Advanced Driver Assistance Systems (ADAS), which suffers difficulties including head pose, capture direction, glass occlusion, and real-time requirement, etc. Most previous methods combine face modalities and head pose using concat process, which may result in over-fitting due to the unbalanced dimension. Focusing on gaze zone estimation problems, we propose the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Head Pose Fusion Assisted supervision &amp; Eye Region Weighted Encoding</i> ( <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HP-ERW</b> ) structure, which fuses head pose attribute and face modalities together through spatial attention and Kronecker product mechanisms. Firstly, we introduce a pre-processing module dealing with head pose and face information, with the purpose of extracting input vectors and improving the fusion speed of the HP-ERW structure. Secondly, an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Eye Region Weighted Encoding Network</i> (ERW-Net) based on spatial attention is proposed to strengthen the networks perception ability for encoding features. Finally, we propose a dual-channel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Head Pose Fusion Network</i> ( <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HP-Net</b> ) based on the Kronecker product mechanism, with the purpose of fusing head pose and improving the estimation accuracy. Experiments show that the HP-ERW outperforms compared existing methods on several public datasets. The designed ADAS using the proposed method achieves 23.5 fps real-time application with small memory requirement of 4,884 KB.

Self-calibrated driver gaze estimation via gaze pattern learning

Driver’s eye gaze is regarded as an important clue for evaluating the driver’s awareness in the Advanced Driver Assistance Systems and the Automated Driving Systems. Existing driver gaze estimation methods have poor performance on uncalibrated drivers or camera views due to the tedious calibration process. To solve this problem, a knowledge-based solution based on domain prior of typical driver gaze patterns is proposed to realize self-calibration for real driving in naturalistic driving scenarios, which implicitly select the representative time samples of the forward-view gaze zone, the left-side mirror, the right-side mirror, and the rear-view mirror, the speedometer, and the center stack as calibration points. Supporting by the gaze pattern learning algorithm, the proposed method can extract the relevant driver status features gradually and update the estimation parameters periodically. Experimental results demonstrate that the auto-calibrated gaze estimation method can achieve automatic gaze calibration for gaze tracking during in-the-wild on-road driving, and does not require any cooperation from the driver.

Eye, Head and Torso Coordination During Gaze Shifts in Virtual Reality

Humans perform gaze shifts naturally through a combination of eye, head and body movements. Although gaze has been long studied as input modality for interaction, this has previously ignored the coordination of the eyes, head and body. This article reports a study of gaze shifts in virtual reality aimed to address the gap and inform design. We identify general eye, head and torso coordination patterns and provide an analysis of the relative movements’ contribution and temporal alignment. We quantify effects of target distance, direction and user posture, describe preferred eye-in-head motion ranges and identify a high variability in head movement tendency. Study insights lead us to propose gaze zones that reflect different levels of contribution from eye, head and body. We discuss design implications for HCI and VR, and in conclusion argue to treat gaze as multimodal input, and eye, head and body movement as synergetic in interaction design.

Continuous Driver's Gaze Zone Estimation Using RGB-D Camera.

The driver gaze zone is an indicator of a driver’s attention and plays an important role in the driver’s activity monitoring. Due to the bad initialization of point-cloud transformation, gaze zone systems using RGB-D cameras and ICP (Iterative Closet Points) algorithm do not work well under long-time head motion. In this work, a solution for a continuous driver gaze zone estimation system in real-world driving situations is proposed, combining multi-zone ICP-based head pose tracking and appearance-based gaze estimation. To initiate and update the coarse transformation of ICP, a particle filter with auxiliary sampling is employed for head state tracking, which accelerates the iterative convergence of ICP. Multiple templates for different gaze zone are applied to balance the templates revision of ICP under large head movement. For the RGB information, an appearance-based gaze estimation method with two-stage neighbor selection is utilized, which treats the gaze prediction as the combination of neighbor query (in head pose and eye image feature space) and linear regression (between eye image feature space and gaze angle space). The experimental results show that the proposed method outperforms the baseline methods on gaze estimation, and can provide a stable head pose tracking for driver behavior analysis in real-world driving scenarios.

Driver Gaze Zone Estimation Using Convolutional Neural Networks: A General Framework and Ablative Analysis

Driver gaze has been shown to be an excellent surrogate for driver attention in intelligent vehicles. With the recent surge of highly autonomous vehicles, driver gaze can be useful for determining the handoff time to a human driver. While there has been significant improvement in personalized driver gaze zone estimation systems, a generalized system which is invariant to different subjects, perspectives, and scales is still lacking. We take a step toward this generalized system using convolutional neural networks (CNNs). We finetune four popular CNN architectures for this task, and provide extensive comparisons of their outputs. We additionally experiment with different input image patches, and also examine how the image size affects performance. For training and testing the networks, we collect a large naturalistic driving dataset comprising of 11 long drives, driven by ten subjects in two different cars. Our best performing model achieves an accuracy of 95.18% during cross-subject testing, outperforming current state-of-the-art techniques for this task. Finally, we evaluate our best performing model on the publicly available Columbia gaze dataset comprising of images from 56 subjects with varying head pose and gaze directions. Without any training, our model successfully encodes the different gaze directions on this diverse dataset, demonstrating good generalization capabilities.

Driver-Gaze Zone Estimation Using Bayesian Filtering and Gaussian Processes

In this paper, we propose a Bayesian filtering approach that uses information from camera-based driver monitoring systems and filtering techniques to find the probability that the driver is looking in different zones. In particular, the focus is on a set of zones directly related either to active driving or to visual distraction, such as the road, the mirrors, the infotainment display, or control buttons. For systems that do not provide direct observations of the gaze direction or as a complement to noisy gaze data, we propose to use probabilistic functions that describe the gaze direction as a function of head pose and eye closure. It is further shown how these functions can be estimated from data with know visual focus points using Gaussian processes. Evaluation on data from two driver monitoring systems shows a significant improvement compared with the gaze zone estimates based on unprocessed data.

시선추적을 이용한 선택적 시각탐색에 대한 기초적 연구 - 백화점매장 공간 이미지를 중심으로 -

Gaze induction characteristics in space vary depending on characteristics of spatial components and display. This study analyzed dominant eye-fixation characteristics of three zones of department store space. Eye-fixation characteristics depending on spatial components and positional relationship can be defined as follows. First, [**.jpg] was used as an extension in the process of storing the image photographed during image data processing for analysis in pixels and due to compressed storage of image data, the image produced with a clear boundary was stored in neutral colors. To remove this problem, the image used in operation was re-processed in black and white and stored in the [**.bmp] format with large capability, at the same time. As the result, the effort caused by unnecessary colors in the program operation process was corrected. Second, the gaze ratio to space area can be indicated as a strength of each gaze zone and when analyzing the gaze strength of the three zones, the left store was a zone with a gaze strength of 102.8, the middle space was a zone with an extremely weak gaze strength of 89.6 and the right store was a zone with an extremely gaze strength of 117.2. Third, the IV section had a strong strength of gaze on the middle space and the right store and the V section showed a markedly strong strength of gaze on the left and right stores. This tendency was the same as the VI section with the strongest gaze strength and the right store had a little strong gaze strength than the left store.

Automatic Calibration Method for Driver's Head Orientation in Natural Driving Environment

Gaze tracking is crucial for studying driver's attention, detecting fatigue, and improving driver assistance systems, but it is difficult in natural driving environments due to nonuniform and highly variable illumination and large head movements. Traditional calibrations that require subjects to follow calibrators are very cumbersome to be implemented in daily driving situations. A new automatic calibration method, based on a single camera for determining the head orientation and which utilizes the side mirrors, the rear-view mirror, the instrument board, and different zones in the windshield as calibration points, is presented in this paper. Supported by a self-learning algorithm, the system tracks the head and categorizes the head pose in 12 gaze zones based on facial features. The particle filter is used to estimate the head pose to obtain an accurate gaze zone by updating the calibration parameters. Experimental results show that, after several hours of driving, the automatic calibration method without driver's corporation can achieve the same accuracy as a manual calibration method. The mean error of estimated eye gazes was less than 5°in day and night driving.

Real-Time Gaze Estimator Based on Driver's Head Orientation for Forward Collision Warning System

This paper presents a vision-based real-time gaze zone estimator based on a driver's head orientation composed of yaw and pitch. Generally, vision-based methods are vulnerable to the wearing of eyeglasses and image variations between day and night. The proposed method is novel in the following four ways: First, the proposed method can work under both day and night conditions and is robust to facial image variation caused by eyeglasses because it only requires simple facial features and not specific features such as eyes, lip corners, and facial contours. Second, an ellipsoidal face model is proposed instead of a cylindrical face model to exactly determine a driver's yaw. Third, we propose new features—the normalized mean and the standard deviation of the horizontal edge projection histogram—to reliably and rapidly estimate a driver's pitch. Fourth, the proposed method obtains an accurate gaze zone by using a support vector machine. Experimental results from 200 000 images showed that the root mean square errors of the estimated yaw and pitch angles are below 7 under both daylight and nighttime conditions. Equivalent results were obtained for drivers with glasses or sunglasses, and 18 gaze zones were accurately estimated using the proposed gaze estimation method.