Driver's Gaze Research Articles

Did you look at the moose? Driver gaze behaviour while searching for hazards in dynamic road scenes

Did you look at the moose? Driver gaze behaviour while searching for hazards in dynamic road scenes

A gaze-based driver distraction countermeasure: Comparing effects of multimodal alerts on driver's behavior and visual attention

This study, introduces and evaluates different countermeasures using real-time eye-tracking data. The countermeasures detect when driver gaze deviates from the road for longer than a predetermined threshold and then redirect the driver's attention back to the road. The countermeasures include bimodal and trimodal alerts using combinations of auditory, tactile, and visual modalities. These countermeasures showcase the utility of adopting eye-tracking technologies in the context of driver monitoring and advanced driver's assistance systems. They enhance safety as a safeguard for the increased use of devices such as in-vehicle infotainment systems. Results show that countermeasures effectively redirect drivers’ attention to the road, with higher on-road gaze time. Additionally, bimodal alerts that include the visual modality are less effective at redirecting participants’ gaze on-road and result in poorer driving performance.

Comprehensive Assessment of Artificial Intelligence Tools for Driver Monitoring and Analyzing Safety Critical Events in Vehicles.

Human factors are a primary cause of vehicle accidents. Driver monitoring systems, utilizing a range of sensors and techniques, offer an effective method to monitor and alert drivers to minimize driver error and reduce risky driving behaviors, thus helping to avoid Safety Critical Events (SCEs) and enhance overall driving safety. Artificial Intelligence (AI) tools, in particular, have been widely investigated to improve the efficiency and accuracy of driver monitoring or analysis of SCEs. To better understand the state-of-the-art practices and potential directions for AI tools in this domain, this work is an inaugural attempt to consolidate AI-related tools from academic and industry perspectives. We include an extensive review of AI models and sensors used in driver gaze analysis, driver state monitoring, and analyzing SCEs. Furthermore, researchers identified essential AI tools, both in academia and industry, utilized for camera-based driver monitoring and SCE analysis, in the market. Recommendations for future research directions are presented based on the identified tools and the discrepancies between academia and industry in previous studies. This effort provides a valuable resource for researchers and practitioners seeking a deeper understanding of leveraging AI tools to minimize driver errors, avoid SCEs, and increase driving safety.

Detection of distracted driving through the analysis of real-time driver, vehicle, and roadway volatilities

Distracted driving adversely impacts drivers’ decision-making and leads to safety-critical events (SCEs). Early detection of driver distraction is critical to prevent traffic crashes by providing warning messages to drivers and the surrounding vehicles. This study harnesses real-time multidimensional data collected through sensors that examine the variations in driver biometrics, vehicle kinematics, and roadway surroundings in different driving scenarios conducted on a Multimodal Virtual Reality Simulator. The driving behaviors of the study participants were examined under various visual detection response tasks of increasing complexity. The study classifies driving behaviors on a 5-level ordinal scale by estimating a Panel Ordered Logit Model, Random Forest, and Artificial Neural Network, using real-time volatilities in driver biometric signals, vehicle speed and acceleration, and roadway surroundings. The study results reveal that the driver gaze and the coefficients of variation in vehicle speed, driver eye movements, vehicular distances from the lane centerline, and the following vehicle significantly impact distracted driving. The study’s findings align with the principles of the safe systems approach by emphasizing the development of proactive safety measures in the form of feedback and warning the driver and surrounding vehicles of a potential distracted driving event, helping to foster safer user behavior and vehicles.

In-vehicle displays for driving automation: a scoping review of display design and evaluation using driver gaze measures

ABSTRACT Recent research has extensively examined in-vehicle display designs for supporting the operation of driving automation. As automation relieves drivers from various driving tasks including vehicle control (e.g. steering, accelerating, and braking), driving performance measures (e.g. speed, lane deviations) may not be informative indicators for evaluating the effectiveness of in-vehicle displays. Gaze-based measures are a better alternative given their link to driver visual attention, an indication of driver engagement. A scoping review was conducted to review the literature on the design of displays for supporting the operation of driving automation and the evaluation of these displays using gaze-based measures. Forty-three articles were included in the review. Most of the studies investigated visual (and mixed visual-auditory) displays that provide alerts to the driver for when to intervene automation classified as Level 3. The adopted gaze measures mostly relied on static areas of interest (AOIs), with fewer studies looking at more fine-grained, context dependent AOIs. The paper summarises the findings of the review, including research trends and gaps, as well as recommendations for future research.

Domain adaptation for driver's gaze mapping for different drivers and new environments

Distracted driving is a leading cause of traffic accidents, and often arises from a lack of visual attention on the road. To enhance road safety, monitoring a driver's visual attention is crucial. Appearance-based gaze estimation using deep learning and Convolutional Neural Networks (CNN) has shown promising results, but it faces challenges when applied to different drivers and environments. In this paper, we propose a domain adaptation-based solution for gaze mapping, which aims to accurately estimate a driver's gaze in diverse drivers and new environments. Our method consists of three steps: pre-processing, facial feature extraction, and gaze region classification. We explore two strategies for input feature extraction, one utilizing the full appearance of the driver and environment and the other focusing on the driver's face. Through unsupervised domain adaptation, we align the feature distributions of the source and target domains using a conditional Generative Adversarial Network (GAN). We conduct experiments on the Driver Gaze Mapping (DGM) dataset and the Columbia Cave-DB dataset to evaluate the performance of our method. The results demonstrate that our proposed method reduces the gaze mapping error, achieves better performance on different drivers and camera positions, and outperforms existing methods. We achieved an average Strictly Correct Estimation Rate (SCER) accuracy of 81.38% and 93.53% and Loosely Correct Estimation Rate (LCER) accuracy of 96.69% and 98.9% for the two strategies, respectively, indicating the effectiveness of our approach in adapting to different domains and camera positions. Our study contributes to the advancement of gaze mapping techniques and provides insights for improving driver safety in various driving scenarios.

An advanced multimodal driver-assistance prototype for emergency-vehicle detection

In the automotive industry, intelligent monitoring systems for advanced human-vehicle interaction aimed at enhancing the safety of drivers and passengers represent a rapidly growing area of research. Safe driving behavior relies on the driver’s awareness of the road context, enabling them to make appropriate decisions and act consistently in anomalous circumstances. A potentially dangerous situation can arise when an emergency vehicle rapidly approaches with sirens blaring. In such cases, it is crucial for the driver to perform the correct maneuvers to prioritize the emergency vehicle. For this purpose, an Advanced Driver Assistance System (ADAS) can provide timely alerts to the driver about an approaching emergency vehicle. In this work, we present a driver-assistance prototype that leverages multimodal information from an integrated audio and video monitoring system. In the initial stage, sound analysis technologies based on computational audio processing are employed to recognize the proximity of an emergency vehicle based on the sound of its siren. When such an event occurs, an in-vehicle monitoring system is activated, analyzing the driver’s facial patterns using deep-learning-based algorithms to assess their awareness. This work illustrates the design of such a prototype, presenting the hardware technologies, the software architecture, and the deep-learning algorithms for audio and video data analysis that make the driver-assistance prototype operational in a commercial car. At this initial experimental stage, the algorithms for analyzing the audio and video data have yielded promising results. The area under the precision-recall curve for siren identification stands at 0.92, while the accuracy in evaluating driver gaze orientation reaches 0.97. In conclusion, engaging in research within this field has the potential to significantly improve road safety by increasing driver awareness and facilitating timely and well-informed reactions to crucial situations. This could substantially reduce risks and ultimately protect lives on the road.

A review of driver gaze estimation and application in gaze behavior understanding

Driver gaze plays a key role in different gaze-based applications, such as driver attentiveness detection, visual distraction detection, gaze behavior understanding, and building driver assistance system. The main objective of this study is to perform a comprehensive summary of driver gaze fundamentals, methods to estimate driver gaze using machine learning (ML) based technique, and its applications in real world driving scenarios. We first discuss the fundamentals related to driver gaze, involving head-mounted and remote setup based gaze estimation and the terminologies used for driver gaze behavior understanding. Next, we list out the existing benchmark driver gaze datasets, highlighting the collection methodology, and the equipment used for such data collection. This is followed by a discussion of the algorithms used for driver gaze estimation, which primarily involves traditional machine learning and deep learning (DL) based techniques. The estimated driver gaze is then used for understanding gaze behavior while maneuvering through intersections, on-ramps, off-ramps, lane changing, determining the effect of roadside advertising structures and also for developing driver gaze based applications such as maneuver prediction, driver inattention, and distraction detection systems, etc. Finally, we discuss the limitations in the existing literature, challenges, and future scope in driver gaze estimation and gaze-based applications.

Lane-change intention recognition considering oncoming traffic: Novel insights revealed by advances in deep learning

Lane-changing (LC) intention recognition models have seen limited real-world application due to a lack of research on two-lane two-way road environments. This study constructs a high-fidelity simulated two-lane two-way road to develop a Transformer model that accurately recognizes LC intention. We propose a novel LC labelling algorithm combining vehicle dynamics and eye-tracking (VEL) and compare it against traditional time window labelling (TWL). We find the LC recognition accuracy can be further improved when oncoming vehicle features are included in the LC dataset. The Transformer demonstrates state-of-the-art performance recognizing LC 4.59 s in advance with 92.6 % accuracy using the VEL labelling method compared to GRU, LSTM and CNN + LSTM models. To interpret the Transformer's 'black box', we apply LIME model which reveals the model focuses on eye-tracking features and LC vehicle interactions with preceding and oncoming traffic during LC events. This research demonstrates that modelling additional road users and driver gaze in LC intention recognition achieves significant improvements in model performance and time-to-collision warning capabilities on two-lane two-way roads.

Model-Based 3D Gaze Estimation Using a TOF Camera.

Among the numerous gaze-estimation methods currently available, appearance-based methods predominantly use RGB images as input and employ convolutional neural networks (CNNs) to detect facial images to regressively obtain gaze angles or gaze points. Model-based methods require high-resolution images to obtain a clear eyeball geometric model. These methods face significant challenges in outdoor environments and practical application scenarios. This paper proposes a model-based gaze-estimation algorithm using a low-resolution 3D TOF camera. This study uses infrared images instead of RGB images as input to overcome the impact of varying illumination intensity in the environment on gaze estimation. We utilized a trained YOLOv8 neural network model to detect eye landmarks in captured facial images. Combined with the depth map from a time-of-flight (TOF) camera, we calculated the 3D coordinates of the canthus points of a single eye of the subject. Based on this, we fitted a 3D geometric model of the eyeball to determine the subject's gaze angle. Experimental validation showed that our method achieved a root mean square error of 6.03° and 4.83° in the horizontal and vertical directions, respectively, for the detection of the subject's gaze angle. We also tested the proposed method in a real car driving environment, achieving stable driver gaze detection at various locations inside the car, such as the dashboard, driver mirror, and the in-vehicle screen.

Retracted: Study on Driver Gaze Characteristics in Sight Distance Limited Section of Mountain Highway Based on Visual Information

Retracted: Study on Driver Gaze Characteristics in Sight Distance Limited Section of Mountain Highway Based on Visual Information

An Extensive Analysis of Different Approaches to Driver Gaze Classification

An Extensive Analysis of Different Approaches to Driver Gaze Classification

Driver Gaze Zone Estimation based on 3-Channel Convolution-optimized Vision Transformer with Transfer Learning

Driver Gaze Zone Estimation based on 3-Channel Convolution-optimized Vision Transformer with Transfer Learning

Impact of Road Central Greening Configuration on Driver Eye Movements: A Study Based on Real Vehicle Experiments

Safe driving depends on drivers’ ability to rapidly and accurately process information in varying traffic conditions. The presence of central green landscapes on roads is a key factor in this context. However, there is a gap in current research, which tends to focus on qualitative aspects of landscape design while lacking quantitative data-driven analyses. In this study, we aim to address this gap by investigating the impact of road central greening configuration on the eye movements of young novice drivers, a population particularly sensitive to external environmental changes. Specifically, we explore the influence of central green landscapes on four visual parameters: driver gaze, saccade, blinking, and pupil behavior. Through real vehicle experiments conducted on different road sections, we collected visual feature data to comprehensively analyze the patterns of driver eye movements. Our findings reveal that the introduction of central green landscapes can exert cognitive pressure on drivers, leading to specific alterations in their visual parameters. These changes include dispersed gaze points, reduced effective gaze durations, increased gaze frequencies, extended saccade durations and angles, heightened blink durations and frequencies, and reduced pupil area. By shedding light on the intricate interplay between road central greenery and driver behavior, this study provides valuable insights for optimizing landscape design in transportation planning and enhancing road safety considerations.

Real Time Driver Gaze Tracking and Eyes off the Road Detection System

Abstract: Driver Gaze Tracking and Eyes Off the Road Detection System is a technology-driven solution designed to enhance road safety by monitoring driver behavior and detecting instances when the driver's attention is diverted from the road. The system utilizes computer vision techniques and machine learning algorithms to track the driver's eye movements and identify potential distractions. By providing real-time alerts and warnings, this system aims to mitigate the risk of accidents caused by distracted driving. Distracted driving poses a significant risk to road safety, and traditional methods have limitations in ensuring immediate compliance. By leveraging computer vision and machine learning, the system provides a proactive approach to detect distractions and alert drivers, ultimately reducing the occurrence of accidents caused by diverted attention. In conclusion, the Driver Gaze Tracking and Eyes Off the Road Detection System offers a technological solution to enhance road safety by monitoring driver behavior and detecting distractions. By promptly alerting drivers when their attention is diverted, this system contributes to mitigating the risks associated with distracted driving and promoting safer road environments.

Exploratory Development of Algorithms for Determining Driver Attention Status.

Varying driver distraction algorithms were developed using vehicle kinematics and driver gaze data obtained from a camera-based driver monitoring system (DMS). Distracted driving characteristics can be difficult to accurately detect due to wide variation in driver behavior across driving environments. The growing availability of information about drivers and their involvement in the driving task increases the opportunity for accurately recognizing attention state. A baseline for driver distraction levels was developed using a video feed of 24 separate drivers in varying naturalistic driving conditions. This initial assessment was used to develop four buffer-based algorithms that aimed to determine a driver's real-time attentiveness, via a variety of metrics and combinations thereof. Of those tested, the optimal algorithm included ungrouped glance locations and speed. Notably, as an algorithm's performance of detecting very distracted drivers improved, its accuracy for correctly identifying attentive drivers decreased. At a minimum, drivers' gaze position and vehicle speed should be included when designing driver distraction algorithms to delineate between glance patterns observed at high and low speeds. Distraction algorithms should be designed with an understanding of their limitations, including instances in which they may fail to detect distracted drivers, or falsely notify attentive drivers. This research adds to the body of knowledge related to driver distraction and contributes to available methods to potentially address and reduce occurrences. Machine learning algorithms can build on the data elements discussed to increase distraction detection accuracy using robust artificial intelligence.

A Novel Way of Optimizing Headlight Distributions Based on Real Life Traffic and Eye Tracking Data Part 1: Idealized Baseline Distribution

In order to find optimized headlight distributions based on real traffic data, a three-step approach is chosen. Since the complete investigations are too extensive to fit into a single publication, this paper is the first in a series of three publications. Over three papers, a novel way to optimize automotive headlight distributions based on real-life traffic and eye-tracking data is presented, based on 119 test subjects who participated in over 15,000 km of driving, including recordings of gaze behavior, light data, detection distances, and other objects in traffic. In the present paper, a baseline headlight distribution is derived from a series of detection tests conducted under ideal conditions, with a total of three tests, each with 19–30 subjects, conducted within the same test environment. In the first test, the influence of low beam intensity on the detection of pedestrians on the sidewalk (5.0 m from the center of the driving lane) is investigated. In the second test, the influence of different high beam intensities was investigated for the same detection task. In the third test, the headlight distribution and intensity are kept constant at a representative high beam level, but the detection task is changed. In this test, the pedestrian detection target is placed along different detection angles, ranging from immediately adjacent to the road (2.5°) to 15.5 m away from the center of the driving lane (8.0°). As mentioned, all of these tests were conducted under ideal conditions, with the studies taking place on an airfield with a 1.2 km long straight road and normal road markings, but without oncoming traffic, tasks other than keeping the vehicle with cruise control within its lane, or other distracting objects present. The tests yielded two sets of data; the first is the intensity, based on the first two studies, needed to ensure sufficient intensity to detect objects under ideal conditions at distances needed for different driving speeds. The last test then uses these intensities and necessary variations in the required intensity to create an idealized, symmetric headlight distribution as a baseline for subsequent publications. Although the distribution applies only to passenger vehicles like the one used in the test, the same approach could be applied to other vehicle types. The second paper of this series will focus on real traffic objects and their distributions within the traffic space in order to identify relevant areas in headlight distribution when driving under real traffic conditions. The third paper of this series will analyze driver gaze distributions during real driving scenarios. The data from all three papers are used to create optimized headlight distributions, thereby showing how such an optimized distribution relates to current headlight distributions in terms of luminous flux, intensity, and overall distribution.

A Novel Way of Optimizing Headlight Distributions Based on Real-Life Traffic and Eye Tracking Data Part 3: Driver Gaze Behaviour on Real Roads and Optimized Light Distribution

In order to find optimized headlight distributions based on real traffic data, a three-step approach has been chosen. The complete investigations are too extensive to fit into a single paper; this paper is the last of a three part series. Over the three papers, a novel way to optimize automotive headlight distributions based on real-life traffic and eye tracking data is presented. Across all three papers, a total of 119 test subjects participated in the studies with over 15,000 km of driving, including recordings of gaze behaviour, light data, detection distances, and other objects in traffic. In this third paper, driver gaze behaviour is recorded while driving a 128 km round course, covering urban roads, country roads, and motorways. This gaze behaviour is then analysed and compared to prior work covering driver gaze behaviour. Comparing the gaze distributions with roadway object distributions from part two of this series, Analysis of Real-World Traffic Data in Germany and combining them with the idealized baseline headlight distribution from part one, different optimized headlight distributions can be generated. These headlight distributions can be optimized for different driving requirements based on the data that is used and weighting the different road types differently. The resulting headlight distribution is then compared to a standard light distribution in terms of the required luminous flux, angular distribution, and overall shape. Nonetheless, it is the overall approach that has been taken that we see as the primary novel outcome of this investigation, even more than the actual distribution resulting from this effort.

Analysis of Visual Load on Curve and Longitudinal Slope Combination of Extra-Long Underwater Tunnel Based on Visual Sensitive Region

Extra-long underwater tunnels are important traffic nodes in urban road networks and have a high incidence of traffic accidents. The frequent occurrence of curve and longitudinal slope combinations in such tunnels increases crash risk and has a negative impact on traffic safety. Vision is the most important way to obtain traffic information, and visual load is a good measure of driving risk. To explore the impact of curve and longitudinal slope combinations on a driver’s visual load in an extra-long underwater tunnel, eye-movement data and speed data were obtained through a vehicle experiment. Based on driver gaze distribution, the area and location of the visual sensitive region are calculated, and a model of the relationship between the region’s stability and the curve and longitudinal slope at different speeds is established, and applied to the visual-load evaluation. The results show that the curve or longitudinal slope has a significant influence on vehicle speed. High-visual-load areas are mainly located in small-radius curve and large longitudinal slope combination areas and extremely small-radius curve areas. For example, for a combination of left-turn curve and longitudinal slope with a speed of 50–60 km/h, when the curve radius is less than 1150 m and the longitudinal slope is less than −2.4%, or the curve radius is less than 1100 m and the longitudinal slope is greater than 2.2%, the visual load is high and driving safety is poor. This study can provide a theoretical basis for tunnel alignment design and safe operation.

Fusion of Gaze and Scene Information for Driving Behaviour Recognition: A Graph-Neural-Network- Based Framework

Accurate recognition of driver behaviours is the basis for a reliable driver assistance system. This paper proposes a novel fusion framework for driver behaviour recognition that utilises the traffic scene and driver gaze information. The proposed framework is based on the graph neural network (GNN) and contains three modules, namely, the gaze analysing (GA) module, scene understanding (SU) module and the information fusion (IF) module. The GA module is used to obtain gaze images of drivers, and extract the gaze features from the images. The SU module provides trajectory predictions for surrounding vehicles, motorcycles, bicycles and other traffic participants. The GA and SU modules are parallel and the outputs of both modules are sent to the IF module that fuses the gaze and scene information using the attention mechanism and recognises the driving behaviours through a combined classifier. The proposed framework is verified on a naturalistic driving dataset. The comparative experiments with the state-of-the-art methods demonstrate that the proposed framework has superior performance for driving behaviour recognition in various situations.