Glance Patterns Research Articles

Exploratory Development of Algorithms for Determining Driver Attention Status.

Varying driver distraction algorithms were developed using vehicle kinematics and driver gaze data obtained from a camera-based driver monitoring system (DMS). Distracted driving characteristics can be difficult to accurately detect due to wide variation in driver behavior across driving environments. The growing availability of information about drivers and their involvement in the driving task increases the opportunity for accurately recognizing attention state. A baseline for driver distraction levels was developed using a video feed of 24 separate drivers in varying naturalistic driving conditions. This initial assessment was used to develop four buffer-based algorithms that aimed to determine a driver's real-time attentiveness, via a variety of metrics and combinations thereof. Of those tested, the optimal algorithm included ungrouped glance locations and speed. Notably, as an algorithm's performance of detecting very distracted drivers improved, its accuracy for correctly identifying attentive drivers decreased. At a minimum, drivers' gaze position and vehicle speed should be included when designing driver distraction algorithms to delineate between glance patterns observed at high and low speeds. Distraction algorithms should be designed with an understanding of their limitations, including instances in which they may fail to detect distracted drivers, or falsely notify attentive drivers. This research adds to the body of knowledge related to driver distraction and contributes to available methods to potentially address and reduce occurrences. Machine learning algorithms can build on the data elements discussed to increase distraction detection accuracy using robust artificial intelligence.

How do drivers allocate visual attention to vulnerable road users when turning at urban intersections?

IntroductionDrivers turning at urban intersections pose a high risk to Vulnerable Road Users (VRUs), such as cyclists and pedestrians. In vehicle collisions with VRUs, driver attention misallocation is considered a leading contributor. While previous naturalistic studies have examined driver gaze behaviors at intersections, findings are limited to general gaze directions obtained through video analysis, meaning specific areas to which drivers attend cannot be determined. MethodWe present a secondary analysis of an on-road instrumented vehicle dataset collected in 2019 which offers eye-tracking and video data from 26 experienced drivers (13 cyclists and 13 non-cyclists). Three coders jointly examined eye-tracking footage from four right-signalized turns (n = 96) to quantify drivers’ glance distributions to various areas of interest, including those most relevant to VRU safety when drivers turn. Individual temporal glance patterns and general attention allocation trends are presented and described. Results(1) Relevant pedestrians were the top objects of glance irrespective of signal status, and (2) at red light turns, driver attention was heavily skewed toward leftward traffic. ConclusionsThis analysis provides a detailed report of driver glance distributions toward scene-specific areas (as opposed to general directions) at urban intersections and discusses how these patterns may influence VRU safety. Practical applicationsThis study provides important information regarding the human factors challenges of vehicle-VRU collisions and their prevention.

A model for naturalistic glance behavior around Tesla Autopilot disengagements

We present a model for visual behavior that can simulate the glance pattern observed around driver-initiated, non-critical disengagements of Tesla's Autopilot (AP) in naturalistic highway driving. Drivers may become inattentive when using partially-automated driving systems. The safety effects associated with inattention are unknown until we have a quantitative reference on how visual behavior changes with automation. The model is based on glance data from 290 human initiated AP disengagement epochs. Glance duration and transition were modelled with Bayesian Generalized Linear Mixed models. The model replicates the observed glance pattern across drivers. The model's components show that off-road glances were longer with AP active than without and that their frequency characteristics changed. Driving-related off-road glances were less frequent with AP active than in manual driving, while non-driving related glances to the down/center-stack areas were the most frequent and the longest (22% of the glances exceeded 2 s). Little difference was found in on-road glance duration. Visual behavior patterns change before and after AP disengagement. Before disengagement, drivers looked less on road and focused more on non-driving related areas compared to after the transition to manual driving. The higher proportion of off-road glances before disengagement to manual driving were not compensated by longer glances ahead. The model can be used as a reference for safety assessment or to formulate design targets for driver management systems.

Drivers Glance Like Lizards during Cell Phone Distraction in Assisted Driving

Head pose has been proposed as a surrogate for eye movement to predict areas of interest (AOIs) where drivers allocate their attention. However, head pose may disassociate with AOIs in glance behavior involving zero or subtle head movements, commonly known as “lizard” glance pattern. In contrast, “owl” glance pattern is used to describe glance behavior along with larger head movements. It remains unclear which glance pattern is prevalent during driver cell phone distraction and what are appropriate metrics to detect such distraction. To address this gap, we analyzed the gaze direction and head pose of 36 participants who completed an email-sorting task using a cell phone while driving a Tesla on the test track in Autopilot mode. The dispersion-threshold algorithm identified driver gaze fixations and synchronized them with head movements. The results showed that when using a cell phone either near the lap or behind the steering wheel, participants exhibited a dominant lizard-type glance pattern with minimal shift in head position. As a result, head pose alone may not provide sufficient information for cell phone distraction detection, and gaze metrics should be involved in enhancing this application.

Patterns in transitions of visual attention during baseline driving and during interaction with visual–manual and voice-based interfaces

Voice interfaces reduce visual demand compared with visual-manual interfaces, but the extent depends on design. This study compared visual demand during baseline driving with driving while using voice or manual inputs to place calls with Chevrolet MyLink, Volvo Sensus, or a smartphone. Mean glance duration and total eyes-off-road-time increased when using manual input compared with baseline driving; only eyes off road time increased with voice input. Confusion matrices developed with hidden Markov modelling characterise the similarity of glance sequences during baseline driving and while making phone calls. Glance sequences with the MyLink voice interface were misclassified as baseline driving more frequently than the other voice interfaces. Conversely, glance sequences with the Sensus and smartphone voice interfaces were more often misclassified as manual phone calling. Thus, the MyLink voice interface not only reduced the overall visual demand of placing calls, but produced glance patterns more similar to driving without another task. Practitioner Summary: The attention map and confusion matrix methodologies provide ways of characterising similarities and differences in glance behaviour across secondary task conditions, complementing traditional temporally based metrics (e.g. mean glance duration, long duration glances) while addressing some of the limitations of total-eyes-off-road-time (TEORT) for comparing secondary task behaviour to baseline driving.

Individual Differences in Glance Patterns under Distraction in Level 2 Automated Driving

This paper investigated individual differences in attentional strategies during the non-driving-related tasks in Level 2 automated driving. Ward’s method was used to cluster participants into different groups according to the characteristics of their sequential off-road glances in the email-sorting task: duration, frequency, variance, and intensity. The clustering results showed two types of sequential off-road glance patterns in distracted Level 2 driving: infrequent long glances vs. frequent short glances. However, participants in the two groups showed similar workload, driving engagement, and email-sorting accuracy. They also reported similar feelings of safety and feedback about the Level 2 vehicle automation. The glance differences demonstrated the complexity of attentional strategies among individual drivers, which is a necessary aspect of driver state to be monitored in automated driving.

Manipulating music to communicate automation reliability in conditionally automated driving: A driving simulator study

Abstract With increasing vehicle automation, drivers are likely to engage in non-driving related tasks. However, before fully autonomous vehicles can be achieved, drivers remain the fallback when automation limits are reached and/or in unexpected situations. Transition from automated to manual control could be particularly difficult for drivers who are “out-of-the-loop”. To support safe and smooth control transition, we adopted the blended sonification approach in manipulating background music to convey information about the reliability level of the automation. A driving simulator study consisting of 36 participants was conducted to investigate the effectiveness of sonification feedback—three levels of system reliability was mapped to background music pitch levels—on takeover events. Participants were assigned to one of three groups that received continuous sonification feedback, intermittent sonification feedback, or no feedback about automation reliability. The proposed blended music sonification was found effective in increasing monitoring behavior and decreasing visual response times to takeover requests during automated driving. Steering control in a takeover scenario that required substantial amount of maneuvering also benefitted from having sonification feedback. There was some evidence that sonification feedback provided continuously was more effective than when it was provided intermittently, possibly related to observed differences in the associated glance patterns. In conclusion, blended sonification may be an effective approach in helping drivers stay on the loop during automated driving, although the study was limited to the choice of music examined. Our future work will investigate how best to combine automation reliability information and traffic situational information to better support situation awareness and facilitate task-switching in automated driving.

Using Eye-Tracking Technology to Compare Hazard Detection on the Road Versus a Driving Simulator at Night Across Two Age Groups

Abstract Date Presented 03/28/20 This poster will illustrate the results of a two x two (old/young and on-road/simulator) repeated-measures study on hazard detection. The purpose is to analyze driving performance at night by combining eye-tracking technology and driving simulation to analyze on-road scanning behavior, visual attention, glance patterns, and hazard detection. Validity of driving simulators in predicting on-road performance with older-adult drivers in a safe and cost-effective manner will be discussed. Primary Author and Speaker: Juliette Leonardo Additional Authors and Speakers: Anne Dickerson Contributing Authors: Mary Grace McDonald

Investigating lane change behaviors and difficulties for senior drivers using naturalistic driving data

Introduction: Lane changes can be a complicated maneuver occurring a dynamic environment requiring the integration of many streams of information. Older drivers may struggle with lane changes which may elevate crash risk. Methods: Real-world lane change behaviors were examined using the Second Strategic Highway Research Program Naturalistic Driving Study database. A total of 393 lane changes were observed for two age groups: middle-aged (30–49), and older (70+) drivers. Results: Older drivers were highly likely to fail to execute an over-the-shoulder glance prior to initiation of a lane change (in 98% of left lane changes and 92% for right lane changes). Older drivers also showed higher rates of OTS glance errors at any point during the lane change in 95% of left lane changes and 86% of right-lane changes. Additionally, older drivers frequently failed to activate the turn signal prior to lane change initiation (60% of lane changes for right changes and 59% for left lane changes). Of the older drivers that made side mirror glances, many occurred after the initiation of the maneuver (46% of left lane changes and 58% of right lane changes) suggesting glances were occurring while changing lanes. Conclusions: Results for older drivers showed that many key glances (particularly side mirror checks) and turn signal actuations observed in the current study occurred after the initiation of the lane change, ostensibly when this action may be too late to gather relevant information and avoid a conflict. Practical Applications: Knowledge of glance patterns during lane changes for older drivers can help older drivers maintain travel mobility as they age. Either through training to reinforce OTS and side mirror glances, or supplemental devices like convex mirrors or oversized rearview mirrors, older drivers can reduce high-risk lane change maneuvers and help older drivers to maintain their mobility and independence longer.

The influence of driver’s mood on car following and glance behaviour: Using cognitive load as an intervention

Driving safety relies on a driver’s ability to maintain their attentional focus and that mood is one of the factors which influences this ability. This driving simulator study used mind wandering theory to understand the changes in car following behaviour and driver glance patterns when affected by neutral, happy, sad and angry moods during car following. Two types of cognitive load were used to investigate ways of disengaging drivers from the mind wandering state. The moods were induced via music and mental imagery and assessed via self-reports and physiological measures. The results show that mood valence and arousal have different effects on driving safety, with negative moods resulting in the most dangerous driving, regardless of arousal. The cognitive load, in some cases, disengaged drivers from mood-related mind wandering. However, more detailed research is needed to understand the amount of load necessary for this disengagement in different moods. The importance of using driving-related measures together with glance patterns in mood research was highlighted to overcome ambiguities resulting from conclusions based on single measurements.

Eye glances towards conflict-relevant cues: the roles of anticipatory competence and driver experience

ObjectiveThis paper analyzes the effects of anticipatory competence and driver experience on glance patterns towards visual cues that indicate conflict situations. Background: Prior research has shown that experienced drivers’ visual scanning patterns differ from those of novices. Experienced drivers are less erratic and more systematic in their monitoring of the environment. We have also shown in an earlier study that driving experience improves anticipatory competence in that it leads to a higher number of timely proactive actions in conflict-scenarios (avoidance actions prior to, as opposed to in reaction to a conflict). This paper investigates glance patterns specifically to relevant visual cues in conflict scenarios to determine whether glance patterns of anticipatory drivers who exhibit proactive actions differ from those who do not. It also investigates whether experienced drivers pay more attention to these cues compared to novices. Method: We conducted a simulator experiment with 24 experienced and 24 novice drivers. As part of the experiment, all drivers completed three distinct traffic scenarios, each with a conflict situation. Results: The results show that drivers who exhibited proactive actions had more frequent and longer glances towards conflict-relevant cues than those who did not exhibit any. Similarly, experienced drivers focused on these visual cues more often, and for longer durations compared to novices. Further, experienced drivers who exhibited proactive actions looked at the cues more often compared to experienced drivers who did not exhibit any; there was no significant difference for novice drivers. Conclusion: These findings speak to the role of situation-specific visual cues for anticipatory competence, and to the importance of driver experience to aid in the interpretation of these cues. Future research should seek to confirm our findings in a wider variety of driving scenarios.

Engaging in NDRTs affects drivers’ responses and glance patterns after silent automation failures

The aim of this study was to understand driver responses to “silent” failures in automated driving, where automation failed during a simulator drive, without a take-over warning. The effect of a visual non-driving related task (NDRT) and a road-based vigilance task presented drivers’ take-over response and visual attention was also investigated. Currently, automated driving systems face a number of limitations that require control to be handed back to the driver. Much of the research to date has focused on explicit take-over requests (ToRs) and shows that drivers struggle to resume control safely, exacerbated by disengagement from the driving task, for instance, due to the presence of NDRTs. However, little is known about whether, and how, drivers will respond to more subtle automation failures that come without a warning, and how this is affected by NDRT engagement. Thirty participants drove a simulated automated drive in two conditions, which had 6 silent automation failures each (3 on a Curve, 3 in a Straight), with no ToRs. In one condition, drivers were required to constantly monitor the road, which was enforced by a road-based vigilance task (VMS Only). In the other, drivers performed an additional visual NDRT, requiring them to divide their attention (VMS + Arrows). Results showed that, in both conditions, all drivers eventually detected and responded to all silent automation failures. However, engaging in an additional NDRT during automation resulted in significantly more lane excursions and longer take-over times. Adding a visual NDRT not only changed the distribution of drivers’ visual attention before and after the failure but also how they divided their attention between information on the road environment and the human–machine interface, which provided information on automation status. These results provide support for how driver monitoring systems may be used to detect drivers’ visual attention to the driving task and surroundings, and used as a tool for encouraging driver intervention, when required.

Modeling microstructure of drivers’ task switching behavior

Abstract A computational model is created to simulate drivers’ task switching behavior, or dynamic allocation of visual attention, while they are driving and engaging in a secondary task. The model takes the following into account: uncertainty about the roadway, task structure, and individual differences. The first factor, uncertainty, means a lack of information about the roadway that plays a significant role in switching attention back to the roadway. The second factor, task structure, reflects the driver's tendency to switch visual attention from the secondary task to the roadway at subtask boundaries as well as the tendency to continue to perform task to reach subtask boundaries. Lastly, the model considers the variability of performance speed across the driver population. The factors jointly influence the probability of switching attention in the model. We use the ABC-MCMC (Approximate Bayesian Computation – Markov Chain Monte Carlo) method to estimate model parameters that produce the microstructure of task switching. The fitted model generates glance patterns at a micro level that are consistent with those generated by participants in an experiment.

A distraction index for quantification of driver eye glance behavior: A study using SHRP2 NEST database

Distracted driving behavior and driving inattention are two leading causes of roadway crashes. The state-of-the-art safety research made several attempts to understand and quantify distracted driving and driver inattention. While each attempt had its limitation, there was a consensus on the relevance of eye glance behavior as a promising parameter in understanding distracted driving. In this study, a renewal cycle approach is implemented to provide deeper insights into how drivers allocate their attention while driving. This approach is then applied to the Naturalistic Engagement in Secondary Tasks (NEST) dataset to analyze drivers’ eye glance patterns and determine the relationship between their visual behavior and engagement in different types of secondary tasks (activities performed while driving). The analysis revealed that distracted driving behavior could be well characterized by two new measures: the number of renewal cycles per event (NRC) and a distraction level index (DI). Consequently, mixed-effects modeling is implemented to test the effectiveness of the two measures to differentiate crash/near-crash events from non-crash events. The analysis showed that the two measures increase significantly for crash/near-crash events compared to non-crash driving events with p-values less than 0.0001. The findings of this paper are promising to the quantification of the risk associated with distraction related visual behavior. The finding can also help build reliable algorithms for in-vehicle driving assistance systems to alert drivers before crash/near-crash events.

How safe is tuning a radio?: using the radio tuning task as a benchmark for distracted driving

Drivers engage in non-driving tasks while driving, such as interactions entertainment systems. Studies have identified glance patterns related to such interactions, and manual radio tuning has been used as a reference task to set an upper bound on the acceptable demand of interactions. Consequently, some view the risk associated with radio tuning as defining the upper limit of glance measures associated with visual-manual in-vehicle activities. However, we have little knowledge about the actual degree of crash risk that radio tuning poses and, by extension, the risk of tasks that have similar glance patterns as the radio tuning task. In the current study, we use counterfactual simulation to take the glance patterns for manual radio tuning tasks from an on-road experiment and apply these patterns to lead-vehicle events observed in naturalistic driving studies. We then quantify how often the glance patterns from radio tuning are associated with rear-end crashes, compared to driving only situations. We used the pre-crash kinematics from 34 crash events from the SHRP2 naturalistic driving study to investigate the effect of radio tuning in crash-imminent situations, and we also investigated the effect of radio tuning on 2,475 routine braking events from the Safety Pilot project. The counterfactual simulation showed that off-road glances transform some near-crashes that could have been avoided into crashes, and glance patterns observed in on-road radio tuning experiment produced 2.85–5.00 times more crashes than baseline driving.

Examining drivers' eye glance patterns during distracted driving: Insights from scanning randomness and glance transition matrix

IntroductionVisual attention to the driving environment is of great importance for road safety. Eye glance behavior has been used as an indicator of distracted driving. This study examined and quantified drivers' glance patterns and features during distracted driving. MethodData from an existing naturalistic driving study were used. Entropy rate was calculated and used to assess the randomness associated with drivers' scanning patterns. A glance-transition proportion matrix was defined to quantity visual search patterns transitioning among four main eye glance locations while driving (i.e., forward on-road, phone, mirrors and others). All measurements were calculated within a 5s time window under both cell phone and non-cell phone use conditions. ResultsResults of the glance data analyses showed different patterns between distracted and non-distracted driving, featured by a higher entropy rate value and highly biased attention transferring between forward and phone locations during distracted driving. Drivers in general had higher number of glance transitions, and their on-road glance duration was significantly shorter during distracted driving when compared to non-distracted driving. ConclusionsResults suggest that drivers have a higher scanning randomness/disorder level and shift their main attention from surrounding areas towards phone area when engaging in visual-manual tasks. Practical applicationsDrivers' visual search patterns during visual-manual distraction with a high scanning randomness and a high proportion of eye glance transitions towards the location of the phone provide insight into driver distraction detection. This will help to inform the design of in-vehicle human-machine interface/systems.

The Effects of Augmented Reality Head-Up Displays on Drivers' Eye Scan Patterns, Performance, and Perceptions

This paper reports on an experiment comparing Head-Up Display (HUD) and Head-Down Display (HDD) use while driving in a simulator to explore differences in glance patterns, driving performance, and user preferences. Sixteen participants completed both structured (text) and semi-structured (grid) visual search tasks on each display while following a lead vehicle in a motorway (highway) environment. Participants experienced three levels of complexity (low, medium, high) for each visual search task, with five repetitions of each level of complexity. Results suggest that the grid task was not sensitive enough to the varying visual demands, while the text task showed significant differences between displays in user preference, perceived workload, and distraction. As complexity increased, HUD use during the text task corresponded with faster performance as compared to the HDD, indicating the potential benefits when using HUDs in the driving context. Furthermore, HUD use was associated with longer sustained glances (at the respective display) as compared to the HDD, with no differences in driving performance observed. This finding suggests that AR HUDs afford longer glances without negatively affecting the longitudinal and lateral control of the vehicle – a result that has implications for how future researchers should evaluate the visual demands for AR HUDs.

Relationships Between Older Drivers' Cognitive Abilities as Assessed on the MoCA and Glance Patterns During Visual-Manual Radio Tuning While Driving.

Research has established that long off-road glances increase crash risk, and other work has shown increased off-road glance behavior in older drivers. This study investigated the relationship between older drivers' (M = 66.3, range 61-69 years) cognitive abilities and the duration of off-road glances while engaged in secondary visual-manual activities. Twenty-two drivers completed the Montreal Cognitive Assessment (MoCA) prior to driving an instrumented vehicle and completing a set of radio-tuning tasks. Glance behavior was recorded and manually coded into 7 glance regions (toward the forward roadway, instrument cluster, center stack, rearview mirror, left, right, and other). On average, older drivers with higher MoCA scores used shorter glances and glanced away from the forward roadway for less total time when manually tuning the radio. These findings suggest that lower MoCA scores may represent a driving force behind the "age" differences reported in earlier studies of off-road glance behavior. Questions are raised concerning the identification of MoCA scores that might be used as inclusion cut-points in driving research and in identifying individuals needing further evaluation related to suitability for continuance of driving.

Augmented Mirrors

This study investigates the effects of Augmented Reality (AR) graphics on a drivers’ distance estimation and depth perception when using a video-based, AR-enhanced driver’s side mirror. Sixteen participants took part in the study, eight in a driving simulator and eight outside in a stationary vehicle. Participants experienced three different AR display image conditions, three different glance patterns, three different target vehicle speeds, and two own-vehicle image conditions. Distance data and confidence data were collected for each particpant and analyzed for any correlation between the conditions and performance. The results suggest that various AR images affected depth judgements and confidence levels. In addition, the vehicle speed and glance pattern of the videos also had significant effects.

Sensation Seeking and Drivers’ Glance Behavior while Engaging in a Secondary Task

To investigate possible relationships between drivers’ sensation seeking and glance behavior while interacting with human-machine interfaces, a total of 70 drivers’ eye-glance data, Sensation Seeking Scale (SSS), and Driver Behavior Questionnaire (DBQ) data were collected and analyzed. Participants conducted radio tuning tasks with two standard production interfaces while driving on a highway, and their glance allocations to defined regions were recorded and manually annotated. Results showed that sensation seeking scores were related with self-reported violation scores, off-road glance patterns, and driving speed: (1) violation scores of DBQ were positively correlated with sensation seeking, (2) mean and standard deviation of off-road glance duration were positively correlated with sensation seeking for younger drivers (under 40 years), (3) total off-road glance time per minute and number of off-road glances per minute were positively correlated with sensation seeking for older drivers (over 40 years), and (4) percentage of speed change was negatively correlated with sensation seeking for both younger and older drivers. The results indicate that sensation seeking is associated with drivers’ off-road glance patterns and driving behavior. These observations further highlight the relationship between personal traits and driver behavior.