Glance Behavior Research Articles

Want a ride? Attitudes towards autonomous driving and behavior in autonomous vehicles

SummaryResearch conducted previously has focused on either attitudes toward or behaviors associated with autonomous driving. In this paper, we bridge these two dimensions by exploring how attitudes towards autonomous driving influence behavior in an autonomous car. We conducted a field experiment with twelve participants engaged in non-driving related tasks. Our findings indicate that attitudes towards autonomous driving do not affect participants’ driving interventions in vehicle control and eye glance behavior. Therefore, studies on autonomous driving technology lacking field tests might be unreliable for assessing the potential behaviors, attitudes, and acceptance of autonomous vehicles.

Assessing driver distraction from in-vehicle information system: an on-road study exploring the effects of input modalities and secondary task types

In-vehicle information system (IVIS) use is prevalent among young adults. However, their interaction with IVIS needs to be better understood. Therefore, an on-road study aims to explore the effects of input modalities and secondary task types on young drivers' secondary task performance, driving performance, and visual glance behavior. A 2 × 4 within-subject design was undertaken. The independent variables are input modalities (auditory-speech and visual-manual) and secondary task types (calls, music, navigation, and radio). The dependent variables include secondary task performance (task completion time, number of errors, and SUS), driving performance (average speed, number of lane departure warnings, and NASA-TLX), and visual glance behavior (average glance duration, number of glances, total glance duration, and number of glances over 1.6 s). The statistical analysis result showed that the main effect of input modalities is significant, with more distraction during visual-manual than auditory-speech. The main impact of secondary task types was also substantial across most metrics, aside from average speed and average glance duration. Navigation and music were the most distracting, followed by calls, and radio came in last. The distracting effect of input modalities is relatively stable and generally not moderated by the secondary task types, except radio tasks. The findings practically benefit the driver-friendly human–machine interface design, preventing IVIS-related distraction.

The effect of human-machine interface modality, specificity, and timing on driver performance and behavior while using vehicle automation

The effectiveness of the human–machine interface (HMI) in a driving automation system during takeover situations is based, in part, on its design. Past research has indicated that modality, specificity, and timing of the HMI have an impact on driver behavior. The objective of this study was to examine the effectiveness of two HMIs, which vary by modality, specificity, and timing, on drivers’ takeover time, performance, and eye glance behavior. Drivers’ behavior was examined in a driving simulator study with different levels of automation, varying traffic conditions, and while completing a non-driving related task. Results indicated that HMI type had a statistically significant effect on velocity and off-road eye glances such that those who were exposed to an HMI that gave multimodal warnings with greater specificity exhibited better performance. There were no effects of HMI on acceleration, lane position, or other eye glance metrics (e.g., on road glance duration). Future work should disentangle HMI design further to determine exactly which aspects of design yield between safety critical behavior.

Text-based information design for in-vehicle displays: A systematic review

Reading short texts at a brief glance behaviour has been increasing, especially in the in-vehicle interface of the multitask context of driving. This behaviour led to a change in the approach of preparing and designing in-car information. Consequently, understanding the current status of existing design principles is necessary for designers to find the appropriate solution for text presentation on the in-vehicle display under driving conditions. Thus, the present study conducts a systematic review of text-based information design for in-vehicle displays. A total of 1,495 studies (1980–2022) were screened, and 360 emerged for full-text review; 27 studies met the inclusion criteria, including journal articles concerning the design of text-based information for in-vehicle displays involving text design and the effect on safe driving or reducing workload and distraction. The results were summarised in three themes of literature: existing design guidelines, design factors, and evaluation methodology. The existing guidelines and recommendations focus on the physical presence of text (modality, location, colour, typeface, font size), with few recommendations on the linguistic presence of the text (message style and message length). The discussion and future work section enables further work on developing usable and safe informational text displays on the in-vehicle interface and other short glance reading engagements.

The correlation between drivers’ road familiarity and glance behavior using real vehicle experimental data and mathematical models

Objective Road familiarity is an important factor affecting drivers’ visual features. Analyzing the quantitative correlation between drivers’ road familiarity and visual features in complex environment is of great help to improve driving safety. However, there are few relevant studies. This paper takes urban plane intersection as the environmental object to explore the correlation between drivers’ glance behavior and road familiarity, and conducts research on the quantitative evaluation model of road familiarity based on this correlation. Method First, a real vehicle experiment was carried out to record the eye movement data of 24 drivers with different road familiarity. The driver’s visual field plane was divided into 10 areas of interest (AOIs) based on the driver’s perspective. Three measures, including average glance duration, number of glances, and fixation transition probabilities between AOIs at urban plane intersections, were extracted. Finally, based on the experimental results, the driver road familiarity evaluation model was constructed using the factor analysis method. Results There are significant differences between unfamiliar and familiar drivers regarding the average glance duration toward the forward (FW) area, the left window (LW) area, the left rearview mirror (LVM) area and the left forward (LF) area, the number of glances toward the other (OT) area, and the fixation transition probabilities of LW→RF (right forward), LF→LF, LF→FW, FW→LW, FW→FW, FW→RVM (right rearview mirror). The comprehensive evaluation results show that the accuracy rate of the driver road familiarity evaluation model reached 83%. Conclusions This paper revealed that there is a strong correlation between drivers’ road familiarity and drivers’ glance behavior. Based on this correlation, we can include road familiarity as a part of drivers’ working status and establish a high accuracy evaluation model of driver road familiarity. The conclusion of this paper can provide some reference for the humanized design and improvement of advanced driving assistance system, which is of great significance for reducing the driving workload of drivers and improving the driving safety.

Visual Attention Patterns in Automated and Manual Lane Change Maneuvers

Tesla auto lane change is a feature which automatically steers the vehicle into an adjacent lane when prompted by the driver. 2,473 naturalistic motorway lane changes were annotated to compare driver eye glance behavior before, during, and after maneuvers performed manually and with automation. Change point detection and glance area prevalence revealed distinct patterns in the drivers’ glance behavior. In manual lane changes drivers had more glances to adjacent lanes and over-the-shoulder prior to and during lane crossing compared to automated lane changes which were associated with fewer glances laterally but more glances to the rearview mirror. These differences were more pronounced in lane changes into the leftmost lane. However, in automated exiting maneuvers, drivers had more preparatory glances to adjacent lanes compared to manual. These behaviors could be related to differences in driver visual attention associated with the use of automation.

The role of group membership in active road user attention across different age groups

This study aimed to explore how travelling in different group constellations (alone, with known or with unknown people) affects children's and adults’ visual behaviour in traffic when cycling or walking. Additionally, mobile phone/earphone usage was considered, too. A follow-along study (n = 43) and an observation study (n = 898) were conducted to observe travellers in a natural setting. In the follow-along study, eye-tracking was used to investigate children’s glances behaviour on their way to school and how well they manage to fulfil attentional requirements. The observational study focused on children's and adults' visual behaviour at several intersections. The main result of the study was that group membership appears to have a large influence on individuals' visual sampling strategy. In formal groups reliance on each other was found to be stronger than in informal groups. People with a natural responsibility in the group, such as parents or other adults, take a more active role in visual monitoring. Reliance on others is found to a greater extent among pedestrians than cyclists. Regarding communication devices, the use of earphones did not significantly affect glance behaviour towards relevant areas. In naturalistic situations, group constellation, age and phone/earphone usage are interlinked, which needs to be considered when studying these factors.

Multitasking While Driving: How Drivers Self-Regulate Their Interaction with In-Vehicle Touchscreens in Automated Driving

Driver assistance systems are designed to increase comfort and safety by automating parts of the driving task. At the same time, modern in-vehicle information systems with large touchscreens provide the driver with numerous options for entertainment, information, or communication, and are a potential source of distraction. However, little is known about how driving automation affects how drivers interact with the center stack touchscreen, i.e., how drivers self-regulate their behavior in response to different levels of driving automation. To investigate this, we apply multilevel models to a real-world driving dataset consisting of 31,378 sequences. Our results show significant differences in drivers’ interaction and glance behavior in response to different levels of driving automation, vehicle speed, and road curvature. During automated driving, drivers perform more interactions per touchscreen sequence and increase the time spent looking at the center stack touchscreen. Specifically, at higher levels of driving automation (level 2), the mean glance duration toward the center stack touchscreen increases by 36% and the mean number of interactions per sequence increases by 17% compared to manual driving. Furthermore, partially automated driving has a strong impact on the use of more complex UI elements (e.g., maps) and touch gestures (e.g., multitouch). We also show that the effect of driving automation on drivers’ self-regulation is greater than that of vehicle speed and road curvature. The derived knowledge can inform the design and evaluation of touch-based infotainment systems and the development of context-aware driver monitoring systems.

The influence of alcohol and automation on drivers’ visual behavior during test track driving

BackgroundDriving under the influence of alcohol severely increases crash risk. Impairment detection during driving is therefore key to improve traffic safety. However, future detection systems need to capture impairment for more reasons than alcohol intoxication and must also function in all driving modes (manual, assisted, autonomous). Driver Monitoring Systems (DMS) are promising candidates for such broader impairment detection. MethodA test track study investigated the effects of alcohol intoxication on drivers’ visual behavior both when just driving and when engaged in a non-driving related task. Twenty-six participants performed two drives: 1) sober baseline, 2) with a target blood alcohol concentration of 0.1%. The participants drove in either manual, assisted, or autonomous drive (AD) mode. ResultsIntoxication influenced glance behavior in all driving modes. It was most evident during visually demanding non-driving related tasks where it resulted in longer single and total off-path glance durations. Additionally, when just driving in manual mode, almost one third of the drivers displayed gaze concentration to the forward roadway when intoxicated. For sober driving, the difference in visual behavior between manual and assisted mode were moderate. In contrast, there was a huge shift towards longer off-path glances, lower percent road center, and lower off-path glance frequency in AD mode. ConclusionsIntoxication clearly affects drivers’ on/off road glance behavior. However, it is necessary to account for both driving mode and engagement in non-driving related tasks to reliably distinguish sober from drunk driving. Glance metrics has the potential to serve as a sub-set of indicators for a broader DMS-based detection of impaired driving, which can inform the decisions on when to activate in-vehicle countermeasures.

Driver eye glance behavior and performance with camera-based visibility systems versus mirrors

Objective Drivers’ ability to extract visual information efficiently from mirrors or camera-based visibility systems impacts driving performance when carrying out maneuvers such as lane changes. The objective of the research was to compare drivers’ eye gaze behavior and driving performance with mirrors versus camera-based visibility systems (i.e., CMS, or camera monitor system) to identify any differences and possible impacts on safety. Methods A test track study was conducted comparing drivers’ eye gaze and lane change behavior when driving a vehicle equipped with outside mirrors versus a prototype CMS. Participants’ opinions regarding usability and comfort in using mirrors versus the tested CMS were also obtained using a post-drive questionnaire. Results Study results were somewhat mixed but did demonstrate that with the tested CMS, participants took longer to pass a slower moving vehicle and maintained a greater resultant distance from the passed vehicle. Additionally, participants had a greater number of fixations to the CMS displays compared to the outside rearview mirrors. Results also found slight perceived advantages for the tested CMS in regard to ease of use, comfortability, and visibility. When asked to choose which rear visibility technology they would prefer to use in everyday driving, most participants preferred the outside rearview mirrors over the tested prototype CMS or having both systems. However, not all lane change and gaze metrics followed the same pattern. Conclusions In this study, participants’ longer time to pass a slower moving vehicle, greater distance when passing a slower moving vehicle, greater number of fixations, and lower subjective ratings with the tested CMS may indicate difficulty in judging distances and focusing on the electronic image. This study provides preliminary findings that suggest differences in driving behavior exist between a single tested prototype CMS and outside rearview mirrors and is a foundational step toward evaluating whether these trends are consistent across different systems and overall implications for safe driving behavior.

Impact of level 2 automation and ADHD symptomatology on young drivers’ attention maintenance

Attention Deficit Hyperactivity Disorder (ADHD) is a common mental disorder that affects drivers’ attention maintenance skills. Vehicle automation is another factor that has an increasingly important effect on drivers’ attention or concentration level during driving. This driving simulator study aimed to investigate how drivers’ glance behavior changed under different levels of vehicle automation and according to high or low ADHD symptomology. During the drive, participants were asked to complete a map task on a tablet while their glance behavior was recorded. Results imply that the participants tend to have a longer mean off-road glance duration and a larger long off-road glance ratio when they were under the L2 automation level compared to manual mode. ADHD symptomatology and the interaction between ADHD and vehicle automation were not found to be significant. These findings indicate that vehicle automation negatively impacts drivers’ attention maintenance skills. Future work would consider more kinds of secondary tasks that can assess the participants’ attention maintenance skills from various aspects.

Driver response and recovery following automation initiated disengagement in real-world hands-free driving

Objective Advanced driver assistance systems are increasingly available in consumer vehicles, making the study of drivers’ behavioral adaptation and the impact of automation beneficial for driving safety. Concerns over driver’s being out-of-the-loop, coupled with known limitations of automation, has led research to focus on time-critical, system-initiated disengagements. This study used real-world data to assess drivers’ response to, and recovery from, automation-initiated disengagements by quantifying changes in visual attention, vehicle control, and time to steady-state behaviors. Methods Fourteen drivers drove for one month each a Cadillac CT6 equipped with Super Cruise (SC), a partial automation system that, when engaged, enables hands-free driving. The vehicles were instrumented with data acquisition systems recording driving kinematics, automation use, GPS, and video. The dataset included 265 SC-initiated disengagements identified across 5,514 miles driven with SC. Results Linear quantile mixed-effects models of glance behavior indicated that following SC-initiated disengagement, the proportions of glances to the Road decreased (Q50Before=0.91, Q50After=0.69; Q85Before=1.0, Q85After=0.79), the proportions of glances to the Instrument Cluster increased (Q50Before=0.14, Q50After=0.25; Q85Before=0.34, Q85After=0.45), and mean glance duration to the Road decreased by 4.86 sec in Q85. Multinomial logistic regression mixed-models of glance distributions indicated that the number of transitions between glance locations following disengagement increased by 43% and that glances were distributed across fewer locations. When driving hands-free, take over time was significantly longer (2.4 sec) compared to when driving with at least one hand on the steering wheel (1.8 sec). Analysis of moment-to-moment distributional properties of visual attention and steering wheel control following disengagement indicated that on average it took drivers 6.1 sec to start the recovery of glance behavior to the Road and 1.5 sec for trend-stationary proportions of at least one hand on the steering wheel. Conclusions Automation-initiated disengagements triggered substantial changes in driver glance behavior including shorter on-road glances and frequent transitions between Road and Instrument Cluster glance locations. This information seeking behavior may capture drivers’ search for information related to the disengagement or the automation state and is likely shaped by the automation design. The study findings can inform the design of more effective driver-centric information displays for smoother transitions and faster recovery.

Distraction pattern classification and comparisons under different conditions in the full-touch HMI mode

Understanding driver distraction patterns is an important part of human–machine interaction (HMI), which is beneficial for the development of control strategies in human–machine co-driving systems. However, comparatively few studies have focused on driver distraction patterns. To address this issue, this study proposes a framework to characterize distraction patterns using glance behavior and manual behavior, and classifies distraction patterns into: aggressive, moderate, and conservative patterns based on real road experiments. Subsequently, differences in distraction behavior and effects on lateral vehicle control ability across distraction pattern groups, as well as distraction behavior differences exhibited by drivers in the same distraction pattern group under different conditions, are analyzed. Firstly, the results show that the aggressive distraction patterns have a smaller number of eyes-off-road (NoEOR) incidences but longer mean single eyes-off-road time (MSEORT), maximum single eyes-off-road time (MaxEORT) and a higher percentage of long eyes-off-road (PoLEOR) incidences than the other patterns. There are slight differences in the single eyes-off-road times (EORTs) between the conservative and moderate patterns and in the manual behavior for the aggressive and moderate distraction patterns. Secondly, the same distraction pattern exhibited by drivers for different road and secondary task conditions has differences in terms of the behavioral performance. Finally, there is few differences in the lateral motion of a vehicle with different distraction patterns. Surprisingly, the standard deviation of the steering wheel angle (SDSWA) is the smallest in the aggressive distraction pattern.

Exploring the influence of driver affective state and auditory display urgency on takeover performance in semi-automated vehicles: Experiment and modelling

As semi-automated vehicles become more available to the general public, it is important to investigate human factors, including both the driver side and the interface side. Despite much research on semi-automated vehicles, little research has conducted considering both driver states and takeover request display design. The present study investigated the effects of drivers’ affective states and auditory display urgency on takeover response time and performance quality. Thirty-six participants experienced takeover scenarios in a semi-automated vehicle using a driving simulator, while playing an online game. For takeover quality, angry drivers drove faster, took longer to change lanes and had lower steering wheel angles than neutral drivers, which made riskier driving. However, there was no difference in eye glance behaviors. Higher frequency and more repetitions of the auditory displays led to faster takeover reaction times, but there was no time difference between angry and neutral drivers. Drivers’ response time to takeover displays from both affect groups was modelled using the QN-MHP framework, which resulted in a R2 of 0.505 with the empirical data collected. In sum, results suggest that drivers’ anger state influenced takeover quality, while display urgency influenced takeover response time. This study is expected to make a significant contribution to research on the influence of emotion, specifically, anger on takeover performance in semi-automated vehicles as well as to the takeover display design.

Age differences in driver visual behavior and vehicle control when driving with in-vehicle and on-road deliveries of service logo signs

With the advances in vehicle technologies, more information is communicated in real-time to the driver via an in-vehicle interface. In-vehicle messaging may deliver safety-related information such as warnings as well as non-safety-related information such as an upcoming lodging place. While much research has focused on the design of messaging safety-related information, little is known about the best practice in in-vehicle messaging of non-safety-related information. This study investigated the effects of information source and load on driver signage logo identification, glance behavior, and vehicle control among younger, middle-aged and older drivers. The logos were presented on: (1) an on-road sign panel, (2) an in-vehicle display, or (3) a combination of both, with half of the drives showing logo only, and the other half of the drives showing logo plus additional text. The general findings support the use of in-vehicle displays, especially when it is presented simultaneously with on-road signs. In-vehicle displays did not lead to a higher workload or more visual distraction, and simultaneous presentations resulted in slightly better speed control. The findings also showed minimal negative impacts on logo identification from increased information load. Older drivers performed less well in signage identification and vehicle control, and they made longer glances to logo information suggesting design considerations should be made to accommodate specific driver characteristics.

Factors affecting drivers’ off-road glance behavior while interacting with in-vehicle voice interfaces

In-vehicle voice-based interfaces have been massively embedded in modern vehicles as a countermeasure to visual-manual distractions. However, limited data are available regarding the actual visual demands imposed on the driver when interacting with such an interface. How those factors that are associated with the drivers themselves affect their visual behavior when interacting with an in-vehicle voice-based interface remains understudied. This study focused on investigating factors affecting drivers’ off-road visual behavior while interacting with a voice-based interface. A secondary data analysis leveraging structural equation modeling was performed based on data collected from a large-scale field experiment wherein participants drove a vehicle on the highway and performed a series of tasks using an in-vehicle voice interface. The results indicate that while voice-based interfaces are designed to help reduce drivers’ visual demands, drivers can still direct their eyes off the road for a prolonged period of time during the interaction and exhibit potentially risky visual behavior. In addition, individual-level factors can exert influence on drivers’ off-road visual behavior in such a way that older drivers and drivers with higher trust in technologies are associated with more long eyes-off-road glances when interacting with a voice-based interface. To promote the general public’s adoption of in-vehicle voice interfaces and support safe interactions, we recommend relevant research and system design consider drivers’ trust in, perceptions, and previous usage of similar technologies, as well as their individual characteristics such as age.

Bears in our midst: Familiarity with Level 2 driving automation and attending to surprise on-road events

This study presents an on-road paradigm to measure the effect of Level 2 (L2) system familiarity on attention strategies to salient, but non-hazardous, driving-related events when using the driving automation. A vehicle with an oversized pink teddy bear on the back overtook participants three times while they drove a 2019 Mercedes-Benz C300 equipped with a L2 system for 1 h. This L2 system requires drivers to keep their hands on the wheel while activated. The L2 system was turned on or off, depending on the assigned condition, and participants varied in their familiarity with L2 systems. Cameras recorded participant eye glance behavior. After the drive, participants were asked to recall the bear and the number of times their mind had wandered from the driving task during the drive. Results show that the driving automation support gave only participants familiar with L2 systems an advantage for greater bear recall. Unfamiliar participants were at a relative disadvantage when assisted by the L2 system, with generally poorer bear recall than unfamiliar participants who drove with the system off. Better bear recall corresponded with wider on-road gaze dispersion and more instances of mind wandering. Our findings support the effectiveness of this paradigm to measure driver attention when using a L2 system under real-world conditions and highlight the need to consider the role of driving automation familiarity in future research.

Differences in Drivers’ Glance Behavior and Lateral Control Ability during Full-Touch Interaction Mode and Conventional Interaction Mode: A Case Study of Road Experiments

In recent years, the full-touch human–machine interaction (HMI) mode has gained popularity in the automotive market. However, little research has been conducted on how this interaction mode affects drivers’ glance behavior and lateral control ability. In this study, we evaluated the visual engagement and driving performance of 30 participants while driving two vehicles equipped with either the full-touch interaction mode (FTIM) or the conventional interaction mode (CIM) provided by the original equipment manufacturer (OEM). We found that both air conditioning–related tasks required more visual engagement, longer task completion time, and worse lateral vehicle control under FTIM. Furthermore, the gray correlation analysis demonstrated that FTIM exhibited slightly different disadvantages in the two secondary tasks. In the temperature adjustment task, the correlations of glance behavior and lateral control ability between the two interactive modes were 0.688 and 0.680, respectively. In the airflow adjustment task, the correlations of glance behavior and lateral control ability between the two interactive modes were 0.659 and 0.668, respectively. In addition, this study revealed that driving speed had significant effects on glance behavior and lateral driving performance in both interaction modes. As speed increased, self-adjusting glance behavior was evident in performing the secondary task; however, this behavior could not compensate for the deterioration in lateral driving performance caused by the increased speed. The findings will help improve drivers’ perception of FTIM and provide theoretical guidance for the design development of HMI mode.

Comparative Evaluation of Twelve Automotive CMS Display Layout Designs in Two Lane Change Task Scenarios

Recently, the automotive industry has shown a growing interest in replacing conventional automotive mirrors with camera monitor systems (CMS) as CMS may provide multiple advantages, including the enhancement of driving safety through the human factors design of display layout. Despite past research, however, pertinent CMS display layout design guidelines have not been firmly established, due to insufficient empirical knowledge. To alleviate the current research gaps and inform the optimal design, this study comparatively evaluated twelve CMS display layout designs employing driving performance and eye glance behavior measures: six separate display layouts (three with the side-view displays near the A-pillars, and, three, on each side of the steering wheel), and, six integrated display layouts (three “segmented” and three “stitched” view layouts). Each design was evaluated in a safety-critical and a less safety-critical lane change scenario. The study demonstrated that the display layout design effects could vary across different driving task scenarios.

How does navigating with Augmented Reality information affect drivers’ glance behaviour in terms of attention allocation?

Drivers can benefit from Augmented Reality (AR) information especially in ambiguous navigation situations compared to conventional Head-up displays (HUD). AR information is correctly superimposed on the relevant objects in the environment and therefore directly related to the driving situation. Hence, it is assumed, that drivers no longer have to switch glances between the AR information and the environment (Kim & Dey, 2009). It has to be investigated whether switching glances between the presented navigation information and the environment can be reduced with AR information compared to HUD information. Furthermore, the question arises whether AR information might capture drivers’ attention and therefore distract from the traffic situation compared to a HUD as AR information is presented on the driver’s primary visual axis. The aim of the driving simulator study was to examine glance behaviour in terms of attention allocation while participants navigated in an ambiguous left turn situation with an oncoming car in an urban area (N = 58). Hence, drivers were faced with the decision to turn in front of it or let it pass. A conventional HUD and an AR display presented the navigation information to the driver. The drives differed in traffic complexity (low vs. high) to provide indications whether drivers adapt glance behaviour to altered environmental conditions. Besides the navigation task, drivers performed a non-driving-related task to raise drivers’ mental load while navigating. Results showed that with the AR display participants payed more attention to an oncoming car in the ambiguous left turn situation than with the HUD, which indicates that AR information was not distracting. Furthermore, participants switched glances significantly less between the AR navigation information and the environment, which indicates that with the AR display the driver did not have to map the virtual information onto the real driving situation. Independently of the display type 88% of the participants let the oncoming car pass the first time in this situation. Moreover, subjective data showed that drivers benefitted from AR information. The results of this study contribute to the investigation and development of AR user interfaces.