Non-driving Related Tasks Research Articles

Awakening the Disengaged: Can Driving-Related Prompts Engage Drivers in Partial Automation?

This study explores the effectiveness of conversational prompts on enhancing driver monitoring behavior and takeover performance in partially automated driving under two non-driving-related task (NDRT) scenarios with varying workloads. Driver disengagement in partially automated driving is a serious safety concern. Intermittent conversational prompts that require responses may be a solution. However, existing literature is limited with inconsistent findings. There is little consideration of NDRTs as an important context, despite their ubiquitous involvement. A method is also lacking to measure driver engagement at the cognitive level, beyond manual and visual engagements. Participants operated a partially automated vehicle in a simulator across six predefined drives. In each drive, participants either received driving-related prompts, daily-conversation prompts, or no prompts, with or without a takeover notification. The first experiment instructed participants to engage in NDRTs at their choice and the second experiment incentivized solving demanding anagrams with monetary rewards. When participants were voluntarily engaged in NDRTs, answering driving-related prompts and receiving takeover notifications improved their monitoring behavior and takeover performance. However, when participants were involved in the more demanding and incentivized NDRT, answering prompts had little effect. The study supports the importance of both maintaining appropriate workload and processing driving-related information during partially automated driving. Driving-related prompts improve driver engagement and takeover performance, but they are not robust enough to compete with NDRTs that have high motivational appeals and cognitive demands. The design of driver engagement tools should consider the workload and information processing mechanisms.

Takeover and non-driving related task performance in conditional automated driving: EEG and behavior Parameters interaction

In this study, a conditional automated driving scenario is simulated using virtual reality (VR) technology to explore whether office works presented through augmented reality (AR) affect task and takeover performance, and the neural mechanism was revealed. Sixty-four participants were recruited and their electroencephalography (EEG) was used to measure the brain activities. The results indicated that non-driving-related tasks (NDRTs) requiring higher internal attention focus resulted in poorer task and takeover performance. The alpha power decline magnitude in the parietotemporal (PT) was positively correlated with the takeover time; and the greater the alpha power decline in the right centroparietal (CP) hemisphere, the worse is the participants’ memory quality for NDRTs. The ventral attention network (VAN) and right parietal cortex, which are active during working memory, are more likely to explain these findings. The results can provide suggestions for the design of AR-ADS and help improve the safety in L3 driving automation systems.

Modeling takeover time for non-driving related tasks using hybrid deep neural networks in conditionally automated vehicles

To enhance the safety of the transition from autonomous to manual driving control, one of the key factors is quantifying the safe takeover transition, namely the takeover time. Previous studies have focused on factors influencing takeover time and predicting takeover time using traditional methods, but there is a lack of research utilizing deep learning networks and driver physiological factors for takeover time prediction. Toward this end, we propose a hybrid deep learning network model dedicated to predicting drivers’ takeover time with a dataset obtained from a bench experiment involving 46 drivers. Firstly, to address the potential information loss commonly associated with traditional manual feature extraction techniques, a convolutional neural network (CNN) is developed to automatically extract physiological spatial representations from drivers. Additionally, a double-layer bidirectional long short-term memory network (DBiLSTM) is incorporated to model the physiological time representation dimension, enabling a more refined representation of physiological signals. Then, to tackle the complexities introduced by individual driver variations, a self-attention mechanism is enabling the model to automatically prioritize critical features, thereby enhancing the accuracy and reliability of its predictions. The results demonstrate that the proposed approach surpasses traditional machine learning network models in predictive accuracy for takeover time prediction, thereby exhibiting novelty in the field of vehicle takeover control research. This provides a more reliable technical foundation for anticipating the durations of vehicle takeovers.

Want a ride? Attitudes towards autonomous driving and behavior in autonomous vehicles

SummaryResearch conducted previously has focused on either attitudes toward or behaviors associated with autonomous driving. In this paper, we bridge these two dimensions by exploring how attitudes towards autonomous driving influence behavior in an autonomous car. We conducted a field experiment with twelve participants engaged in non-driving related tasks. Our findings indicate that attitudes towards autonomous driving do not affect participants’ driving interventions in vehicle control and eye glance behavior. Therefore, studies on autonomous driving technology lacking field tests might be unreliable for assessing the potential behaviors, attitudes, and acceptance of autonomous vehicles.

Meta-Analysis on the Impact of Different Channel Loads on Takeover Performance in Autonomous Driving under Non-Driving Tasks

Abstract Under conditions of high automation, drivers’ excessive engagement in non-driving related tasks can severely impact their takeover ability, posing a significant threat to road safety. Therefore, it’s essential to explore the channel resources occupied by non-driving related tasks to enhance driving safety. However, many studies have merely categorized different types of tasks in experiments without clarifying the relationship between the tasks and the channel resources they occupy. To determine whether different channel loads under non-driving related tasks have varying impacts on autonomous driving takeover performance, a meta-analysis was conducted on literature retrieved from January 2021 to January 2024. The results indicate that tasks occupying visual, auditory, motor, and cognitive channel resources during takeover all affect takeover performance. Among these, cognitive channel resource occupation significantly reduces takeover efficiency, followed by motor and auditory channels, with the visual channel having the least impact.

Where's my TOR?: Evaluating the Effect of Take-Over Request Source on Older Drivers' Control Transition in Level 3 Cars

It is challenging for older drivers to transition to manual control after a Take-Over Request (TOR) has been issued by a Level 3 car. This study investigated if the presentation source of the TOR affects driver performance when resuming control. We measured take-over performance, hazard perception, and user acceptance when the TOR was presented on (1) a smartphone displaying a Non-Driving Related Task (NDRT) simultaneously with the In-Vehicle Information System (IVIS), or (2) presented on the IVIS only. Two NDRTs that varied in cognitive demand were tested with older drivers aged 60-69 and 70+. For the lower cognitive demand NDRT, presenting the TOR on the smartphone+IVIS improved takeover performance, hazard perception, and user acceptance, with greater benefits observed in the 70+ group. For the cognitively demanding NDRT, the smartphone+IVIS presentation did not benefit either group of drivers. TOR designers can apply these findings to enhance TORs and assist older drivers in managing control transitions considering the NDRT cognitive demand.

Assessing advanced driver assistance systems in police vehicles under demanding conditions

Advanced driver-assistance systems (ADAS) are technologies that can enhance drivers’ safety by relieving them from some driving related activities. However, police driving conditions and demands are different from those of civilian drivers. The objective of this study was to assess the impact of ADAS such as forward collision warning (FCW), automatic emergency braking (AEB), and blind spot monitoring (BSM) on police officers’ driving performance, workload, and trust in vehicle safety to provide personalised solutions for police vehicles. A driving simulation study was conducted with 18 police officers. ADAS use was assessed under various driving conditions and while officers were engaged in non-driving related tasks. Findings suggested that the FCW and AEB systems improved officers’ driving performance, while the BSM system had limited effectiveness due to low salience. ADAS were beneficial under normal driving conditions and when officers were using in-vehicle technology; however, they did not help officers in pursuit conditions.

Verifying Motion Sickness Induction in Automated Vehicles Using Motion-Based Driving Simulators

Motion sickness in passengers of automated vehicles could prevent users from taking advantage of the automation features, as well as lead to unsafe takeover conditions. To conduct motion sickness research, the first step requires motion sickness induction, and its verification. In the current study, twenty-nine participants experienced being a passenger in a fully self-driving vehicle on a motion-based driving simulator. A Control route designed to mimic typical driving conditions on a driving simulator, and a Treatment route that produced large lateral accelerations were tested. The Treatment route was also tested with a non-driving related task (NDRT). Results showed that exposure duration and increased lateral accelerations significantly increased motion sickness. Addition of the cognitively demanding NDRT led to a reduction in motion sickness, suggesting the possibility of a masking effect when engaged in an NDRT in an automated vehicle. Overall, the designed Treatment route was verified to successfully induce motion sickness.

Capturing the Mind: Non-Driving-Related Tasks as a Window into Cognitive Engagement in Automated Driving

Supervising automated driving requires visual, manual, and cognitive engagement from the driver. While visual and manual engagement measures are already developed and implemented, cognitive engagement is much more difficult to observe. Considering drivers’ self-regulatory behavior when involved in non-driving-related tasks (NDRTs) during partial automation, the demand and duration of these NDRTs may indicate a certain level of driver cognitive engagement. The study explores a rating measure to partially estimate driver cognitive engagement based on the demand and duration of NDRTs and evaluate the psychometric properties of the NDRT rating. We examined the internal consistency of the measure by computing split-half reliability, tested its sensitivity to factor manipulation, and assessed criterion validity. Our initial findings suggest that the method is reliable and valid, showing satisfactory internal consistency, sensitivity to experimental manipulation, and criterion validity in predicting the takeover performance of supervising vehicle automation.

Driver Situation Awareness for Regaining Control from Conditionally Automated Vehicles: A Systematic Review of Empirical Studies.

An up-to-date and thorough literature review is needed to identify factors that influence driver situation awareness (SA) during control transitions in conditionally automated vehicles (AV). This review also aims to ascertain SA components required for takeovers, aiding in the design and evaluation of human-vehicle interfaces (HVIs) and the selection of SA assessment methodologies. Conditionally AVs alleviate the need for continuous road monitoring by drivers yet necessitate their reengagement during control transitions. In these instances, driver SA is crucial for effective takeover decisions and subsequent actions. A comprehensive review of influential SA factors, SA components, and SA assessment methods will facilitate driving safety in conditionally AVs but is still lacking. A systematic literature review was conducted. Thirty-four empirical research articles were screened out to meet the criteria for inclusion and exclusion. A conceptual framework was developed, categorizing 23 influential SA factors into four clusters: task/system, situational, individual, and nondriving-related task factors. The analysis also encompasses an examination of pertinent SA components and corresponding HVI designs for specific takeover events, alongside an overview of SA assessment methods for conditionally AV takeovers. The development of a conceptual framework outlining influential SA factors, the examination of SA components and their suitable design of presentation, and the review of SA assessment methods collectively contribute to enhancing driving safety in conditionally AVs. This review serves as a valuable resource, equipping researchers and practitioners with insights to guide their efforts in evaluating and enhancing driver SA during conditionally AV takeovers.

Drivers’ situational awareness of surrounding vehicles during takeovers: Evidence from a driving simulator study

This study aimed to understand the influence of surrounding vehicle configuration, driving lane, and traffic density on drivers’ situational awareness (SA), takeover performance, and eye-tracking behaviors in conditionally automated driving. An experiment was conducted with the participation of 40 university students using a fixed-base driving simulator configured to simulate SAE Level 3 automation. During the experiment, participants were engaged in playing Tetris on a tablet as a non-driving related task in automated driving mode. Upon hearing an auditory takeover request, participants were instructed to take control of the vehicle, and then complete a scene reconstruction task to report their SA after transferring control back to the automated driving system. Our findings showed that drivers often neglected vehicles at their sides and rear during the takeover, which was associated with higher collision risks. Higher oncoming traffic density led to drivers’ worse SA of surrounding vehicles but more cautious driving behavior. Driving in the right lane generally resulted in smoother takeovers with lower collision risks. Interestingly, while SA did moderate the impacts of driving conditions on safety margins, a higher level of SA did not consistently relate to improved performance, especially in complex scenarios. This suggests the need for support systems that guide drivers to focus on safety–critical objects rather than simply amplifying SA in general. These insights have significant implications for the design of driver monitoring and support systems in automated vehicles.

TimelyTale: A Multimodal Dataset Approach to Assessing Passengers' Explanation Demands in Highly Automated Vehicles

Explanations in automated vehicles enhance passengers' understanding of vehicle decision-making, mitigating negative experiences by increasing their sense of control. These explanations help maintain situation awareness, even when passengers are not actively driving, and calibrate trust to match vehicle capabilities, enabling safe engagement in non-driving related tasks. While design studies emphasize timing as a crucial factor affecting trust, machine learning practices for explanation generation primarily focus on content rather than delivery timing. This discrepancy could lead to mistimed explanations, causing misunderstandings or unnecessary interruptions. This gap is partly due to alack of datasets capturing passengers' real-world demands and experiences with in-vehicle explanations. We introduce TimelyTale, an approach that records passengers' demands for explanations in automated vehicles. The dataset includes environmental, driving-related, and passenger-specific sensor data for context-aware explanations. Our machine learning analysis identifies proprioceptive and physiological data as key features for predicting passengers' explanation demands, suggesting their potential for generating timely, context-aware explanations. The TimelyTale dataset is available at https://doi.org/10.7910/DVN/CQ8UB0.

Attentional warnings caused by driver monitoring systems: How often do they appear and how well are they understood?

The present study investigated the effects of a driver monitoring system that triggers attention warnings in case distraction is detected. Based on the EuroNCAP protocol, distraction could either be long glances away from the forward roadway (≥3s) or visual attention time sharing (>10 cumulative seconds within a 30 s time interval). In a series of manual driving simulator drives, 30 participants completed both driving related tasks (e.g., changing multiple lanes in dense traffic) and non-driving related tasks (e.g., infotainment operations). Results of warning frequencies revealed that visual attention time sharing warnings occurred more frequently than long distraction warnings. Moreover, there was a large number of attention warnings during driving related tasks. Results also revealed that participants’ mental models tended to be less accurate when it came to understanding of the visual attention time sharing warnings as compared to the long distraction warnings, which were understood more accurately. Based on these observations, the work discusses the applicability and design of driver monitoring warnings.

How do visual and cognitive non-driving related tasks affect drivers’ visual attention and takeover performance in conditionally automated driving?

This study examines changes in visual attention during drivers’ engagement in visual or cognitive NDRTs during conditionally automated driving, and determines how these changes affect takeover performance. Seventy-five participants took part in a driving simulator study, performing three pre-takeover tasks: the three pre-takeover tasks: an auditory-imagery-verbal task (cognitive NDRT); a video-watching task (visual NDRT); and a monitoring task (baseline/non NDRT). Also, there were two hazardous events (breakdown or sudden merging of the vehicle ahead) leading to takeover requests issued with 7-s or 5-s lead times. The results revealed that NDRTs negatively affected visual attention, which caused lower saccade frequency between different areas of interest and shorter saccade amplitude. Drivers paid more visual attention to the road ahead and in-vehicle information system when performing cognitive and visual NDRTs, respectively. The visual attention of drivers performing NDRTs negatively affected takeover performance (e,g longer reaction time, heavier maximal brake pedal input, etc.). The reduction in takeover request lead time impaired takeover performance. The findings will support the design of eye tracker-based “out-of-loop” discrimination techniques and human-machine interaction interfaces in automated vehicles. This study contributes to the literature by examining how different types of NDRTs affect takeover performance, specifically focusing on drivers’ visual attention.

From Distraction to Action: Elevating Situation Awareness with Visual Assistance in Level 3 Autonomous Driving

This study examines the impact of visual assistance and cognitive load on situation awareness (SA) during takeover events in Level 3 (L3) autonomous driving, where drivers are permitted to engage in Non-Driving Related Tasks (NDRTs). Utilizing a driving simulator, the research explores how different NDRTs, occupying various sensory channels, influence drivers’ SA under two visual assistance conditions: full marking and key marking. Results from the Situation Awareness Global Assessment Technique (SAGAT) and NASA Task Load Index (NASA-TLX) scales, along with eye-tracking data, demonstrate significant interactions between visual assistance conditions and tasks, highlighting key marking’s superior effectiveness in enhancing SA across all NDRTs. The study reveals that cognitive workload, largely arising from NDRTs rather than road attention, remains unaffected by visual assistance conditions. Findings suggest that drivers’ strategies for allocating attention are consistent, regardless of the visual task presence. This research contributes to understanding SA in autonomous driving takeovers, offering valuable insights for developing Human-Machine Interface (HMI) designs to improve SA and takeover performance in autonomous driving systems.

Age-related differences in takeover performance: A comparative analysis of older and younger drivers in prolonged partially automated driving

Objective Vehicle automation technologies have the potential to address the mobility needs of older adults. However, age-related cognitive declines may pose new challenges for older drivers when they are required to take back or “takeover” control of their automated vehicle. This study aims to explore the impact of age on takeover performance under partially automated driving conditions and the interaction effect between age and voluntary non-driving-related tasks (NDRTs) on takeover performance. Method A total of 42 older drivers (M = 65.5 years, SD = 4.4) and 40 younger drivers (M = 37.2 years, SD = 4.5) participated in this mixed-design driving simulation experiment (between subjects: age [older drivers vs. younger drivers] and NDRT engagement [road monitoring vs. voluntary NDRTs]; within subjects: hazardous event occurrence time [7.5th min vs. 38.5th min]). Results Older drivers exhibited poorer visual exploration performance (i.e., longer fixation point duration and smaller saccade amplitude), lower use of advanced driving assistance systems (ADAS; e.g., lower percentage of time adaptive cruise control activated [ACCA]) and poorer takeover performance (e.g., longer takeover time, larger maximum resulting acceleration, and larger standard deviation of lane position) compared to younger drivers. Furthermore, older drivers were less likely to experience driving drowsiness (e.g., lower percentage of time the eyes are fully closed and Karolinska Sleepiness Scale levels); however, this advantage did not compensate for the differences in takeover performance with younger drivers. Older drivers had lower NDRT engagement (i.e., lower percentage of fixation time on NDRTs), and NDRTs did not significantly affect their drowsiness but impaired takeover performance (e.g., higher collision rate, longer takeover time, and larger maximum resulting acceleration). Conclusions These findings indicate the necessity of addressing the impaired takeover performance due to cognitive decline in older drivers and discourage them from engaging in inappropriate NDRTs, thereby reducing their crash risk during automated driving.

Effectiveness of smart LED strips at mid-block crosswalks under distracted driving conditions

We investigated the effectiveness of an LED-based smart mid-block crosswalk system in mitigating the detrimental effects of driver engagement in non-driving-related tasks (NDRTs) with behavioural, performance, and subjective measurements. We designed a 2 (Crosswalk: smart vs conventional) by 2 (Task complexity: low vs. high NDRT) within-subjects experiment. Thirty-six drivers drove along four urban scenarios in a static driving simulator. We collected data on driving behaviour (speed, reaction distance), and safety (minimum time-to-collision [MTTC]), as well as subjective driver ratings on the perceived task load and their trust in the technology used, and performance levels achieved while performing the NDRTs.Behavioural and performance observations showed that the smart mid-block crosswalk resulted in greater reaction distances and MTTC values when drivers interacted with pedestrians, thus indicating improved safety. Remarkably, the results also revealed that increased NDRT complexity does not negatively affect the smart crosswalk effectiveness in terms of driver-pedestrian collision prevention (i.e., MTTC does not decrease significantly). However, the NDRT complexity influenced driving performance in terms of speed and reaction distance at brake pedal pressure, with drivers exhibiting lower speeds and lower reaction distances with higher task loads. Moreover, the subjective ratings and performance levels while performing a NDRT reflected the experimental manipulation, with drivers perceiving higher task loads and performing worse in the higher NDRT complexity condition. Overall, the smart mid-block crosswalk led to a safer driver-pedestrian interaction compared to conventional crosswalks and achieved a good acceptance level both of which augur well for the widespread future installation of this technology.

The effect of human-machine interface modality, specificity, and timing on driver performance and behavior while using vehicle automation

The effectiveness of the human–machine interface (HMI) in a driving automation system during takeover situations is based, in part, on its design. Past research has indicated that modality, specificity, and timing of the HMI have an impact on driver behavior. The objective of this study was to examine the effectiveness of two HMIs, which vary by modality, specificity, and timing, on drivers’ takeover time, performance, and eye glance behavior. Drivers’ behavior was examined in a driving simulator study with different levels of automation, varying traffic conditions, and while completing a non-driving related task. Results indicated that HMI type had a statistically significant effect on velocity and off-road eye glances such that those who were exposed to an HMI that gave multimodal warnings with greater specificity exhibited better performance. There were no effects of HMI on acceleration, lane position, or other eye glance metrics (e.g., on road glance duration). Future work should disentangle HMI design further to determine exactly which aspects of design yield between safety critical behavior.

The Impact of Transparency on Driver Trust and Reliance in Highly Automated Driving: Presenting Appropriate Transparency in Automotive HMI

Automation transparency offers a promising way for users to understand the uncertainty of automated driving systems (ADS) and to calibrate their trust in them. However, not all levels of information may be necessary to achieve transparency. In this study, we conceptualized the transparency of the automotive human–machine interfaces (HMIs) in three levels, using driving scenarios comprised of two degrees of urgency to evaluate drivers’ trust and reliance on a highly automated driving system. The dependent measures included non-driving related task (NDRT) performance and visual attention, before and after viewing the interface, along with the drivers’ takeover performance, subjective trust, and workload. The results of the simulated experiment indicated that participants interacting with an SAT level 1 + 3 (system’s action and projection) and level 1 + 2 + 3 (system’s action, reasoning, and projection) HMI trusted and relied on the ADS more than did those using the baseline SAT level 1 (system’s action) HMI. The low-urgency scenario was associated with higher trust and reliance, and the drivers’ visual attention and NDRT performance improved after viewing the HMI, but not statistically significantly. The findings verified the positive role of the SAT model regarding human trust in the ADS, especially in regards to projection information in time-sensitive situations, and these results have implications for the design of automotive HMIs based on the SAT model to facilitate the human–ADS relationship.

Safety evaluation and prediction of takeover performance in automated driving considering drivers’ cognitive load: A driving simulator study

Automated vehicles alleviate the need for driver attention and control. However, a takeover request (TOR) to the driver remains essential for emergencies and safety–critical scenarios beyond automation's capability. Thus, accessing drivers’ safety performance in various TOR scenarios is crucial for conditionally automated driving (SAE L3). However, TOR safety performance is seldom examined concerning drivers' cognitive load, despite its presumed relevance to TOR scenarios and human–machine interaction. Moreover, the adequacy of the time window preceding a TOR, critical for TOR safety prediction, remains inadequately explored. This study aims to assess safety performance across diverse TOR scenarios in Level 3 conditional automation, incorporating drivers’ cognitive load, and predict TOR safety by considering the time window's impact. A driving simulator experiment gathered eye movement and driving behavior data from 37 recruited participants. Participants were instructed to take control of the vehicle from automated driving within a time budget (TB) of 3 s or 7 s in obstacle avoidance (OA) or lane keeping (LK) scenarios while engaging in non-driving-related tasks (NDRTs). Participants’ subjective cognitive load in the different TOR scenarios was scaled using NASA-TLX. Furthermore, safe TOR performance was predicted utilizing Convolutional Neural Networks (CNNs) across different time window sizes preceding a TOR. The results indicate that: 1) cognitive loads in takeover scenarios ranked from highest to lowest as TB = 3s_OA, TB = 3s_LK, TB = 7s_OA, TB = 7s_LK; and the cognitive loads of the NDRTs ranked from highest to lowest as mistake finding, texting, chatting, monitoring; 2) the takeover safety performance in the four scenarios from lowest to highest was TB = 3s_OA, TB = 3s_LK, TB = 7s_OA, TB = 7s_LK; likewise, the takeover safety performance during the four NDRTs ranked from lowest to highest as mistake finding, monitoring, texting, chatting; 3) the time window size before the TORs significantly affected the prediction performance of the model. A 30-second window was recommended as optimal for predicting takeover safety using the CNN model, achieving an average F1 score of 0.8120 and 81.98 % accuracy. This study's findings enhance our comprehension of driving behavior characteristics during TOR and offer valuable insights for detecting driver states in conditionally automated driving contexts.