Driver Takeover Performance Research Articles

Differences in driver takeover performance and physiological responses in conditionally automated driving: Links to emotional instability

When the automated driving system reaches its technical limit, drivers will have to take over control from the vehicle within a limited time. The drivers’ emotional instability may affect whether they can safely control the vehicle during takeover transitions. Accordingly, this paper explores the association between emotional instability and physiological responses and driver takeover performance. 42 drivers engaged in negative emotion induced by movie clips were involved in a sequence of takeover events in a driving simulator. Personality questionnaires were used to assess the driver’s level of emotional instability. Their physiological data before the takeover request (TOR) and operational behavior data fragments for the takeover process were extracted. The results demonstrated that integration of subjective evaluations with electroencephalogram (EEG) and electrocardiogram (ECG) signals allows for more objective measurement of drivers’ level of emotional instability. The statistical analyses indicated that emotional instability has a significant effect on drivers’ heartbeat interval, heart rate, the frequency bands of θ(4–8 Hz), α(8–12 Hz), β(13–30 Hz), γ (30–50 Hz) and lateral takeover performance. Those findings of differences in driver physiological responses and takeover performance under emotional instability may provide additional support for the design of driver state monitoring and adaptive warning systems.

Getting back in the loop: Does autonomous driving duration affect driver's takeover performance?

The level 3 autonomous driving function allows the driver to perform non-driving-related tasks such as watching movies or reading while the system manages the driving task. However, when a difficult situation arises, the driver is requested to return to the loop of control. This switching from driver to passenger then back to driver may modify the driving paradigm, potentially causing an out-of-the-loop state. We tested the hypothesis of a linear (progressive) impact of various autonomous driving durations: the longer the level 3 autonomous function is used, the poorer the driver's takeover performance. Fifty-two participants were divided into 4 groups, each group being assigned a specific period of autonomous driving (5, 15, 45, or 60 min), followed by a takeover request with a time budget of 8.3 s. Takeover performance was assessed over two successive drives via reaction times and manual driving metrics (trajectories). The initial hypothesis (linearity) was not confirmed: there was a nonlinear relationship between autonomous driving duration and takeover performance, with one duration (15 min) appearing safer overall and mixed performance within groups. Repetition induced a major change in performance during the second drive, indicating rapid adaptation to the situation. The non-driving-related task appears critical in several respects (dynamics, content, driver interest) to proper use of level 3 automation. All this supports previous research prompting reservations about the prospect of car driving becoming like train travel.

Effects of Traffic Conditions on Driver Takeover Performance and Situational Awareness

This study aimed to investigate the impacts of oncoming traffic density, surrounding vehicle configuration, and driving lane on drivers' takeover performance and situational awareness in conditionally automated driving. We conducted a driving simulator experiment with 28 participants. Our results showed that high oncoming traffic led to shorter takeover response time and lower situational awareness. Drivers tended to brake more aggressively and had lower collision risks when the traffic was on the front side, and drivers were more likely to ignore traffic from behind and behind side when taking over. Taking over in the right lane on a two-way road led to lower collision risks and smoother driving compared to the left lane. Our results have valuable implications for the design and development of driver monitoring and support systems to improve the safety of conditionally automated vehicles.

Measuring takeover performance in different driving scenarios: Considering the influence of non-driving-related tasks and takeover request lead time

Conditional automation systems (SAE Level 3) allow the driver to perform a secondary task, but the driver must regain control of the vehicle after the takeover request (ToR). Therefore, it is important to understand the factors that influence the driver’s takeover performance. Previous studies have analyzed the variables including ToR modalities, ToR lead time, non-driving-related (NDR) task, driving experience and age. The influence of driving scenario on driver takeover performance and whether this influence varies the effect of the ToR time and NDR task remained unanswered. This study designed a 2(ToR time) × 2(NDR task) × 4 (driving scenario) factorial measure to investigate the effect of these factors on automated vehicle takeover performance. Forty-two participants drove in a simulated SAE Level 3 vehicle and performed a series of takeover tasks. The results showed that the ToR time, NDR task, driving scenario and ToR time × driving scenario had a significant effect on takeover behavior. In particular, the secondary work state task was correlated with the increased takeover and control time. The complexity of the driving scenario indicated a marginally significant influence on the quality of the takeover. A shorter takeover time and a smaller lateral offset were associated with a lower urgency of the driving scenario. The ToR time for the fog-cluster scenario showed an opposite effect to the takeover performance compared to the other three scenarios. Findings from this study can provide implications on the effect of ToR time desired and enabled by different scenarios.

The Effect of Multifactor Interaction on the Quality of Human–Machine Co-Driving Vehicle Take-Over

This paper investigates the effects of non-driving related tasks, take-over request time, and take-over mode interactions on take-over performance in human–machine cooperative driving in a highway environment. Based on the driving simulation platform, a human–machine collaborative driving simulation experiment was designed with various take-over quality influencing factors. The non-driving related tasks included no task, listening to the radio, watching videos, playing games, and listening to the radio and playing games; the take-over request time was set to 6, 5, 4, and 3 s, and the take-over methods include passive and active take-over. Take-over test data were collected from 65 drivers. The results showed that different take-over request times had significant effects on driver take-over performance and vehicle take-over steady state (p < 0.05). Driver reaction time and minimum TTC decreased with decreasing take-over request time, maximum synthetic acceleration increased with decreasing take-over request time, accident rate increased significantly at 3 s take-over request time, and take-over safety was basically ensured at 4 s request time. Different non-driving related tasks have a significant effect on driver take-over performance (p < 0.05). Compared with no task, non-driving related tasks significantly increase driver reaction time, but they only have a small effect on vehicle take-over steady state. Vehicle take-over mode has a significant effect on human–machine cooperative driving take-over quality; compared with passive take-over mode, the take-over quality under active take-over mode is significantly lower.

Driver Engagement Strategies for L3 ADS: A Review of the Literature

The completion of the driving task by Level 3 (L3) Automated Driving Systems (ADS) allows drivers to turn their attention away from the road and direct it towards other activities; however, the driver must remain receptive to requests to intervene (RTI) and be prepared to complete the driving task upon deactivation of the driving automation. This transfer of control is called a takeover. The objectives of this research activity were to understand the human factors underlying driver disengagement while using L3 ADS and to identify factors influencing takeover quality and performance when the ADS-equipped vehicle has exited its operational design domain. A structured, comprehensive literature review approach was used to synthesize findings in the following areas: common measures used to assess takeover performance, factors that affect takeovers, and strategies that improve driver takeover performance.

Modeling the effects of perceived intuitiveness and urgency of various auditory warnings on driver takeover performance in automated vehicles

Existing driver models mainly account for drivers’ responses to visual cues in manually controlled vehicles. The present study is one of the few attempts to model drivers’ responses to auditory cues in automated vehicles. It developed a mathematical model to quantify the effects of characteristics of auditory cues on drivers’ response to takeover requests in automated vehicles. The current study enhanced queuing network-model human processor (QN-MHP) by modeling the effects of different auditory warnings, including speech, spearcon, and earcon. Different levels of intuitiveness and urgency of each sound were used to estimate the psychological parameters, such as perceived trust and urgency. The model predictions of takeover time were validated via an experimental study using driving simulation with resultant R squares of 0.925 and root-mean-square-error of 73 ms. The developed mathematical model can contribute to modeling the effects of auditory cues and providing design guidelines for standard takeover request warnings for automated vehicles.

Objective assessment of trait attentional control predicts driver response to emergency failures of vehicular automation

With the advent of autonomous driving, the issue of human intervention during safety-critical events is an urgent topic of research. Supervisory monitoring, taking over vehicle control during automation failures and then bringing the vehicle to safety under time pressure are cognitively demanding tasks that pose varying difficulties across the driving population. This underpins a need to investigate individual differences (i.e., how people differ in their dispositional traits) in driver responses to automation system limits, so that autonomous vehicle design can be tailored to meet the safety-critical needs of higher-risk drivers. However, few studies thus far have examined individual differences, with self-report measures showing limited ability to predict driver takeover performance. To address this gap, the present study explored the utility of an established brain activity-based objective index of trait attentional control (frontal theta/beta ratio; TBR) in predicting driver interactions with conditional automation. Frontal TBR predicted drivers’ average takeover reaction time, as well as the likelihood of accident after takeover. Moving towards practical applications, this study also demonstrated the utility of streamlined estimates of frontal TBR measured from the forehead electrodes and from a single crown electrode, with the latter showing better fidelity and predictive value. Overall, TBR is behaviourally relevant, measurable with minimal sensors and easily computable, rendering it a promising candidate for practical and objective assessment of drivers’ neurocognitive traits that contribute to their AV driving readiness.

Speech-based takeover requests in conditionally automated driving: Effects of different voices on the driver takeover performance

This study explored the possibility of applying personalized takeover requests (TORs) in an automated driving system (ADS), which required drivers to regain control when the system reached its limits. A driving simulator experiment was conducted to investigate how speech-based TOR voices impacted driver performance in takeover scenarios with two lead time conditions in conditionally automated driving (level 3). Eighteen participants drove in three sessions, with each session having a different TOR voice (a synthesized male voice, a synthesized female voice, and a significant other voice). Two scenarios with a lead time of 5 s and two scenarios with a lead time of 12 s were provided per session. The driver takeover time and quality data were collected. A follow-up interview was conducted to gain a clearer understanding of the drivers’ psychological feelings about each TOR voice during takeovers. Changes in takeover time and takeover quality caused by TOR voices were similar in both lead time conditions, except for the lateral acceleration. The synthesized male voice led to a larger maximum lateral acceleration than the other two voices in the 5 s condition. Interestingly, most drivers preferred choosing the synthesized female voice for future takeovers and showed negative attitudes toward the significant other voice. Our results implied that choosing TOR voices should consider the drivers’ daily voice-usage habits as well as specific context of use, and personalized TOR voices should be incorporated into the ADS prudently.

PADA-AD Test - a standardized test method for assessing takeover performance of drivers during automated driving

The presented method describes a standardized test procedure for the evaluation of takeover performance of drivers during automated driving. It was primarily developed to be used for evaluating Level 3 systems (conditional automated driving). It should be applied in a driving simulator environment during the development phase of a system. The method consists of a test course on a three-lane highway with 12 test scenarios where the driver repeatedly has to take over control from the automated system. Each scenario requires the driver to build up an adequate situation awareness in order to take the decision for the correct action. The method was explicitly designed to map the four relevant steps in the takeover process of automated driving: Perception – Awareness – Decision – Action and is therefore called PADA-AD Test for automated driving. The method description contains guidelines with regard to the specification of the test course and the included test scenarios, the design and duration of the automated drives, the non-driving related task to be performed during the automated drives, the instructions to be given to the subjects and finally the measures for evaluating takeover performance of the drivers.• A test procedure for the evaluation of takeover performance of drivers during automated driving was developed for usage in a driving simulator during the development phase of a system/HMI•The test course enables the assessment of the driver's takeover performance in various test scenarios including higher cognitive processes•The method is highly standardized and thus replicable through use of a predetermined test course with clearly defined scenarios, reduced environmental conditions and "popping up” of situational elements.

Effects of non-driving related tasks on readiness to take over control in conditionally automated driving

Objective At conditionally automated driving, the driver can temporarily engage in non-driving related tasks (NDRTs). However, they must safely take over control when the automated driving system reaches its operation limit. Thus, understanding the effects of the NDRTs on driver take-over performance is essential. The present work investigates the effects of various NDRTs on motor readiness in take-over scenarios during conditionally automated driving. Methods Three driving simulator studies were conducted. 48, 49, and 22 participants were recruited in three experiments, respectively. The participants were distracted by different NDRTs (everyday task in Experiment 1, arrow task in Experiment 2, and SuRT in Experiment 3) on a tablet mounted in the vehicle. The everyday task included reading the news and watching a video, and the arrow task included a set of arrow matrices presented to the participants in sequence. The time budgets in Experiment 1 included 3 s, 4 s, and 5 s, and the time budgets in Experiment 2 and 3 included 5 s and 7 s. A take-over request (TOR) warning was issued in the automated driving condition when the participants encountered a broken-down car in front. The participants must regain control of the vehicle with the given time budget. The hands-on time was evaluated, measuring the time from the TOR until the hands touch the steering wheel. Results The task (arrow task and SuRT), time budget (5 s and 7 s), and gender did not affect the hands-on time. However, the hands-on time for the drivers with the everyday task was significantly shorter than that for the drivers with the arrow task in the 5 s time budget. Conclusions In conditionally automated driving, the arrow task and SuRT imposed a similar workload on readiness to take over control. Compared to the everyday task, the engagement in the arrow tasks consumed more workload on readiness to take over control.

Modeling the Effects of Perceived Intuitiveness and Urgency of Various Auditory Warnings on Driver Takeover Performance in Automated Vehicles

Modeling the Effects of Perceived Intuitiveness and Urgency of Various Auditory Warnings on Driver Takeover Performance in Automated Vehicles

Driver takeover performance in conditionally automated driving: sudden system failure situation versus ODD exit situation

Conditionally automated driving is expected to become available to the public in near future. However, the driver is expected to take over control from the system when a sudden system failure occur...

Predicting Driver Takeover Performance and Designing Alert Systems in Conditionally Automated Driving

Predicting Driver Takeover Performance and Designing Alert Systems in Conditionally Automated Driving

Predicting driver takeover performance in conditionally automated driving

In conditionally automated driving, drivers have difficulty taking over control when requested. To address this challenge, we aimed to predict drivers’ takeover performance before the issue of a takeover request (TOR) by analyzing drivers’ physiological data and external environment data. We used data sets from two human-in-the-loop experiments, wherein drivers engaged in non-driving-related tasks (NDRTs) were requested to take over control from automated driving in various situations. Drivers’ physiological data included heart rate indices, galvanic skin response indices, and eye-tracking metrics. Driving environment data included scenario type, traffic density, and TOR lead time. Drivers’ takeover performance was categorized as good or bad according to their driving behaviors during the transition period and was treated as the ground truth. Using six machine learning methods, we found that the random forest classifier performed the best and was able to predict drivers’ takeover performance when they were engaged in NDRTs with different levels of cognitive load. We recommended 3 s as the optimal time window to predict takeover performance using the random forest classifier, with an accuracy of 84.3% and an F1-score of 64.0%. Our findings have implications for the algorithm development of driver state detection and the design of adaptive in-vehicle alert systems in conditionally automated driving.

Promote or inhibit: An inverted U-shaped effect of workload on driver takeover performance

Objective In conditional automated driving (SAE Level 3), drivers are required to take over their vehicles when the automated systems fail. Non-driving related tasks (NDRTs) can positively or negatively affect takeover safety, but the underlying reasons for this inconsistency remain unclear. This study aims to investigate how various workload levels generated by NDRTs may influence the takeover performance of drivers and the lead time they require. Method Fifty drivers were randomly distributed into five groups, which corresponded to five workload levels (1-4 levels generated by Tetris game; control level generated by monitoring). Each driver completed vehicle takeover tasks upon receiving takeover requests with various lead times (3, 5, 7, 9, and 11 s) while engaging in NDRTs. The drivers’ takeover performance and subjective opinions were recorded. Results Drivers in the moderate workload condition (i.e., level 3) had significantly shorter takeover times and better takeover quality than those in the lower (i.e., level 1 and level 2) or higher (i.e., level 4) workload conditions. They also subjectively required less lead time in the moderate condition. Moreover, the drivers rated 7 s as the most appropriate lead time despite the improvement in their overall takeover performances with increased lead time. Conclusions This study found an inverted U-shaped relationship between the drivers’ workload generated by NDRTs and takeover performance. The moderate workload level (rather than the lower or higher workload level) led to a faster and better takeover performance, and it seemed to require minimal lead time for drivers. These findings help understand the relationship of drivers’ workload during the automation and takeover performance in conditional automated driving. An important recommendation emerging from this work is to investigate what should be the most efficient method to detect the drivers’ workload state real-time and give feedback to them when it comes to overload or underload during the automated driving.

Effects of Distraction in On-Road Level 2 Automated Driving: Impacts on Glance Behavior and Takeover Performance.

The paper aimed to investigate glance behaviors under different levels of distraction in automated driving (AD) and understand the impact of distraction levels on driver takeover performance. Driver distraction detrimentally affects takeover performance. Glance-based distraction measurement could be a promising method to remind drivers to maintain enough attentiveness before the takeover request in partially AD. Thirty-six participants were recruited to drive a Tesla Model S in manual and Autopilot modes on a test track while engaging in secondary tasks, including temperature-control, email-sorting, and music-selection, to impose low and high distractions. During the test drive, participants needed to quickly change the lane as if avoiding an immediate road hazard if they heard an unexpected takeover request (an auditory warning). Driver state and behavior over the test drive were recorded in real time by a driver monitoring system and several other sensors installed in the Tesla vehicle. The distribution of off-road glance duration was heavily skewed (with a long tail) by high distractions, with extreme glance duration more than 30 s. Moreover, being eyes-off-road before takeover could cause more delay in the urgent takeover reaction compared to being hands-off-wheel. The study measured off-road glance duration under different levels of distraction and demonstrated the impacts of being eyes-off-road and hands-off-wheel on the following takeover performance. The findings provide new insights about engagement in Level 2 AD and are useful for the design of driver monitoring technologies for distraction management.

Safety Compensation for Improving Driver Takeover Performance in Conditionally Automated Driving

The topic of transition from automated driving to manual maneuver in conditionally automated driving (SAE level-3) has acquired increasing interest. In such conditionally automated driving, drivers are expected to take over the vehicle control if the situation goes beyond the system’s functional limit of operation. However, it is challenging for drivers to resume control timely and perform well after being engaged in non-driving related tasks. Facing this challenge, this paper investigated a safety compensation in which the system conducts automatic deceleration to prolong the time budget for drivers to response. The purpose of the paper is to evaluate the effect of safety compensation on takeover performance in different takeover scenarios such as fog, route choosing, and lane closing. In the experiment, 16 participants were recruited. Results showed no significant effect of safety compensation on the takeover time, but a significant effect on the longitudinal driving performance (viz. driver brake input and the time to event). Moreover, it indicated a significant effect of safety compensation on the lateral acceleration in the lane closing scenario. This finding is useful for the automotive manufacturers to supply users a safer transition scheme from automated driving to manual maneuver.

Driver Takeover Performance and Monitoring Behavior with Driving Automation at System-Limit versus System-Malfunction Failures

Today’s vehicles are becoming highly automated, however, if the automation fails, drivers must take over control of the vehicle. Automation may fail as a result of known system limits (system-limit failure) or of malfunctions that are unforeseen by system designers (system-malfunction failure). The aim of this research was to quantify the differences between how these two failure types influence driver takeover performance and monitoring behaviors. In a simulator with SAE Level 2 driving automation, 18 drivers experienced both a system-limit and system-malfunction failure while engaging in a secondary task. Results show that drivers put their hands on the wheel 0.62 s sooner and took over 0.51 s faster for the system-limit failure compared with the system-malfunction failure. Eye tracking data revealed that the percent of time looking at the secondary task display was 12.7% lower and the percent of time looking at the roadway was 11.2% higher before the system-limit failure compared with before the system-malfunction failure. Given that takeover performance and monitoring behavior differ significantly based on failure type, a distinction should be made in the literature between system-limit and system-malfunction failures, and comparisons between previous studies using these failures should not be done without considering this distinction. Furthermore, as SAE Level 2 vehicles are currently available to consumers, efforts should be focused on supporting drivers’ mental models of automated systems, so that drivers are able to successfully predict system-limit failures.

Effects of request-to-intervene message contents on driver performance and safety under conditionally automated driving system

Abstract The current study addressed the issue of designing request-to-intervene (RtI) of a level 3 automated driving system. We hypothesized that providing the driver with step-by-step operating instructions and surrounding traffic environmental information in the RtI message could smooth driver takeover performance and enhance safety. To test this hypothesis, we conducted an experiment with a driving simulator. In the experiment, 40 participants were divided into four Human-Machine Interface (HMI) groups: Beeps HMI (Baseline), Operating Instruction HMI, Environmental HMI, and Combined HMI. The results revealed that both additional operating instructions and environmental information could lower the steering wheel reversal rate. The combination of operating instructions and surrounding traffic environmental information contributed to reducing the accident rate.