Take-over Request Research Articles

Supporting Driver Attention Toward Potential Hazards During Takeover: A Preliminary Result

This study investigates the impact of a supportive system on takeover transitions in conditionally automated driving (SAE Level 3). The supportive system is designed to direct drivers’ attention toward potential hazards in the environment when a takeover request occurs. The study comprises two components: (a) identifying various types of potential hazards using naturalistic driving data, and (b) conducting a driving simulator study to develop and assess a gaze guidance system based on the N-SEEV model of visual attention. Results indicate that drivers using a highly salient attention guidance system were less likely to collide with a secondary hazard during takeover transitions. This suggests that gaze guidance support is an effective approach for assisting drivers during takeover transitions.

How can level of trust in conditional driving automation vehicles be improved? Road condition feedback strategies and the mediating role of anxiety

ABSTRACT Conditional driving automation (CDA, Level 3) vehicles will become widespread in the next few years. The driver only needs to take over the vehicle in an emergency and pay attention to the feedback from the vehicle. However, the mechanism by which feedback strategies and drivers' emotional factors influence drivers' levels of trust in automation is not clear. This study compares the effect of three road condition feedback strategies on drivers' trust in automation: no feedback, concentrated feedback (multiple feedback messages are presented together before the takeover request is issued) and real-time feedback (the road condition messages are fed in real time based on changes in the external environment). The mediating effect of anxiety in the relationship is also examined by a within-subjects factorial simulated driving experiment (N = 41). The anxiety score was significantly lower in the real-time feedback group than in the other two groups, but the level of trust in automation was significantly higher. Furthermore, an analysis of the mediating effect showed that anxiety plays a fully mediating role in the relationship between real-time feedback and the level of trust in automation. This finding will help professionals design more trustworthy feedback strategies from a psychological perspective.

Comparing Static and Dynamic Vibration Cues in Wristband Haptic Feedback for Enhanced Driver Response in Automated Vehicles

Higher levels of driving automation make effective takeover requests critical. The wrist’s sensitivity to vibration makes wristband devices a potential carrier for sending these requests. However, the impacts of conveying takeover requests through directional vibrotactile patterns such as dynamic patterns (sequential stimuli occurring at different locations on the wrist) and static patterns (fixed stimuli at the same locations on the wrist) are unclear. Therefore, this study examined the effects of directional vibrotactile patterns on takeover performance among younger and older adults. Participants responded to four patterns (two dynamic, one static, and one baseline) in a simulated SAE Level 3 automated vehicle. Takeover performance was evaluated using reaction time and takeover time. The results show that the static and baseline patterns had shorter reaction and takeover times compared to the dynamic patterns. In addition, younger adults react faster to takeover requests compared to older adults. Findings provide important insights for the future design of human-machine interfaces via wristband devices for automated vehicles.

Effects of integrated takeover request warning with personal portable device on takeover time and post-takeover performance in level 3 driving automation

Introduction: Level 3 driving automation defined by the Society of Automotive Engineers (SAE) requests human drivers to drive manually when the vehicle cannot perform the driving task. In this regard, researchers have studied the integrated takeover request (TOR) which provides visual and auditory TOR warning in both vehicle interface (e.g., dashboard, windshield (head-up display) and personal portable device (PPD) (e.g., cell phone, tablet). However, these studies neither used auditory TOR warning in PPD nor examined the effect of use of headphone on takeover. Thus, this study evaluates the effects of the integrated TOR with the use of headphones on the takeover time and the post-takeover performance. Method: The behavior of 60 drivers was observed in the driving simulator experiment. During the experiment, the drivers watched a video on a tablet in automated driving, received the TOR warning, and manually drove in the lane change and pullover scenarios. The survey was also conducted to ask drivers’ experience and preference for TOR warning. Results: The integrated TOR significantly reduced the takeover time compared to the conventional TOR which provides the TOR warning in vehicle interface only. The integrated TOR also improved the post-takeover performance as indicated by more stable steering operation and safer driving behavior after TOR warning. However, the use of headphones did not significantly reduce the takeover time or improve the post-takeover performance for the integrated TOR. The participants generally perceived higher subjective comfort and safety level with the integrated TOR than the conventional TOR. Conclusions: The integrated TOR with auditory warning in PPD can significantly reduce the takeover time and improve the post-takeover performance in both urgent and less urgent conditions. Practical applications: The integrated TOR with auditory warning in PPD can be applied to SAE Level 3 driving automation for safe transition from automated to manual driving.

Where's my TOR?: Evaluating the Effect of Take-Over Request Source on Older Drivers' Control Transition in Level 3 Cars

It is challenging for older drivers to transition to manual control after a Take-Over Request (TOR) has been issued by a Level 3 car. This study investigated if the presentation source of the TOR affects driver performance when resuming control. We measured take-over performance, hazard perception, and user acceptance when the TOR was presented on (1) a smartphone displaying a Non-Driving Related Task (NDRT) simultaneously with the In-Vehicle Information System (IVIS), or (2) presented on the IVIS only. Two NDRTs that varied in cognitive demand were tested with older drivers aged 60-69 and 70+. For the lower cognitive demand NDRT, presenting the TOR on the smartphone+IVIS improved takeover performance, hazard perception, and user acceptance, with greater benefits observed in the 70+ group. For the cognitively demanding NDRT, the smartphone+IVIS presentation did not benefit either group of drivers. TOR designers can apply these findings to enhance TORs and assist older drivers in managing control transitions considering the NDRT cognitive demand.

Enhancing Safety in Autonomous Vehicles: The Impact of Auditory and Visual Warning Signals on Driver Behavior and Situational Awareness

Semi-autonomous vehicles (AVs) enable drivers to engage in non-driving tasks but require them to be ready to take control during critical situations. This “out-of-the-loop” problem demands a quick transition to active information processing, raising safety concerns and anxiety. Multimodal signals in AVs aim to deliver take-over requests and facilitate driver–vehicle cooperation. However, the effectiveness of auditory, visual, or combined signals in improving situational awareness and reaction time for safe maneuvering remains unclear. This study investigates how signal modalities affect drivers’ behavior using virtual reality (VR). We measured drivers’ reaction times from signal onset to take-over response and gaze dwell time for situational awareness across twelve critical events. Furthermore, we assessed self-reported anxiety and trust levels using the Autonomous Vehicle Acceptance Model questionnaire. The results showed that visual signals significantly reduced reaction times, whereas auditory signals did not. Additionally, any warning signal, together with seeing driving hazards, increased successful maneuvering. The analysis of gaze dwell time on driving hazards revealed that audio and visual signals improved situational awareness. Lastly, warning signals reduced anxiety and increased trust. These results highlight the distinct effectiveness of signal modalities in improving driver reaction times, situational awareness, and perceived safety, mitigating the “out-of-the-loop” problem and fostering human–vehicle cooperation.

The Impact of Meaningful Vibrotactile Displays on User Preferences Across Age Groups in Automated Driving

Automated vehicles may enhance the quality of life by empowering individuals and promoting independent mobility, especially for older adults. However, takeovers are still required occasionally. To ensure a safe transition during the takeover, meaningful tactile displays as takeover requests could be a good option to improve takeover efficiency. Previous studies have examined the effects of tactile takeover request more from an objective perspective, however, subjective evaluations that could reach larger and more diverse sample sizes have not been extensively conducted yet. Therefore, the goal of this study was to investigate drivers of different age groups (younger and older adults) on their preference of vibrotactile displays, varied in information formats (instructional and informative) and locations (seat back and seat pan), on a large population scale ( n = 353) through a national survey. The results indicated a preference for informative patterns over instructional patterns and displays on the seat back over the seat pan. Also, a higher level of perceived urgency was noted in older participants (age ≥ 65) compared with younger participants. These findings may inform the design of next-generation tactile interfaces in automated systems.

Effect of Takeover Request Time and Warning Modality on Trust in L3 Automated Driving.

This study investigated the effects of four takeover request (TOR) times and seven warning modalities on performance and trust in automated driving on a mildly congested urban road scenario, as well as the relationship between takeover performance and trust. Takeover is crucial in L3 automated driving, where human-machine codriving is employed. Establishing trust in takeover scenarios among drivers can enhance the acceptance of autonomous vehicles, thereby promoting their widespread adoption. Using a driving simulator, data from 28 participants, including collision counts, takeover time (ToT), electrodermal activity (EDA) data, and self-reported trust scores, were collected and analyzed primarily using Generalized Linear Mixed Models (GLMM). Collisions during the takeover undermined participants' trust in the autonomous driving system. As TOR time increased, participants' trust improved, and the longer TOR time did not lead to participant confusion. There was no significant relationship between warning modality and trust. Furthermore, the combination of three warning modalities did not exhibit a notable advantage over the combination of two modalities. The study examined the effects of TOR time and warning modality on trust, as well as preliminarily explored the potential association between takeover performance, including collisions and ToT, and trust in autonomous driving takeovers. Researchers and designers of automotive interactions were given referenceable TOR time and warning modality by this study, which extended the autonomous driving takeover scenarios. These findings contributed to boosting drivers' confidence in transferring control to the automated system.

Who is performing the driving tasks after interventions? Investigating drivers' understanding of mode transition logic in automated vehicles

Mode awareness is important for the safe use of automated vehicles, yet drivers' understanding of mode transitions has not been sufficiently investigated. In this study, we administered an online survey to 838 respondents to examine their understanding of control responsibilities in partial and conditional driving automation with four types of interventions (brake pedal, steering wheel, gas pedal, and take-over request). Results show that most drivers understand that they are responsible for speed and distance control after brake pedal interventions and steering control after steering wheel interventions. However, drivers have mixed responses regarding the responsibility for speed and distance control after steering wheel interventions and the responsibility for steering control after gas pedal interventions. With a higher automation level (conditional driving automation), drivers expect automation to remain responsible more often compared to a lower automation level (partial driving automation). Regarding Hands-on requirements, more than 99% of respondents answered that drivers would keep their hands on the steering wheel after all intervention types in partial automation, while 60–95% would place their hands on the wheel after various intervention types in conditional automation. A misalignment between actual logic and drivers' expectations regarding control responsibilities is observed by comparing survey responses to the mode transition logic of commercial partially automated vehicles. To resolve confusion about control responsibilities and ensure consistent expectations, we propose implementing a consistent mode design and providing enhanced information to drivers.

Drivers’ situational awareness of surrounding vehicles during takeovers: Evidence from a driving simulator study

This study aimed to understand the influence of surrounding vehicle configuration, driving lane, and traffic density on drivers’ situational awareness (SA), takeover performance, and eye-tracking behaviors in conditionally automated driving. An experiment was conducted with the participation of 40 university students using a fixed-base driving simulator configured to simulate SAE Level 3 automation. During the experiment, participants were engaged in playing Tetris on a tablet as a non-driving related task in automated driving mode. Upon hearing an auditory takeover request, participants were instructed to take control of the vehicle, and then complete a scene reconstruction task to report their SA after transferring control back to the automated driving system. Our findings showed that drivers often neglected vehicles at their sides and rear during the takeover, which was associated with higher collision risks. Higher oncoming traffic density led to drivers’ worse SA of surrounding vehicles but more cautious driving behavior. Driving in the right lane generally resulted in smoother takeovers with lower collision risks. Interestingly, while SA did moderate the impacts of driving conditions on safety margins, a higher level of SA did not consistently relate to improved performance, especially in complex scenarios. This suggests the need for support systems that guide drivers to focus on safety–critical objects rather than simply amplifying SA in general. These insights have significant implications for the design of driver monitoring and support systems in automated vehicles.

Effects of various in-vehicle human–machine interfaces on drivers’ takeover performance and gaze pattern in conditionally automated vehicles

With the era of automated driving approaching, designing an effective and suitable human–machine interface (HMI) to present takeover requests (TORs) is critical to ensure driving safety. The present study conducted a simulated driving experiment to explore the effects of three HMIs (instrument panel, head-up display [HUD], and peripheral HMI) on takeover performance, simultaneously considering the TOR type (informative and generic TORs). Drivers’ eye movement data were also collected to investigate how drivers distribute their attention between the HMI and surrounding environment during the takeover process. The results showed that using the peripheral HMI to present TORs can shorten takeover time, and drivers rated this HMI as more useful and satisfactory than conventional HMIs (instrument panel and HUD). Eye movement analysis revealed that the peripheral HMI encourages drivers to spend more time gazing at the road ahead and less time gazing at the TOR information than the instrument panel and HUD, indicating a better gaze pattern for traffic safety. The HUD seemed to have a risk of capturing drivers’ attention, which resulted in an ‘attention tunnel,’ compared to the instrument panel. In addition, informative TORs were associated with better takeover performance and prompted drivers to spend less time gazing at rear-view mirrors than generic TORs. The findings of the present study can provide insights into the design and implementation of in-vehicle HMIs to improve the driving safety of automated vehicles.

Machine Learning for Prediction of Driver Takeover Time in Automated Driving: Insights from Non-urgent Low Consequence Scenarios

This work investigates driver takeover times in non-urgent, low consequence scenarios within conditionally automated driving. Using physiological and behavioral data from 46 participants in a driving simulator, classification algorithms were applied to predict metrics of takeover time following a takeover request (TOR). Eye-tracking, heart rate variability, and computer-vision based body posture features were analyzed for their predictive power. The Naïve Bayes algorithm outperformed other models, achieving an accuracy of 78% when predicting the time to first gaze in the driving scene following a TOR. Results from feature selection showed eye-tracking metrics to have the most predictive power. These results suggest that eye-tracking metrics and simple, computationally efficient, 2-class algorithms may be sufficient for predicting takeover time in non-urgent, low-consequence scenarios. This research provides evidence for integration of physiological sensing into adaptive automated driving systems (ADS) to develop context-aware TOR alert systems to improve road safety.

Differences in driver takeover performance and physiological responses in conditionally automated driving: Links to emotional instability

When the automated driving system reaches its technical limit, drivers will have to take over control from the vehicle within a limited time. The drivers’ emotional instability may affect whether they can safely control the vehicle during takeover transitions. Accordingly, this paper explores the association between emotional instability and physiological responses and driver takeover performance. 42 drivers engaged in negative emotion induced by movie clips were involved in a sequence of takeover events in a driving simulator. Personality questionnaires were used to assess the driver’s level of emotional instability. Their physiological data before the takeover request (TOR) and operational behavior data fragments for the takeover process were extracted. The results demonstrated that integration of subjective evaluations with electroencephalogram (EEG) and electrocardiogram (ECG) signals allows for more objective measurement of drivers’ level of emotional instability. The statistical analyses indicated that emotional instability has a significant effect on drivers’ heartbeat interval, heart rate, the frequency bands of θ(4–8 Hz), α(8–12 Hz), β(13–30 Hz), γ (30–50 Hz) and lateral takeover performance. Those findings of differences in driver physiological responses and takeover performance under emotional instability may provide additional support for the design of driver state monitoring and adaptive warning systems.

Enhancing safety in conditionally automated driving: Can more takeover request visual information make a difference in hazard scenarios with varied hazard visibility?

Autonomous driving technology has the potential to significantly reduce the number of traffic accidents. However, before achieving full automation, drivers still need to take control of the vehicle in complex and diverse scenarios that the autonomous driving system cannot handle. Therefore, appropriate takeover request (TOR) designs are necessary to enhance takeover performance and driving safety. This study focuses on takeover tasks in hazard scenarios with varied hazard visibility, which can be categorized as overt hazards and covert hazards. Through ergonomic experiments, the impact of TOR interface visual information, including takeover warning, hazard direction, and time to collision, on takeover performance is investigated, and specific analyses are conducted using eye-tracking data. The following conclusions are drawn from the experiments: (1) The visibility of hazards significantly affects takeover performance. (2) Providing more TOR visual information in hazards with different visibility has varying effects on drivers’ visual attention allocation but can improve takeover performance. (3) More TOR visual information helps reduce takeover workload and increase human–machine trust. Based on these findings, this paper proposes the following TOR visual interface design strategies: (1) In overt hazard scenarios, only takeover warning is necessary, as additional visual information may distract drivers’ attention. (2) In covert hazard scenarios, the TOR visual interface should better assist drivers in understanding the current hazard situation by providing information on hazard direction and time to collision to enhance takeover performance.

How to design driver takeover request in real-world scenarios: A systematic review

Conditional automated driving systems have become a reality. Ensuring the safety during the driving takeover process has become a crucial concern, with the takeover request (TOR) playing an important role. Existing review studies often focus only on a particular aspects of the TOR and fail to consider the varied requirements of TOR in specific scenarios. In this review, we conducted a systematic literature search and identified 75 studies related to TOR design. Through the summarization of literature, we naturally arrived at the final research themes, which encompassed the key aspects of TOR design and specific situational factors. Our findings indicate the necessity for dynamic time budget (TB) in automated driving systems, emphasizing that longer TB contributes to improved safety. There is a need for multimodal capabilities within TOR, followed by tailored modal selection and combinations based on specific takeover scenarios. The integration of additional informative prompt in TOR is beneficial, provided that the type and content are carefully considered. Our study provides a comprehensive theoretical framework for future TOR design and suggests directions for future research efforts.

How various urgencies and visibilities influence drivers’ takeover performance in critical car-following conditions? A driving simulation study

Drivers of Level 3 automated vehicles are relieved from the driving task in specific circumstances but are required to take over control once the takeover request is prompted. Previous studies have investigated drivers’ takeover performance in non-critical car-following. However, little is known about drivers’ takeover behaviors in critical car-following, especially in low-visibility weather, which remarkably increases the risk of car-following. A driving simulator experiment with a 2 × 3 × 3 factor within-group design was conducted. The design matrix contained two weather conditions (clear and foggy), three car-following time headways (2 s, 3 s, 4 s) and three deceleration rates of the lead vehicle (LV) (0, 2 m/s2, 4 m/s2). A total of 30 participants completed the experiment. The results showed that in critical car-following situations, drivers faced greater challenges in negotiating with adjacent vehicles rather than the LV itself. Urgency and visibility did not significantly impact the likelihood of a crash with the LV due to drivers’ adoption of stronger braking. However, decreased visibility and higher LV’s deceleration increased the crash rate when drivers attempted lane changes. As urgency increased, drivers tended to change lanes earlier, leading to higher lane-changing risks and compromised lateral stability. This study can provide some insights for the car-following strategies of automated driving vehicles and the design of dedicated takeover schemes in various transportation environments.

How do visual and cognitive non-driving related tasks affect drivers’ visual attention and takeover performance in conditionally automated driving?

This study examines changes in visual attention during drivers’ engagement in visual or cognitive NDRTs during conditionally automated driving, and determines how these changes affect takeover performance. Seventy-five participants took part in a driving simulator study, performing three pre-takeover tasks: the three pre-takeover tasks: an auditory-imagery-verbal task (cognitive NDRT); a video-watching task (visual NDRT); and a monitoring task (baseline/non NDRT). Also, there were two hazardous events (breakdown or sudden merging of the vehicle ahead) leading to takeover requests issued with 7-s or 5-s lead times. The results revealed that NDRTs negatively affected visual attention, which caused lower saccade frequency between different areas of interest and shorter saccade amplitude. Drivers paid more visual attention to the road ahead and in-vehicle information system when performing cognitive and visual NDRTs, respectively. The visual attention of drivers performing NDRTs negatively affected takeover performance (e,g longer reaction time, heavier maximal brake pedal input, etc.). The reduction in takeover request lead time impaired takeover performance. The findings will support the design of eye tracker-based “out-of-loop” discrimination techniques and human-machine interaction interfaces in automated vehicles. This study contributes to the literature by examining how different types of NDRTs affect takeover performance, specifically focusing on drivers’ visual attention.

Analysis of drivers’ take-over ability improvement and behavioral steady state in human–machine codriving vehicles

During human–machine codriving, drivers need to take over the vehicle when a take-over request (TOR) appears. If drivers have not received relevant training before driving, they may be unable to complete the take-over within the limited time, or the stability of subsequent vehicle control may be insufficient, which can lead to accidents. In this study, two types of take-over ability improvement methods are proposed. Participants were recruited and randomly divided into a control group (n = 15, no take-over training) and two experimental groups (n = 15, text-based training; n = 15, behavioral spectrum-based training). One-way ANOVA or the Kruskal–Wallis test and post hoc contrasts were used to analyze the differences in data indicators between the three groups of drivers after 20 take-over operations, and another method was proposed to validate the efficiency of the take-over operations on the stability of take-over ability. The results show that compared with the control group, both experimental groups demonstrated a significant improvement in take-over ability, with the behavioral spectrum-based training group exhibiting better take-over performance than the text-based training group. Moreover, after 14 take-over operations, drivers’ take-over ability in the behavioral spectrum-based training group stabilized. The findings of this study can contribute to the safety of human–machine codriving vehicles and the design of future driver training systems.

Willingness to take responsibility: Self-sacrifice versus sacrificing others in takeover decisions during autonomous driving

In Level-3 autonomous driving, drivers are required to take over in an emergency upon receiving a request from an autonomous vehicle (AV). However, before the deadline for the takeover request expires, drivers are not considered fully responsible for the accident, which may make them hesitant to assume control and take on full liability before the time runs out. Therefore, to prevent problems caused by late takeover, it is important to know which factors influence a driver's willingness to take over in an emergency. To address this issue, we recruited 250 participants each for both video-based and text-based surveys to investigate the takeover decision in a dilemmatic situation that can endanger the driver, with the AV either sacrificing a group of pedestrians or the driver if the participants do not intervene. The results showed that 88.2% of respondents chose to take over when the AV intended to sacrifice the driver, while only 59.4% wanted to take over when the pedestrians would be sacrificed. Additionally, when the AV's chosen path matched the participant's intention, 77.4% chose to take over when the car intended to sacrifice the driver compared with only 34.3% when the pedestrians would be sacrificed. Furthermore, other factors such as sex, driving experience, and driving preferences partially influenced takeover decisions; however, they had a smaller effect than the situational context. Overall, our findings show that regardless of the driving intention of an AV, informing drivers that their safety is at risk can enhance their willingness to take over control of an AV in critical situations.

Safety evaluation and prediction of takeover performance in automated driving considering drivers’ cognitive load: A driving simulator study

Automated vehicles alleviate the need for driver attention and control. However, a takeover request (TOR) to the driver remains essential for emergencies and safety–critical scenarios beyond automation's capability. Thus, accessing drivers’ safety performance in various TOR scenarios is crucial for conditionally automated driving (SAE L3). However, TOR safety performance is seldom examined concerning drivers' cognitive load, despite its presumed relevance to TOR scenarios and human–machine interaction. Moreover, the adequacy of the time window preceding a TOR, critical for TOR safety prediction, remains inadequately explored. This study aims to assess safety performance across diverse TOR scenarios in Level 3 conditional automation, incorporating drivers’ cognitive load, and predict TOR safety by considering the time window's impact. A driving simulator experiment gathered eye movement and driving behavior data from 37 recruited participants. Participants were instructed to take control of the vehicle from automated driving within a time budget (TB) of 3 s or 7 s in obstacle avoidance (OA) or lane keeping (LK) scenarios while engaging in non-driving-related tasks (NDRTs). Participants’ subjective cognitive load in the different TOR scenarios was scaled using NASA-TLX. Furthermore, safe TOR performance was predicted utilizing Convolutional Neural Networks (CNNs) across different time window sizes preceding a TOR. The results indicate that: 1) cognitive loads in takeover scenarios ranked from highest to lowest as TB = 3s_OA, TB = 3s_LK, TB = 7s_OA, TB = 7s_LK; and the cognitive loads of the NDRTs ranked from highest to lowest as mistake finding, texting, chatting, monitoring; 2) the takeover safety performance in the four scenarios from lowest to highest was TB = 3s_OA, TB = 3s_LK, TB = 7s_OA, TB = 7s_LK; likewise, the takeover safety performance during the four NDRTs ranked from lowest to highest as mistake finding, monitoring, texting, chatting; 3) the time window size before the TORs significantly affected the prediction performance of the model. A 30-second window was recommended as optimal for predicting takeover safety using the CNN model, achieving an average F1 score of 0.8120 and 81.98 % accuracy. This study's findings enhance our comprehension of driving behavior characteristics during TOR and offer valuable insights for detecting driver states in conditionally automated driving contexts.