Take-over Request Research Articles

The effects of age and physical exercise on multimodal signal responses: Implications for semi-autonomous vehicle takeover requests

The present study examined whether the non-chronological age factor, engagement in physical exercise, affected responses to multimodal (combinations of visual, auditory, and/or tactile) signals differently between younger and older adults in complex environments. Forty-eight younger and older adults were divided into exercise and non-exercise groups, and rode in a simulated Level 3 autonomous vehicle under four different task conditions (baseline, video watching, headway estimation, and video-headway combination), while being asked to respond to various multimodal warning signals. Overall, bi- and trimodal warnings had faster response times for both age groups across driving conditions, but was more pronounced for older adults. Engagement in physical exercise was associated with smaller maximum braking force for younger participants only, and also corresponded to longer average fixation durations, compared to the non-exercise group. Findings from this research can help to guide decisions about the design of warning and information systems for semi-autonomous vehicles.

Effects of different takeover request interfaces on takeover behavior and performance during conditionally automated driving

Automated driving is a developing trend that is coming to the consumer market, and conditionally automated driving (CAD) is anticipated to become the primary automated driving system. For enhancing both the comfort and security of human drivers in self-driving cars, the most significant concern of CAD is ensuring that not only can the driver conduct non-driving related tasks (NDRT) while automated driving is in progress, but also quickly and competently take over when the system reaches a limit and issues a takeover request (TOR). However, the level of distraction by NDRTs may affect the transition from automated driving to the human driver taking over. The focus of the present study was allowing a driver immersed in NDRTs to discover the TOR and take control of the driving quickly. A 3×2×2 factor experimental design was used: vehicle display interface information load (basic vs. prediction vs. advanced prediction interfaces); TOR information load (directional vs. non-directional information notifications); and degree of NDRT immersion (not performing vs. performing an NDRT when TOR prompt was issued). 48 participants were recruited, and different automotive display interfaces were used as TOR prompts with different information loads during driving to analyze the takeover behavior, performance, and subjective perception of the drivers, who were immersed in a smartphone-related task. The takeover process out of NDRT immersion was found to be more efficient with the advanced prediction interface, compared to the other two interfaces. All groups achieved faster takeovers and demonstrated better takeover performance if given directional rather than non-directional information, regardless of interface type or NDRT immersion.

The Relationship Between Takeover Request Lead Time and Drivers’ Situation Awareness for Freeway Exiting in Conditionally Automated Driving

Conditionally automated vehicles require the out-of-the-loop driver to intervene when the system is unable to handle forthcoming situations, such as freeway exiting. The takeover request (ToR) for exiting a freeway can be scheduled in advance. Upon a ToR, the driver needs to gain situation awareness (SA) and resume manual control. This study examined how the ToR lead time affects driver SA for resuming control and when to send the ToR is most appropriate for freeway exiting. A web-based, supervised experiment was conducted with 31 participants. Each participant experienced 12 levels of ToR lead time (6, 8, 10, 12, 14, 16, 18, 20, 25, 30, 45, and 60 s). The results showed positive effects of longer ToR lead times (16–60 s) on driver SA for resuming control to exit from freeways in comparison to shorter ToR lead times (6–14 s), and the effects level off at 16–30 s.

Takeover Request Design in Automated Driving: A Systematic Review

Autonomous vehicles (AV) with “level 3” automation and above are expected to take full longitudinal and lateral control, which relinquishes the driver from manual control and allows for engagement with non-driving-related tasks. Despite the advance nature of a level 3 vehicle, system limitations can occur, and the driver is expected to re-engage in manual driving at a moment’s notice. Current literature has been focused on takeover performance during a takeover request (TOR) and the effects of multimodal warnings, but there is little consensus on how modality stimulus is presented. This systematic review summarizes the current designs and implementations of TORs of level 3 AVs and above. Identified themes in the review were categorized into three sections: non-driving-related tasks, driving scenarios, and takeover modality. A summary of how researchers utilized these themes in the current literature are discussed as well as implications and further research.

Automated Vehicle Takeover: A Pilot Study on the Effects of Age, Physical exercise, and Takeover Request Modality on Post-Takeover Performance

Automated Vehicle Takeover: A Pilot Study on the Effects of Age, Physical exercise, and Takeover Request Modality on Post-Takeover Performance

Take-Over Time: A Cross-Cultural Study of Take-Over Responses in Highly Automated Driving

The article presents a cross-cultural study of take-over performance in highly automated driving. As take-over performance is an important measure of safe driving, potential cultural differences could have important implications for the future development of automated vehicles. The study was conducted in two culturally different locations, Seattle, WA (n = 20) and Ljubljana, Slovenia (n = 18), using a driving simulator. While driving, participants voluntarily engaged in secondary tasks. The take-over request (TOR) was triggered at a specific time during the drive, and take-over time and type of response (none, brake, steer) were measured for each participant. Results show significant differences in take-over performance between the two locations. In Seattle 30% of participants in Seattle did not respond to TOR; the remaining 70% responded by braking only, compared to Slovenian participants who all responded by either braking or steering. Participants from Seattle responded significantly more slowly to TOR (M = +1285 ms) than Slovenian participants. Secondary task engagement at TOR also had an effect, with distracted US participants’ response taking significantly longer (M = +1596 ms) than Slovenian participants. Reported differences in take-over performance may indicate cultural differences in driving behavior and trust in automated driving.

Measuring drivers’ takeover performance in varying levels of automation: Considering the influence of cognitive secondary task

Mixed control by driver and automated system will remain in use for decades until fully automated driving is perfected. Thus, drivers must be able to accurately regain control of vehicles in a timely manner when the automated system sends a takeover request (TOR) at its limitation. Therefore, determining the factors that affect drivers’ takeover quality at varying levels of automated driving is important. Previous studies have shown that visually distracting secondary tasks impair drivers’ takeover performance and increase the subjective workload. However, the influence of purely cognitive distracting secondary tasks on drivers’ takeover performance and how this influence varies at different levels of automation are still unknown. Hence, a 5 (driving modes) × 3 (cognitive secondary tasks) factorial design with the within-subject factors was adopted for this driving simulator experiment. The sample consisted of 21 participants. The participants’ subjective workloads were recorded by the NASA-Task Load Index (NASA-TLX). Results showed that compared to manual driving conditions, the drivers’ subjective workloads were significantly reduced in both partially and highly automated driving conditions, even with a TOR, confirming the benefit of the automated driving system in terms of reducing the driving workload. Moreover, the drivers exhibited a lower takeover behavior quality at high levels of automation than manual driving in terms of increased reaction time, abnormal performance, standard deviation of lane position, lane departure probability, and reduced minimum of time to collision. However, at the highly automated driving condition, the drivers’ longitudinal driving safety and ability to follow instructions improved when performing a highly cognitive secondary task. This phenomenon possibly occurred because automated driving conditions lead to an underload phenomenon, and the execution of highly cognitive tasks transfers drivers into moderate load, which helps with the drivers’ takeover performance.

Effects of non-driving related tasks on readiness to take over control in conditionally automated driving

Objective At conditionally automated driving, the driver can temporarily engage in non-driving related tasks (NDRTs). However, they must safely take over control when the automated driving system reaches its operation limit. Thus, understanding the effects of the NDRTs on driver take-over performance is essential. The present work investigates the effects of various NDRTs on motor readiness in take-over scenarios during conditionally automated driving. Methods Three driving simulator studies were conducted. 48, 49, and 22 participants were recruited in three experiments, respectively. The participants were distracted by different NDRTs (everyday task in Experiment 1, arrow task in Experiment 2, and SuRT in Experiment 3) on a tablet mounted in the vehicle. The everyday task included reading the news and watching a video, and the arrow task included a set of arrow matrices presented to the participants in sequence. The time budgets in Experiment 1 included 3 s, 4 s, and 5 s, and the time budgets in Experiment 2 and 3 included 5 s and 7 s. A take-over request (TOR) warning was issued in the automated driving condition when the participants encountered a broken-down car in front. The participants must regain control of the vehicle with the given time budget. The hands-on time was evaluated, measuring the time from the TOR until the hands touch the steering wheel. Results The task (arrow task and SuRT), time budget (5 s and 7 s), and gender did not affect the hands-on time. However, the hands-on time for the drivers with the everyday task was significantly shorter than that for the drivers with the arrow task in the 5 s time budget. Conclusions In conditionally automated driving, the arrow task and SuRT imposed a similar workload on readiness to take over control. Compared to the everyday task, the engagement in the arrow tasks consumed more workload on readiness to take over control.

Eye movements predict driver reaction time to takeover request in automated driving: A real-vehicle study

For automated driving at SAE level 3 or lower, driver performance in responding to takeover requests (TORs) is decisive in providing system safety. A driver state monitoring system that can predict a driver’s performance in a TOR event will facilitate a safer control transition from vehicle to driver. This experimental study investigated whether driver eye-movement measured before a TOR can predict driving performance in a subsequent TOR event. We recruited participants (N = 36) to obtain realistic results in a real-vehicle study. In the experiment, drivers rode in an automated vehicle on a test track for about 32 min, and a critical TOR event occurred at the end of the drive. Eye movements were measured by a camera-based driver monitoring system, and five measures were extracted from the last 2-min epoch prior to the TOR event. The correlations between each eye-movement measure and driver reaction time were examined, and a multiple regression model was built using a stepwise procedure. The results showed that longer reaction time could be significantly predicted by a smaller number of large saccades, a greater number of medium saccades, and lower saccadic velocity. The implications of these relationships are consistent with previous studies. The present real-vehicle study can provide insights to the automotive industry in the search for a safer and more flexible interface between the automated vehicle and the driver.

Effects of Non-Speech Auditory Cues on Control Transition Behaviors in Semi-Automated Vehicles: Empirical Study, Modeling, and Validation

ABSTRACT In semi-automated vehicles, non-speech sounds have been prevalently used as auditory displays for control transitions since these sounds convey urgency well. However, there are no standards of specifications for warning sounds so that diverse non-speech sounds are being employed. To shed light on this, the effects of different non-speech auditory warnings on driver performance were investigated and quantified through the experimental study and human performance modeling approaches. Twenty-four young drivers drove in the driving simulator and experienced both handover and takeover transitions between manual and automated modes while performing a secondary task. The reaction times for handover and takeover, mental workload, and subjective responses were reported. Overall, a traditional warning sound with many repetitions and an indicator sound with decreasing polarity outperformed and were preferred. Additionally, a mathematical model, using the Queuing Network-Model Human Processor (QN-MHP) framework, was applied to quantify the effects of auditory warnings’ acoustic characteristics on drivers’ reaction times in response to takeover request displays. The acoustic characteristics, including the fundamental frequency, the number of repetitions, and the range of dominant frequencies were utilized in modeling. The model was able to explain 99.7% of the experimental data with a root mean square error (RMSE) of 0.148. The present study can contribute to establishing standards and design guidelines for takeover request displays in semi-automated vehicles.

Assessing subjective criticality of take-over situations: Validation of two rating scales

Assessing subjective criticality of take-over situations is crucial for understanding of take-over behavior and comparing studies. However, no validated rating scales exist that assess subjective criticality of take-over situations. In a driving simulator study, two rating scales, the Scale of Criticality Assessment of driving situations from Neukum et al. (2008) and the Criticality Rating Scale, were tested on their validity to assess the subjective criticality of take-over situations. Besides, the subjective and behavioral changes over the repeated experience of take-over situations were investigated. Twenty-five participants experienced a set of five take-over situations with varying time-to-collisions (TTC) at the moment of the take-over request, twice. After each of the first five take-over situations, participants rated the criticality on one scale, after each of the second five situations on the other scale. Correlation coefficients between TTCs and criticality ratings for each scale were calculated. Also, the changes of subjective and behavioral measures over the trials were investigated. Correlation coefficients indicated a strong correlation between criticality ratings and TTCs. Hence, both scales are equally valid for the assessment of the criticality of take-over situations. The repeated experience of the take-over situations did not affect effort ratings, take-over times, or steering wheel positions. But brake input decreased with increasing practice, indicating a safer take-over behavior. Hence, results of studies with repeated experience of take-over situations are relatively valid as only brake behavior changed with increasing practice.

Using eye-tracking to investigate the effects of pre-takeover visual engagement on situation awareness during automated driving

Automated driving systems are becoming increasingly prevalent throughout society. In conditionally automated vehicles, drivers may engage in non-driving-related tasks (NDRTs), which can negatively affect their situation awareness (SA) and preparedness to resume control of the vehicle, when necessary. Previous work has investigated engagement in NDRTs, but questions remain unanswered regarding its effect on drivers’ SA during a takeover event. The objective of the current study is to use eye-tracking to aid in understanding how visual engagement in NDRTs affects changes in SA of the driving environment after a takeover request (TOR) has been issued. Thirty participants rode in a simulated SAE Level 3 automated driving environment and engaged in three separate pre-TOR tasks (Surrogate Reference Task, Monitoring Task, and Peripheral Detection Task) until presented with a TOR. Situation Awareness Global Assessment Technique (SAGAT) scores and gaze behavior were recorded during the post-TOR segment. Overall, longer times spent viewing the driving scene, and more dispersed visual attention allocation, were observed to be associated with better overall SA. Also, location-based eye tracking metrics show most promise in differentiating between task conditions with significantly different SAGAT scores. Findings from this work can inform the development of real-time SA assessment techniques using eye movements and ultimately contribute to improved operator roadway awareness for next-generation automated transportation.

Exploring the benefits of conversing with a digital voice assistant during automated driving: A parametric duration model of takeover time

Vehicle automation allows drivers to disengage from driving causing a potential decline in their alertness. One of the major challenges of highly automated vehicles is to ensure a timely (with respect to safety and situation awareness) takeover in such conditions. For this purpose, the current study investigated the role of an in-vehicle digital voice-assistant (VA) in conditionally automated vehicles, offering spoken discourse relating specifically to contextual factors, such as the traffic situation and road environment. The study involved twenty-four participants, each taking two drives (counterbalanced): with VA and without VA, in a driving simulator. Participants were required to takeover vehicle control following the issuance of a takeover request (TOR) near the end of each drive. A parametric duration model was adopted to find the key factors determining takeover time (TOT). Paired comparisons showed higher alertness and higher active workload (mean NASA-TLX rating) during automation when accompanied by the VA. Paired t-test comparison of gaze behavior prior to takeover showed significantly higher instances of checking traffic signal, roadside objects, and the roadway during the drive with VA, indicating higher situation awareness. The parametric model indicated that the VA increased the likelihood of making a timely takeover by 39%. There was also some evidence suggesting that male drivers are likely to resume control 1.21 times earlier than female drivers. The study findings highlight the benefits of adopting a digital voice assistant to keep the drivers alert and aware about the recent traffic environment in partially automated vehicles.

Exploring the effectiveness of a digital voice assistant to maintain driver alertness in partially automated vehicles

Objective Vehicle automation shifts the driver's role from active operator to passive observer at the potential cost of degrading their alertness. This study investigated the role of an in-vehicle voice-based assistant (VA; conversing about traffic/road environment) to counter the disengaging and fatiguing effects of automation. Method Twenty-four participants undertook two drives– with and without VA in a partially automated vehicle. Participants were subsequently categorized into high and low participation groups (based on their proportion of vocal exchanges with VA). The effectiveness of VA was assessed based on driver alertness measured using Karolinska Sleepiness Scale (KSS), eye-based sleepiness indicators and glance behavior, NASA-TLX workload rating and time to gain motor readiness in response to take-over request and performance rating made by the drivers. Results Paired samples t-tests comparison of alertness measures across the two drives were conducted. Lower KSS rating, larger pupil diameter, higher glances (rear-mirror, roadside vehicles and signals in the drive with VA) and higher feedback ratings of VA indicated the efficiency of VA in improving driver alertness during automation. However, there was no significant difference in alertness or glance behavior between the driver groups (high and low-PR), although the time to resume steering control was significantly lower in the higher engagement group. Conclusion The study successfully demonstrated the advantages of using a voice assistant (VA) to counter these effects of passive fatigue, for example, by reducing the time to gain motor-readiness following a TOR. The findings show that despite the low engagement in spoken conversation, active listening also positively influenced driver alertness and awareness during the drive in an automated vehicle.

Effects of User Interfaces on Take-Over Performance: A Review of the Empirical Evidence

In automated driving, the user interface plays an essential role in guiding transitions between automated and manual driving. This literature review identified 25 studies that explicitly studied the effectiveness of user interfaces in automated driving. Our main selection criterion was how the user interface (UI) affected take-over performance in higher automation levels allowing drivers to take their eyes off the road (SAE3 and SAE4). We categorized user interface (UI) factors from an automated vehicle-related information perspective. Short take-over times are consistently associated with take-over requests (TORs) initiated by the auditory modality with high urgency levels. On the other hand, take-over requests directly displayed on non-driving-related task devices and augmented reality do not affect take-over time. Additional explanations of take-over situation, surrounding and vehicle information while driving, and take-over guiding information were found to improve situational awareness. Hence, we conclude that advanced user interfaces can enhance the safety and acceptance of automated driving. Most studies showed positive effects of advanced UI, but a number of studies showed no significant benefits, and a few studies showed negative effects of advanced UI, which may be associated with information overload. The occurrence of positive and negative results of similar UI concepts in different studies highlights the need for systematic UI testing across driving conditions and driver characteristics. Our findings propose future UI studies of automated vehicle focusing on trust calibration and enhancing situation awareness in various scenarios.

Drivers’ visual-distracted take-over performance model and its application on adaptive adjustment of time budget

There are certain situations that automated driving (AD) systems are still unable to handle, preventing the implementation of Level 5 AD. Thus, a transition of control, colloquially known as take-over of the vehicle, is required when the system sends a take-over request (TOR) upon exiting the operational design domain (ODD). An adaptive TOR along with good take-over performance requires adjusting the time budget (TB) to drivers’ visual distraction state, adhering to a reliable visual-distraction-based take-over performance model. Based on a number of driving simulator experiments, the percentage of face orientation to distraction area (PFODA) and time to boundary at take-over timing (TTBT) were proposed to accurately evaluate the degree of visual distraction based on merely face orientation under naturalistic non-driving related tasks (NDRTs) and to evaluate take-over performance, respectively. In order to elucidate the safety boundary, this study also proposed an algorithm to set a suitable minimum value of the TTBT. Finally, a multiple regression model was built to describe the relationship among PFODA, TB and TTBT along with a corrected coefficient of determination of 0.748. Based on the model, this study proposed an adaptive TB adjustment method for the take-over system.

The Decline of User Experience in Transition from Automated Driving to Manual Driving

Automated driving technologies are rapidly being developed. However, until vehicles are fully automated, the control of the dynamic driving task will be shifted between the driver and automated driving system. This paper aims to explore how transitions from automated driving to manual driving affect user experience and how that experience correlates to take-over performance. In the study 20 participants experienced using an automated driving system during rush-hour traffic in the San Francisco Bay Area, CA, USA. The automated driving system was available in congested traffic situations and when active, the participants could engage in non-driving related activities. The participants were interviewed afterwards regarding their experience of the transitions. The findings show that most of the participants experienced the transition from automated driving to manual driving as negative. Their user experience seems to be shaped by several reasons that differ in temporality and are derived from different phases during the transition process. The results regarding correlation between participants’ experience and take-over performance are inconclusive, but some trends were identified. The study highlights the need for new design solutions that do not only improve drivers’ take-over performance, but also enhance user experience during take-over requests from automated to manual driving.

Adaptive multi-modal interface model concerning mental workload in take-over request during semi-autonomous driving

With the development of automated driving technologies, human factors involved in automated driving are gaining increasing attention for a balanced implementation of the convenience brought by the ...

Westdrive X LoopAR: An Open-Access Virtual Reality Project in Unity for Evaluating User Interaction Methods during Takeover Requests.

With the further development of highly automated vehicles, drivers will engage in non-related tasks while being driven. Still, drivers have to take over control when requested by the car. Here, the question arises, how potentially distracted drivers get back into the control-loop quickly and safely when the car requests a takeover. To investigate effective human–machine interactions, a mobile, versatile, and cost-efficient setup is needed. Here, we describe a virtual reality toolkit for the Unity 3D game engine containing all the necessary code and assets to enable fast adaptations to various human–machine interaction experiments, including closely monitoring the subject. The presented project contains all the needed functionalities for realistic traffic behavior, cars, pedestrians, and a large, open-source, scriptable, and modular VR environment. It covers roughly 25 km2, a package of 125 animated pedestrians, and numerous vehicles, including motorbikes, trucks, and cars. It also contains all the needed nature assets to make it both highly dynamic and realistic. The presented repository contains a C++ library made for LoopAR that enables force feedback for gaming steering wheels as a fully supported component. It also includes all necessary scripts for eye-tracking in the used devices. All the main functions are integrated into the graphical user interface of the Unity® editor or are available as prefab variants to ease the use of the embedded functionalities. This project’s primary purpose is to serve as an open-access, cost-efficient toolkit that enables interested researchers to conduct realistic virtual reality research studies without costly and immobile simulators. To ensure the accessibility and usability of the mentioned toolkit, we performed a user experience report, also included in this paper.

운전 외 작업의 난이도에 따른 운전자의 시선 고정 및 제어권 전환 수행도 연구

Objective: The aim of this study is to investigate driver’s gaze fixation and takeover performance according to difficulty of a non-driving related task (NDRT).<BR>Background: NDRT has significant effect on driver’s behavior in an automated vehicle. When NDRT is being performed, it is impossible for drivers to react immediately to takeover request. The more driver is engaged on NDRT, the less takeover performance will be. Therefore, it is necessary to identify which factors cause drivers to shift their gaze to NDRT. There might be many factors that attract driver’s gaze, but in this paper, we will identify driver’s gaze fixation and takeover performance with NDRT difficulty.<BR>Method: To investigate effects of NDRT difficulty on driver’s gaze fixation and takeover performance, experiment was conducted through a driving simulator. NDRT difficulty was classified by 5 levels. The participants performed five trials depending on difficulty levels. During the experiments, fixation duration and eyes-on time was measured by eye-tracker and takeover time was recorded through driving simulator.<BR>Results: First, NDRT difficult has a significant effect on driver’s fixation during autonomous driving. As considering %fixation duration, driver’s gaze was fixated longer in the first and fifth level of difficulty than others. Also, there was a significant effect on takeover time. In the fifth level of difficulty, takeover time was longer than other levels of difficulty. However, eyes-on-time was constant depending on NDRT difficulty.<BR>Conclusion: NDRT difficult has a significant effect on driver’s gaze fixation and takeover performance. %fixation duration and takeover time represented a u-shape depending on NDRT difficulty.<BR>Application: The results of the study could be utilized in in-vehicle infotainment of automated vehicles. This results make interaction between humans and automated vehicles much better.