Take-over Situations Research Articles

Effects of unreliable automation and takeover time budget on young drivers’ mental workload

Young drivers are involved in a significantly higher number of crashes than any other age group. Highly automated vehicles are expected to improve traffic safety and reduce human errors; however, driver needs to take over in situations where the automation fails. A driving simulation study was conducted with twenty-eight young drivers to investigate the effects of unreliable automation, different takeover time budgets (TOTBs), and secondary tasks on mental workload during takeover situations. Mental workload was measured using a combination of physiological and secondary task performance measures. Results suggested that when faced with critical incidents, young drivers experienced lower mental workload under 10s of TOTB as compared to 5s or 8s of TOTB especially when they were engaged in a demanding secondary task. In addition, drivers experienced higher mental workload when regaining the control of the vehicle in the automated mode than the manual driving condition.

Investigating the Impacts of Road Traffic Conditions and Driver’s Characteristics on Automated Vehicle Takeover Time and Quality Using a Driving Simulator

This study investigates the impacts of road traffic conditions and driver’s characteristics on the takeover time in automated vehicles using a driving simulator. Automated vehicles are barely expected to maintain their fully automated driving capability at all times based on the current technologies, and the automated vehicle system transfers the vehicle control to a driver when the system can no longer be automatically operated. The takeover time is the duration from when the driver requested the vehicle control transition from the automated vehicle system to when the driver takes full control of the vehicle. This study assumes that the takeover time can vary according to the driver’s characteristics and the road traffic conditions; the assessment is undertaken with various participants having different characteristics in various traffic volume conditions and road geometry conditions. To this end, 25 km of the northbound road section between Osan Interchange and Dongtan Junction on Gyeongbu Expressway in Korea is modeled in the driving simulator; the experiment participants are asked to drive the vehicle and take a response following a certain triggering event in the virtual driving environment. The results showed that the level of service and road curvature do not affect the takeover time itself, but they significantly affect the stabilization time, that is, a duration for a driver to become stable and recover to a normal state. Furthermore, age affected the takeover time, indicating that aged drivers are likely to slowly respond to a certain takeover situation, compared to the younger drivers. With these findings, this study emphasizes the importance of having effective countermeasures and driver interface to monitor drivers in the automated vehicle system; therefore, an early and effective alarm system to alert drivers for the vehicle takeover can secure enough time for stable recovery to manual driving and ultimately to achieve safety during the takeover.

Assessing subjective criticality of take-over situations: Validation of two rating scales

Assessing subjective criticality of take-over situations is crucial for understanding of take-over behavior and comparing studies. However, no validated rating scales exist that assess subjective criticality of take-over situations. In a driving simulator study, two rating scales, the Scale of Criticality Assessment of driving situations from Neukum et al. (2008) and the Criticality Rating Scale, were tested on their validity to assess the subjective criticality of take-over situations. Besides, the subjective and behavioral changes over the repeated experience of take-over situations were investigated. Twenty-five participants experienced a set of five take-over situations with varying time-to-collisions (TTC) at the moment of the take-over request, twice. After each of the first five take-over situations, participants rated the criticality on one scale, after each of the second five situations on the other scale. Correlation coefficients between TTCs and criticality ratings for each scale were calculated. Also, the changes of subjective and behavioral measures over the trials were investigated. Correlation coefficients indicated a strong correlation between criticality ratings and TTCs. Hence, both scales are equally valid for the assessment of the criticality of take-over situations. The repeated experience of the take-over situations did not affect effort ratings, take-over times, or steering wheel positions. But brake input decreased with increasing practice, indicating a safer take-over behavior. Hence, results of studies with repeated experience of take-over situations are relatively valid as only brake behavior changed with increasing practice.

Keeping the driver in the loop in conditionally automated driving: A perception-action theory approach

In this paper we investigated if keeping the driver in the perception–action loop during automated driving can improve take-over behavior from conditionally automated driving. To meet this aim, we designed an experiment in which visual exposure (perception) and manual control exposure (action) were manipulated. In a dynamic driving simulator experiment, participants (n = 88) performed a non-driving related task either in a head-up display in the windshield (high visual exposure) or on a head-down display near the gear shift (low visual exposure). While driving, participants were either in an intermittent control-mode with four noncritical take-over situations (high manual control exposure), or in a continuous automation-mode throughout the ride (low manual control exposure). In all conditions, a critical take-over had to be carried out after an approximately 13 min ride. Measurements of take-over behavior showed that only high visual exposure had an effect on hands-on reaction time measurements. Both visual exposure and manual control exposure had small to medium sized main effects on time to system deactivation, the maximum velocity of the steering wheel, and the standard deviation of the steering wheel angle. The combined high visual – and high manual control exposure condition led to 0.55 s faster reaction time and 37% less steering variability in comparison to the worst case low visual – and low manual control exposure condition. Together, results corroborate that maintaining visual exposure and manual control exposure during automated driving can be efficacious and suggest that their positive effects are additive.

Effects of User Interfaces on Take-Over Performance: A Review of the Empirical Evidence

In automated driving, the user interface plays an essential role in guiding transitions between automated and manual driving. This literature review identified 25 studies that explicitly studied the effectiveness of user interfaces in automated driving. Our main selection criterion was how the user interface (UI) affected take-over performance in higher automation levels allowing drivers to take their eyes off the road (SAE3 and SAE4). We categorized user interface (UI) factors from an automated vehicle-related information perspective. Short take-over times are consistently associated with take-over requests (TORs) initiated by the auditory modality with high urgency levels. On the other hand, take-over requests directly displayed on non-driving-related task devices and augmented reality do not affect take-over time. Additional explanations of take-over situation, surrounding and vehicle information while driving, and take-over guiding information were found to improve situational awareness. Hence, we conclude that advanced user interfaces can enhance the safety and acceptance of automated driving. Most studies showed positive effects of advanced UI, but a number of studies showed no significant benefits, and a few studies showed negative effects of advanced UI, which may be associated with information overload. The occurrence of positive and negative results of similar UI concepts in different studies highlights the need for systematic UI testing across driving conditions and driver characteristics. Our findings propose future UI studies of automated vehicle focusing on trust calibration and enhancing situation awareness in various scenarios.

Classification of Drivers' Workload Using Physiological Signals in Conditional Automation.

The use of automation in cars is increasing. In future vehicles, drivers will no longer be in charge of the main driving task and may be allowed to perform a secondary task. However, they might be requested to regain control of the car if a hazardous situation occurs (i.e., conditionally automated driving). Performing a secondary task might increase drivers' mental workload and consequently decrease the takeover performance if the workload level exceeds a certain threshold. Knowledge about the driver's mental state might hence be useful for increasing safety in conditionally automated vehicles. Measuring drivers' workload continuously is essential to support the driver and hence limit the number of accidents in takeover situations. This goal can be achieved using machine learning techniques to evaluate and classify the drivers' workload in real-time. To evaluate the usefulness of physiological data as an indicator for workload in conditionally automated driving, three physiological signals from 90 subjects were collected during 25 min of automated driving in a fixed-base simulator. Half of the participants performed a verbal cognitive task to induce mental workload while the other half only had to monitor the environment of the car. Three classifiers, sensor fusion and levels of data segmentation were compared. Results show that the best model was able to successfully classify the condition of the driver with an accuracy of 95%. In some cases, the model benefited from sensors' fusion. Increasing the segmentation level (e.g., size of the time window to compute physiological indicators) increased the performance of the model for windows smaller than 4 min, but decreased for windows larger than 4 min. In conclusion, the study showed that a high level of drivers' mental workload can be accurately detected while driving in conditional automation based on 4-min recordings of respiration and skin conductance.

The impact of auditory continual feedback on take-overs in Level 3 automated vehicles

ObjectiveTo implement auditory continual feedback into the interface design of a Level 3 automated vehicle and to test whether gaze behavior and reaction times of drivers improved in take-over situations. BackgroundWhen required to assume manual control in take-over situations, drivers of Level 3 automated vehicles are less likely than conventional drivers to spot potential hazards, and their reaction time is longer. Therefore, it is crucial that the interface of Level 3 automated vehicles will be designed to improve drivers’ performance in take-over situations. MethodIn two experiments, participants drove a simulated route in a Level 3 automated vehicle for 35 min with one imminent take-over event. Participants’ gaze behavior and performance in an imminent take-over event were monitored under one of three auditory interface designs: (1) Continual feedback. A system that provides verbal driving-related feedback; (2) Persistent feedback. A system that provides verbal driving-related feedback and a persistent beep; and (3) Chatter feedback. A system that provides verbal non-driving-related feedback. Also, there was a control group without feedback. ResultsUnder all three auditory feedback designs, the number of drivers' on-road glances increased compared to no feedback, but none of the designs shortened reaction time to the imminent event. ConclusionIncreasing the number of on-road glances during automated driving does not necessarily improve drivers’ attention to the road and their reaction times during take-overs. ApplicationPossible implications for the effectiveness of auditory continual feedback should be considered when designing interfaces for Level 3 automated vehicles.

Development of a Situational Awareness Estimation Model Considering Traffic Environment for Unscheduled Takeover Situations

In semi-autonomous vehicles (SAE level 3) that requires drivers to takeover (TO) the control in critical situations, a system needs to judge if the driver have enough situational awareness (SA) for manual driving. We previously developed a SA estimation system that only used driver’s glance data. For deeper understanding of driver’s SA, the system needs to evaluate the relevancy between driver’s glance and surrounding vehicle and obstacles. In this study, we thus developed a new SA estimation model considering driving-relevant objects and investigated the relationship between parameters. We performed TO experiments in a driving simulator to observe driver’s behavior in different position of surrounding vehicles and TO performance such as the smoothness of steering control. We adopted support vector machine to classify obtained dataset into safe and dangerous TO, and the result showed 83% accuracy in leave-one-out cross validation. We found that unscheduled TO led to maneuver error and glance behavior differed from individuals.

Shaping driver-vehicle interaction in autonomous vehicles: How the new in-vehicle systems match the human needs

Autonomous vehicle (AV) technology has brought a shift in the traditional role of the driver. This paper applies a user-centred design approach to designing a new AV interior to better support drivers. Three empirical studies were conducted, involving a total of 92 drivers (with 44 in Study 1, 12 in Study 2, and 36 in Study 3) to explore user needs and requirements in an AV. In Study 1, safety and comfort, together with a variety of non-driving activities, were identified as the principal concerns about future autonomous vehicles. Based on these findings, Study 2 proposes a new rotatable seating position for AVs, with an in-vehicle information display to facilitate users' activities and situational awareness while driving. Study 3 consists of a series of laboratory simulator evaluation studies, and this indicated that drivers in the proposed design condition had better situational awareness in an AV when dealing with take-over situations. Such findings suggest the possibility of applying rear-facing seats in autonomous vehicles to support in-vehicle non-driving activities. Some specific implications of designs to enhance a driver's situational awareness have also been suggested.

Effects of Non-Driving-Related Task Attributes on Takeover Quality in Automated Vehicles

ABSTRACT This study aimed to investigate the effects of non-driving-related tasks (NDRTs) on takeover quality in the context of automated driving. Specifically, we examined the effects of three categories of NDRT attributes (i.e., physical, cognitive, and visual) on longitudinal and lateral driving measures when the drivers resumed control. We designed a driving simulator study where the participants experienced automated driving journeys and takeover situations. When the automated mode was activated, drivers engaged in one of the nine NDRTs. The results showed that the cognitive load of NDRTs had a significant negative correlation with both longitudinal and lateral control measures. However, the effects of two attributes in the physical category and one attribute in the visual category on driving performance did not show statistical significance. Overall, the findings indicated that the influence of cognitive attributes on takeover quality is more salient than that of the physical and visual attributes, which provides insights into the understanding of takeover situations to improve driving safety.

Behavioral changes to repeated takeovers in automated driving: The drivers’ ability to transfer knowledge and the effects of takeover request process

Only a couple of studies evaluated whether drivers of automated vehicles change their takeover behavior when they experience takeover requests repeatedly. Even less evidence was accumulated regarding the question whether drivers are able to transfer learned behavior to takeover situations with varying visibility characteristics and whether drivers’ takeover behavior depends on the takeover process in these situations. This paper therefore examines three research questions. First, it assesses how drivers change their behavior with the repeated experience of a takeover situation with the same visibility (fog or no fog). Second, it tests whether drivers can transfer their learned takeover behavior from a takeover situation with high or low visibility to the same takeover situation with different visibility conditions. Third, it assesses whether drivers’ takeover behavior and their experience of the situation differ between a one-step and a two-step takeover request process. Forty participants experienced a takeover situation three times. Experimental trials varied between-subjects concerning the permanent presence or absence of fog in the adaptation condition, the change of visibility conditions from fog to no fog or vice versa in the transfer condition, and the design of the takeover process with one-step or two-steps. Dependent variables included participants’ takeover time, minimum time-to-collision (TTCmin) with the construction site, deceleration and maximum steering behavior, and their ratings of criticality of the driving situation and perceived effort. Results show that participants adapted their deceleration behavior when repeatedly experiencing a takeover situation with the same visibility characteristics (adaptation condition). Changing these characteristics (transfer condition) lead to increased minimum TTCs, criticality and perceived effort ratings. In general, participants were able to maintain their takeover behavior in takeover situations with varying visibility characteristics indicating that they can transfer their takeover behavior across situations. Finally, the two-step takeover request process was associated with longer takeover times. However, minimum TTCs were larger and maximum steering movements and criticality ratings were lower compared to the one-step process. We conclude that drivers transfer their behavior across takeover situations. However, this performance comes at higher costs in terms of perceived effort and criticality. In addition, two-step takeover request processes should be favored over one-step processes when designing takeover requests. Future studies should examine the validity of the results in various takeover situations and on-the-road studies.

Methodological Approach towards Evaluating the Effects of Non-Driving Related Tasks during Partially Automated Driving

Partially automated driving (PAD, Society of Automotive Engineers (SAE) level 2) features provide steering and brake/acceleration support, while the driver must constantly supervise the support feature and intervene if needed to maintain safety. PAD could potentially increase comfort, road safety, and traffic efficiency. As during manual driving, users might engage in non-driving related tasks (NDRTs). However, studies systematically examining NDRT execution during PAD are rare and most importantly, no established methodologies to systematically evaluate driver distraction during PAD currently exist. The current project’s goal was to take the initial steps towards developing a test protocol for systematically evaluating NDRT’s effects during PAD. The methodologies used for manual driving were extended to PAD. Two generic take-over situations addressing system limits of a given PAD regarding longitudinal and lateral control were implemented to evaluate drivers’ supervisory and take-over capabilities while engaging in different NDRTs (e.g., manual radio tuning task). The test protocol was evaluated and refined across the three studies (two simulator and one test track). The results indicate that the methodology could sensitively detect differences between the NDRTs’ influences on drivers’ take-over and especially supervisory capabilities. Recommendations were formulated regarding the test protocol’s use in future studies examining the effects of NDRTs during PAD.

What happens when drivers of automated vehicles take over control in critical brake situations?

Even with automated vehicles, driving situations with short time headways and extreme vehicle dynamics may arise when unpredictable events occur. If drivers take back control under such conditions, it is uncertain how they behave and how well they can cope with the situation. This issue has not been investigated yet and is subject to our study. In a driving simulator, non-distracted participants (N = 42) experienced nine critical situations caused by a braking vehicle in front of them. Time headway and longitudinal vehicle dynamics were varied to create different degrees of objective criticality. Participants’ criticality ratings, take-over behavior, and driving performance were recorded and analyzed. The results indicate that participants were sensitive to changes in objective criticality and adapted their behavior. Take-over times were very fast under all conditions and participants showed higher criticality ratings, more intense decelerations, and more lane changes with increasing objective criticality. To avoid a collision, participants decelerated much more than the automation and changed lanes, even though this was not necessary. Thereby, they raised the risk of vehicle instability, rear-end collisions, and collisions with overtaking vehicles. To conclude, take-overs in critical brake situations may be a threat to the safety of drivers and other road users because drivers’ reactions are more pronounced than necessary. These results suggest that assistive functions are required to support drivers in critical take-over situations.

Understanding take-over performance of high crash risk drivers during conditionally automated driving

Understanding driver behavior of conditionally automated driving is necessary to ensure a safe transition from automated to manual driving. This study aimed to examine the difference in take-over performance between high crash risk (HCR) and lower crash risk (LCR) drivers in emergency take-over situations during conditionally automated driving. In the current simulator study, a 3 × 3 (within-subjects) factorial design was used, including the task factors (no task, reading the news, and watching a video) and time budget factors (time budget = 3 s, 4 s, and 5 s). Forty-eight participants completed a test drive on an approximately 10 km long two-way six-lane urban road. The participants firstly were in manual control and then switched to the automated driving mode at a speed of 50 km/h. The automated driving system was able to detect a broken car in the ego-lane and requested the driver to take over the control of the vehicle. There are at least one or two other vehicles or motorcycles on each side of the ego-vehicle, resulting in fewer escape paths. For the two non-handheld non-driving-related tasks (NDRTs), the participants were asked to be fully engaged in a task without any need to monitor the road environments. Each participant completed nine emergency take-over situations. The participants were classified into two groups that were labeled LCR (N ≤ 2) and HCR drivers (N ≥ 3) according to the number of accidents per driver. The results show that LCR drivers had shorter brake reaction time compared to HCR drivers. For all drivers, the engagement in a task led to longer response times, and the time budget affected the longitudinal vehicle control. In addition, the task affected the response times for LCR and HCR drivers, but only the time budget affected the longitudinal vehicle control for LCR drivers. For all drivers, LCR and HCR drivers, the time budget and task affected the safety of take-over. Especially, the two non-handheld everyday tasks seem to have a similar effect on the drivers’ workload. Therefore, the HCR drivers had a lower hazard perception compared to the LCR drivers, and the factor regarding the individual difference of driving ability in take-over situations should be considered to design safe take-over concepts for automated vehicles.

Take-over expectation and criticality in Level 3 automated driving: a test track study on take-over behavior in semi-trucks

With the introduction of advanced driving assistance systems managing longitudinal and lateral control, conditional automated driving is seemingly in near future of series vehicles. While take-over behavior in the passenger car context has been investigated intensively in recent years, publications on semi-trucks with professional drivers are sparse. The effects influencing expert drivers during take-overs in this context lack thorough investigation and are required to design systems that facilitate safe take-overs. While multiple findings seem to cohere in passenger cars and semi-trucks, these findings rely on simulated studies without taking environments as found in the real world into account. A test track study was conducted, simulating highway driving with 27 professional non-affiliated truck drivers. The participants drove an automated Level 3 semi-truck while a non-driving-related task was available. Multiple time critical take-over situations were initiated during the drives to investigate four main objectives regarding driver behavior. (1) With these results, comparison of reaction times and behavior can be drawn to previous simulator studies. The effect of situation criticality (2) and training (3) of take-over situations is investigated. (4) The influence of warning expectation on driver behavior is explored. Results obtained displayed very quick time to hands on steering and time to first reaction all under 2.4 s. Highly critical situations generate very quick reaction times M = 0.81 s, while the manipulation of expectancy yielded no significant variation in reaction times. These reaction times serve as a reference of what can be expected from drivers under optimal take-over conditions, with quick reactions at high speed in critical situations.

The Impact of Situational Complexity and Familiarity on Takeover Quality in Uncritical Highly Automated Driving Scenarios

In the development of highly automated driving systems (L3 and 4), much research has been done on the subject of driver takeover. Strong focus has been placed on the takeover quality. Previous research has shown that one of the main influencing factors is the complexity of a traffic situation that has not been sufficiently addressed so far, as different approaches towards complexity exist. This paper differentiates between the objective complexity and the subjectively perceived complexity. In addition, the familiarity with a takeover situation is examined. Gold et al. show that repetition of takeover scenarios strongly influences the take-over performance. Yet, both complexity and familiarity have not been considered at the same time. Therefore, the aim of the present study is to examine the impact of objective complexity and familiarity on the subjectively perceived complexity and the resulting takeover quality. In a driving simulator study, participants are requested to take over vehicle control in an uncritical situation. Familiarity and objective complexity are varied by the number of surrounding vehicles and scenario repetitions. Subjective complexity is measured using the NASA-TLX; the takeover quality is gathered using the take-over controllability rating (TOC-Rating). The statistical evaluation results show that the parameters significantly influence the takeover quality. This is an important finding for the design of cognitive assistance systems for future highly automated and intelligent vehicles.

Repeated Usage of an L3 Motorway Chauffeur: Change of Evaluation and Usage

Most studies on users’ perception of highly automated driving functions are focused on first contact/single usage. Nevertheless, it is expected that with repeated usage, acceptance and usage of automated driving functions might change this perception (behavioural adaptation). Changes can occur in drivers’ evaluation, in function usage and in drivers’ reactions to take-over situations. In a driving simulator study, N = 30 drivers used a level 3 (L3) automated driving function for motorways during six experimental sessions. They were free to activate/deactivate that system as they liked and to spend driving time on self-chosen side tasks. Results already show an increase of experienced trust and safety, together with an increase of time spent on side tasks between the first and fourth sessions. Furthermore, attention directed to the road decreases with growing experience with the system. The results are discussed with regard to the theory of behavioural adaptation. Results indicate that the adaptation of acceptance and usage of the highly automated driving function occurs rather quickly. At the same time, no behavioural adaptation for the reaction to take-over situations could be found.

The Effects of a Predictive HMI and Different Transition Frequencies on Acceptance, Workload, Usability, and Gaze Behavior during Urban Automated Driving

Automated driving research as a key topic in the automotive industry is currently undergoing change. Research is shifting from unexpected and time-critical take-over situations to human machine interface (HMI) design for predictable transitions. Furthermore, new applications like automated city driving are getting more attention and the ability to engage in non-driving related activities (NDRA) starting from SAE Level 3 automation poses new questions to HMI design. Moreover, future introduction scenarios and automated capabilities are still unclear. Thus, we designed, executed, and assessed a driving simulator study focusing on the effect of different transition frequencies and a predictive HMI while freely engaging in naturalistic NDRA. In the study with 33 participants, we found transition frequency to have effects on workload and acceptance, as well as a small impact on the usability evaluation of the system. Trust, however, was not affected. The predictive HMI was used and accepted, as can be seen by eye-tracking data and the post-study questionnaire, but could not mitigate the above-mentioned negative effects induced by transition frequency. Most attractive activities were window gazing, chatting, phone use, and reading magazines. Descriptively, window gazing and chatting gained attractiveness when interrupted more often, while reading magazines and playing games were negatively affected by transition rate.

Should the Urgency of Visual-Tactile Takeover Requests Match the Criticality of Takeover Situations?

Highly-automated driving requires drivers to take over the control of their vehicle in any situation. In takeover situations with small time headways, a fast takeover is crucial. Here, drivers could be supported by an appropriate takeover request. This article presents a driving simulator study investigating whether less critical takeover situations and a match between the urgency of a takeover request and the criticality of the takeover situation benefits drivers regarding their takeover performance and subjective experience. Fifty-two participants experienced two takeover situations with varying time headways (small and large). A low urgent, visual-tactile takeover-request was presented to half of the participants. The other half experienced a highly urgent, visual-tactile takeover request, resulting in two conditions, in which the takeover requests matched the criticality of the takeover situation, and in two conditions, in which the takeover requests did not match. The results showed that participants generally performed better in low critical takeover situations, but they took over quicker in highly critical situations. However, a match neither shortens takeover time nor enhances subjective experience.

The Effect of Takeover Lead Time on Driver Workload

Autonomous driving has been one of the most interesting technologies in recent years with expectation of solving accidents, pollution, and traffic jams (Jo, Lee, & Kim, 2013; Schrank, Eisele, & Lom...