Take-over Request Research Articles

Multimodal Cue Combinations: A Possible Approach to Designing In-Vehicle Takeover Requests for Semi-autonomous Driving

The rapid growth of autonomous vehicles is expected to improve roadway safety. However, certain levels of vehicle automation will still require drivers to ‘takeover’ during abnormal situations, which may lead to breakdowns in driver-vehicle interactions. To date, there is no agreement on how to best support drivers in accomplishing a takeover task. Therefore, the goal of this study was to investigate the effectiveness of multimodal alerts as a feasible approach. In particular, we examined the effects of uni-, bi-, and trimodal combinations of visual, auditory, and tactile cues on response times to takeover alerts. Sixteen participants were asked to detect 7 multimodal signals (i.e., visual, auditory, tactile, visual-auditory, visual-tactile, auditory-tactile, and visual-auditory-tactile) while driving under two conditions: with SAE Level 3 automation only or with SAE Level 3 automation in addition to performing a road sign detection task. Performance on the signal and road sign detection tasks, pupil size, and perceived workload were measured. Findings indicate that trimodal combinations result in the shortest response time. Also, response times were longer and perceived workload was higher when participants were engaged in a secondary task. Findings may contribute to the development of theory regarding the design of takeover request alert systems within (semi) autonomous vehicles.

Evaluation of the effects of age-friendly human-machine interfaces on the driver’s takeover performance in highly automated vehicles

The ability to continue driving into old age is strongly associated with older adults’ mobility and wellbeing for those that have been dependant on car use for most of their adult lives. The emergence of highly automated vehicles (HAVs) may have the potential to allow older adults to drive longer and safer. In HAVs, when operating in automated mode, drivers can be completely disengaged from driving, but occasionally they may be required to take back the control of the vehicle. The human-machine interfaces in HAVs play an important role in the safe and comfortable usage of HAVs. To date, only limited research has explored how to design age-friendly HMIs in HAVs and evaluate their effectiveness. This study designed three HMI concepts based on older drivers’ requirements, and conducted a driving simulator investigation with 76 drivers (39 older drivers and 37 younger drivers) to evaluate the effect and relative merits of these HMIs on drivers’ takeover performance, workload and attitudes. Results showed that the ‘R + V’ HMI (informing drivers of vehicle status together with providing the reasons for the manual driving takeover request) led to better takeover performance, lower perceived workload and highly positive attitudes, and is the most beneficial and effective HMI. In addition, The ‘V’ HMI (verbally informing the drivers about vehicle status, including automation mode and speed, before the manual driving takeover request) also had a positive effect on drivers’ takeover performance, perceived workload and attitudes. However, the ‘R’ HMI (solely informing drivers about the reasons for takeover as part of the takeover request) affected older and younger drivers differently, and resulted in deteriorations in performance and more risky takeover for both older and younger drivers compared to the baseline HMI. Moreover, significant age difference was observed in the takeover performance and perceived workload. Above all, this research highlights the significance of taking account older drivers’ requirements into the design of HAVs and the importance of collaboration between automated vehicle and cooperative ITS research communities.

Take-over requests during highly automated driving: How should they be presented and under what conditions?

Even with highly automated driving, there will exist the occasional need for a human driver to resume control. The design of take-over requests (TOR) and the driving scenario upon their presentation are factors that can be expected to determine the effective resumption of vehicle control. We conducted a driving simulator study in the laboratory that examined how either an abstract or skeuomorphic TOR design influenced driving resumption, given different conditions of road curvature and time headway. We found that drivers react faster and more accurately when experiencing a skeuomorphic visualisation of a TOR prompt compared to an abstract interface, especially when there is less time headway. Moreover, participants reported subjective preferences for the skeuomorphic TOR design and considered them to be more intuitively comprehensible. Furthermore, more accurate TOR responses were found when TORs of lateral control were presented before longitudinal control. Based on these results, we formulated five conclusions relating to the design of take-over requests. These conclusions should be validated in more realistic settings. However, in the future, they could be the basis for more research developing design guidelines that could create safe and reliable communications between drivers and vehicles.

자율주행 시 탑승자의 비운전상태에 따른 운전권 전환 반응 차이에 대한 연구

자율주행 시 탑승자의 비운전상태에 따른 운전권 전환 반응 차이에 대한 연구

Task Interruption and Control Recovery Strategies After Take-Over Requests Emphasize Need for Measures of Situation Awareness.

Our objective was to determine whether there is a need to go beyond measures of automation deactivation time to understand the transition to manual driving after take-over requests (TORs) using the example of office tasks as nondriving-related tasks (NDRTs). Office tasks are likely NDRTs during automated commutes to/from work. Complex tasks can influence how manual control and visual attention is recovered after TORs. N = 51 participants in a driving simulator performed either one of two office tasks or no task (between subjects). We recorded reaction times in a high-urgency and low-urgency scenario (within subjects) and analyzed task interruption strategies. 90% of the participants who performed an NDRT deactivated the automation after 7 to 8 s. However, 90% of the same drivers looked at the side mirror for the first time only after 11 to 14 s. Drivers with office tasks either interrupted the tasks sequentially or in parallel. Strategies were not adapted to the take-over situation or the task but appeared to be due to individual preferences. Drivers engaged in NDRTs may neglect lower priority subtasks after a TOR, such as mirror checking. Therefore, there is a need to go beyond measures of automation deactivation time to understand the transition to manual driving. Using analyses of attentional dynamics during take-over situations may enhance the safety of future car-driver handover assistance systems. If low driver availability is detected, TORs should only be used as a fallback option if sufficient time and adaptive driver support can be provided.

Drivers’ Individual Design Preferences of Takeover Requests in Highly Automated Driving

Abstract Highly automated cars will be on the worlds’ roads within the next decade. In highly automated driving the vehicle’s lateral and longitudinal controls can be passed on from the driver to the vehicle and back again. The design of a vehicle’s take-over requests will largely determine the driver’s performance after taking back vehicle control. In the scope of this paper, potential drivers of highly automated cars were asked about their preferences regarding the human-machine interface design of take-over requests. Participants were asked to evaluate eight different take-over requests that differed with respect to (a) take-over request procedure (one-step or two-step procedure), (b) visual take-over request modality (text or text and pictogram), and (c) auditory take-over request modality (tone or speech). Results showed that participants preferred a two-step procedure using text and speech to communicate take-over requests. A subsequent conjoint analysis revealed that take-over requests ideally use speech output in a two-step procedure. Finally, a detailed evaluation showed that the best take-over request interface received significantly higher user experience ratings regarding product characteristics as well as users’ emotions and consequences of product use than the worst take-over request interface. Results are related to the background literature and practical implications are discussed.

Directional tactile alerts for take-over requests in highly-automated driving

One of the most significant safety concerns regarding Highly-Automated Driving (HAD) is drivers’ ability to regain control of the vehicle safely. Vibro-tactile alerts were already suggested as an effective modality for Take-Over Requests (TOR) in terms of reducing reaction times. However, it is not clear yet whether such alerts should be compatible or incompatible with the location of hazards that might be present when the TOR is initiated. Studies regarding tactile directionality in other domains, and in manual vehicles have found mixed results. It is argued that part of the contradictory evidence may be related to contextual differences between the driving domain and other domains. Thus, this study aimed to test which directional design would be preferable for TORs in time-critical situations. Twenty-seven participants drove a highly-automated vehicle on a highway with two lanes in each travel direction, in a driving simulator. Each participant experienced five TORs in which they were required to take control and divert their vehicle away from an impending hazard that shut down an entire lane and was situated four seconds ahead. The disengagement of the autonomous driver was signaled using a tactile alert. For the first group, the tactile alert was directed towards the hazard (incompatible with the required action), for the second, it was directed away from it (compatible with the required action), and for the control group, the alert was non-directional. Results showed that drivers using the compatible alert reacted faster and more accurately than those using the incompatible alert. Participants using the non-directional alert reacted slower and less accurately than participants in both directional groups. The results contradict previous findings in the manual driving domain, where drivers are faster and more accurate to respond when the alert is compatible with the location of the hazard and not with the direction of the required action. It is argued that these discrepancies stem from the modified HAD driving task demands where drivers are disengaged from the driving task for long periods and are less aware of the driving environment. The implications for the design of autonomous vehicles are discussed.

In a heart beat: Using driver’s physiological changes to determine the quality of a takeover in highly automated vehicles

Developing conditionally automated driving systems is on the rise. Vehicles with full longitudinal and latitudinal control will allow drivers to engage in secondary tasks without monitoring the roadway, but users may be required to resume vehicle control to handle critical hazards. The loss of driver’s situational awareness increases the potential for accidents. Thus, the automated systems need to estimate the driver’s ability to resume control of the driving task.The aim of this study was to assess the physiological behaviour (heart rate and pupil diameter) of drivers. The assessment was performed during two naturalistic secondary tasks. The tasks were the email and the twenty questions task in addition to a control group that did not perform any tasks. The study aimed at finding possible correlations between the driver’s physiological data and their responses to a takeover request. A driving simulator study was used to collect data from a total of 33 participants in a repeated measures design to examine the physiological changes during driving and to measure their takeover quality and response time.Secondary tasks induced changes on physiological measures and a small influence on response time. However, there was a strong observed correlation between the physiological measures and response time. Takeover quality in this study was assessed using two new performance measures called PerSpeed and PerAngle. They are identified as the mean percentage change of vehicle’s speed and heading angle starting from a take-over request time. Using linear mixed models, there was a strong interaction between task, heart rate and pupil diameter and PerSpeed, PerAngle and response time. This, in turn, provided a measurable understanding of a driver’s future responses to the automated system based on the driver’s physiological changes to allow better decision making. The present findings of this study emphasised the possibility of building a driver mental state model and prediction system to determine the quality of the driver's responses in a highly automated vehicle. Such results will reduce accidents and enhance the driver’s experience in highly automated vehicles.

Influence of bicyclist presence on driver performance during automated vehicle take-over requests

One proposed benefit of automated vehicles (AVs) is their potential to mitigate the occurrence of serious crashes due to human error or poor decision making while driving. However, there are still many concerns associated with the use of SAE Level 3 AVs, which require intervention by a human driver after a take-over request (TOR). These concerns intensify when vulnerable road users, such as bicyclists, are introduced to the driving environment. The objective of this research was to investigate how human drivers of AVs interact with bicyclists during a right-turn maneuver after receiving a TOR. Changes in driver performance, including visual attention and crash avoidance behavior, were measured by using a high-fidelity driving simulator, with 43 participants each completing 18 right-turn maneuvers. Three independent variables were studied: the bicyclist’s proximity to the intersection, the driver’s proximity from the intersection when the TOR was received, and the driver’s engagement in a distracting secondary task (a game on a tablet). In general, the results showed that the introduction of the secondary task led to decreased driver performance with respect to time-to-collision and the time that it took a driver to first identify the bicyclist on the roadway. When given more time to react before the intersection, drivers generally had safer interactions with the bicyclist.

Bridging Gaps Among Human, Assisted, and Automated Driving With DVIs: A Conceptional Experimental Study

Partially automated driving releases the driver from controlling the vehicle, but the driver still needs to monitor the driving situation steadily and be able to take over the control in critical situations when needed. In such a situation, problems arise as to how and when should the driver be informed and how to make sure the driver understands the general situation and specifically also the intention of the system correctly. Driver-vehicle interfaces (DVIs) present the necessary information to the driver. Many studies focusing on the time for Takeover-request (TOR) and the takeover time of the driver are performed leading to various results. The question: “Can a general conclusion about upper and lower limits of these two variables be drawn, which is applicable for all drivers in all situations?” should be answered. In the situation in which the driver should take over the control of the vehicle, multiple driving modes are involved. Mode error or confusion may happen when the vehicle has multiple modes. This leads to another question to be discussed in this contribution: Is it possible to use one DVI to show the status of several autonomous driving levels and corresponding functionalities to avoid mode error or confusion? To answer these questions, two DVIs are designed considering multiple autonomous driving levels and their functionalities to increase the situation/mode awareness of the driver. A simulator study with 38 participants was conducted to study the takeover time and behavior of drivers under different critical situations as well as to compare the usability. Results show that the takeover time depends upon the sequence of experiencing the TOR and the complexity of the takeover situation. Furthermore, two proposed interfaces could contribute to increasing the situation awareness of the driver. The limits and trends of developing the TOR are concluded based on the literature review and the experiments performed. It is concluded that using an average TOR time for every individual driver in all critical situations is not appropriate. Discussion with respect to TOR to ensure a safe takeover regarding non-driving-related tasks and takeover experiences is derived from the performed experiments.

Determinants of take-over time from automated driving: A meta-analysis of 129 studies

An important question in automated driving research is how quickly drivers take over control of the vehicle in response to a critical event or a take-over request. Although a large number of studies have been performed, results vary strongly. In this study, we investigated mean take-over times from 129 studies with SAE level 2 automation or higher. We used three complementary approaches: (1) a within-study analysis, in which differences in mean take-over time were assessed for pairs of experimental conditions, (2) a between-study analysis, in which correlations between experimental conditions and mean take-over times were assessed, and (3) a linear mixed-effects model combining between-study and within-study effects. The three methods showed that a shorter mean take-over time is associated with a higher urgency of the situation, not using a handheld device, not performing a visual non-driving task, having experienced another take-over scenario before in the experiment, and receiving an auditory or vibrotactile take-over request as compared to a visual-only or no take-over request. A consistent effect of age was not observed. We also found the mean and standard deviation of the take-over time were highly correlated, indicating that the mean is predictive of variability. Our findings point to directions for new research, in particular concerning the distinction between drivers’ ability and motivation to take over, and the roles of urgency and prior experience.

Long automated driving phase affects take‐over performance

The authors investigated the impact of automated driving phases of different durations (long versus short) on take-over performance and driver state. About 30 participants drove on a dynamic simulator under autonomous mode for three successive periods of automated driving: a short (10 min), a long (1 h) and another short (10 min) period, each ending with a take-over request. They performed a non-driving task, watching a film of their choice, throughout the autonomous phases. Driving performance – reaction time and quality – and driver drowsiness were assessed at each take over. About 1 h of automated driving affected the driver's behaviour, leading to poorer take-over performance (longer reaction time and sharper avoidance manoeuvre) and increased drowsiness compared with a shorter autonomous period. Results also suggest that sequencing the autonomous phase in several short periods should improve the driver's take-over performance and help prevent drowsiness. Their findings sound a warning on the risks associated with long phases of automated driving. In particular, they provide evidence that a series of short periods of automated driving is preferable to a long, continuous automated driving phase.

자율주행 상황에서 제어권 전환 요청 시기가 운전자 작업부하에 미치는 영향

자율주행 상황에서 제어권 전환 요청 시기가 운전자 작업부하에 미치는 영향

Engaging in NDRTs affects drivers’ responses and glance patterns after silent automation failures

The aim of this study was to understand driver responses to “silent” failures in automated driving, where automation failed during a simulator drive, without a take-over warning. The effect of a visual non-driving related task (NDRT) and a road-based vigilance task presented drivers’ take-over response and visual attention was also investigated. Currently, automated driving systems face a number of limitations that require control to be handed back to the driver. Much of the research to date has focused on explicit take-over requests (ToRs) and shows that drivers struggle to resume control safely, exacerbated by disengagement from the driving task, for instance, due to the presence of NDRTs. However, little is known about whether, and how, drivers will respond to more subtle automation failures that come without a warning, and how this is affected by NDRT engagement. Thirty participants drove a simulated automated drive in two conditions, which had 6 silent automation failures each (3 on a Curve, 3 in a Straight), with no ToRs. In one condition, drivers were required to constantly monitor the road, which was enforced by a road-based vigilance task (VMS Only). In the other, drivers performed an additional visual NDRT, requiring them to divide their attention (VMS + Arrows). Results showed that, in both conditions, all drivers eventually detected and responded to all silent automation failures. However, engaging in an additional NDRT during automation resulted in significantly more lane excursions and longer take-over times. Adding a visual NDRT not only changed the distribution of drivers’ visual attention before and after the failure but also how they divided their attention between information on the road environment and the human–machine interface, which provided information on automation status. These results provide support for how driver monitoring systems may be used to detect drivers’ visual attention to the driving task and surroundings, and used as a tool for encouraging driver intervention, when required.

Beyond mere take-over requests: The effects of monitoring requests on driver attention, take-over performance, and acceptance

In conditionally automated driving, drivers do not have to monitor the road, whereas in partially automated driving, drivers have to monitor the road permanently. We evaluated a dynamic allocation of monitoring tasks to human and automation by providing a monitoring request (MR) before a possible take-over request (TOR), with the aim to better prepare drivers to take over safely and efficiently. In a simulator-based study, an MR + TOR condition was compared with a TOR-only condition using a within-subject design with 41 participants. In the MR + TOR condition, an MR was triggered 12 s before a zebra crossing, and a TOR was provided 7 s after the MR onset if pedestrians crossing the road were detected. In the TOR-only condition, a TOR was provided 5 s before the vehicle would collide with a pedestrian if the participant did not intervene. Participants were instructed to perform a self-paced visual-motor non-driving task during automated driving. Eye tracking results showed that participants in the MR + TOR condition responded to the MR by looking at the driving environment. They also exhibited better take-over performance, with a shorter response time to the TOR and a longer minimum time to collision as compared to the TOR-only condition. Subjective evaluations also showed advantages of the MR: participants reported lower workload, higher acceptance, and higher trust in the MR + TOR condition as compared to the TOR-only condition. Participants’ reliance on automation was tested in a third drive (MR-only condition), where automation failed to provide a TOR after an MR. The MR-only condition resulted in later responses (and errors of omission) as compared to the MR + TOR condition. It is concluded that MRs have the potential to increase safety and acceptance of automated driving as compared to systems that provide only TORs. Drivers’ trust calibration and reliance on automation need further investigation.

Toward Computational Simulations of Behavior During Automated Driving Takeovers: A Review of the Empirical and Modeling Literatures.

This article provides a review of empirical studies of automated vehicle takeovers and driver modeling to identify influential factors and their impacts on takeover performance and suggest driver models that can capture them. Significant safety issues remain in automated-to-manual transitions of vehicle control. Developing models and computer simulations of automated vehicle control transitions may help designers mitigate these issues, but only if accurate models are used. Selecting accurate models requires estimating the impact of factors that influence takeovers. Articles describing automated vehicle takeovers or driver modeling research were identified through a systematic approach. Inclusion criteria were used to identify relevant studies and models of braking, steering, and the complete takeover process for further review. The reviewed studies on automated vehicle takeovers identified several factors that significantly influence takeover time and post-takeover control. Drivers were found to respond similarly between manual emergencies and automated takeovers, albeit with a delay. The findings suggest that existing braking and steering models for manual driving may be applicable to modeling automated vehicle takeovers. Time budget, repeated exposure to takeovers, silent failures, and handheld secondary tasks significantly influence takeover time. These factors in addition to takeover request modality, driving environment, non-handheld secondary tasks, level of automation, trust, fatigue, and alcohol significantly impact post-takeover control. Models that capture these effects through evidence accumulation were identified as promising directions for future work. Stakeholders interested in driver behavior during automated vehicle takeovers may use this article to identify starting points for their work.

SAE 기반 레벨 2 및 레벨 3 부분자율주행자동차의 제어권 전환 상황 단일 모달리티 알림방법 설계

SAE 기반 레벨 2 및 레벨 3 부분자율주행자동차의 제어권 전환 상황 단일 모달리티 알림방법 설계

Investigation of older drivers’ requirements of the human-machine interaction in highly automated vehicles

The population of older drivers is increasing in size. However, age-related functional decline potentially reduce their safe driving ability and thereby their wellbeing may decline. Fortunately, the forthcoming highly automated vehicles (HAVs) may have the potential to enhance the mobility of older drivers. HAVs would introduce a revolutionary human-machine interaction in which drivers can be completely disengaged from driving, and their control would be required occasionally. In order to inform the design of an age-friendly human-machine interaction in HAVs, several semi-structured interviews were conducted with 24 older drivers (mean = 71.50 years, SD = 5.93 years; 12 female, 12 male) to explore their opinions of and requirements towards HAV after they had hands-on experience with a HAV on a driving simulator. Results showed that older drivers were positive towards HAVs and welcomed the hands-on experience with HAVs. In addition, they wanted to retain physical and potential control over the HAVs, and would like to perform a range of non-driving related tasks in HAVs. Meanwhile, they required an information system and a monitoring system to support their interactions with HAVs. Moreover, they required the takeover request of HAVs to be adjustable, explanatory and hierarchical, and they would like the driving styles of HAVs to be imitative and corrective. Above all, this research provides recommendations to inform the design of age-friendly human-machine interactions in HAVs and highlights the importance of considering the older drivers’ requirements when designing and developing automated vehicles.

Drivers' Knowledge of Automated Driving System and Take-Over Request Performances

Drivers' Knowledge of Automated Driving System and Take-Over Request Performances

Rolling Out the Red (and Green) Carpet: Supporting Driver Decision Making in Automation-to-Manual Transitions

This paper assessed four types of human-machine interfaces (HMIs), classified according to the stages of automation proposed by Parasuraman et al. [“A model for types and levels of human interaction with automation,” IEEE Trans. Syst. Man, Cybern. A, Syst. Humans, vol. 30, no. 3, pp. 286-297, May 2000]. We hypothesized that drivers would implement decisions (lane changing or braking) faster and more correctly when receiving support at a higher automation stage during transitions from conditionally automated driving to manual driving. In total, 25 participants with a mean age of 25.7 years (range 19-36 years) drove four trials in a driving simulator, experiencing four HMIs having the following different stages of automation: baseline (information acquisition-low), sphere (information acquisition-high), carpet (information analysis), and arrow (decision selection), presented as visual overlays on the surroundings. The HMIs provided information during two scenarios, namely a lane change and a braking scenario. Results showed that the HMIs did not significantly affect the drivers' initial reaction to the take-over request. Improvements were found, however, in the decision-making process: When drivers experienced the carpet or arrow interface, an improvement in correct decisions (i.e., to brake or change lane) occurred. It is concluded that visual HMIs can assist drivers in making a correct braking or lane change maneuver in a take-over scenario. Future research could be directed toward misuse, disuse, errors of omission, and errors of commission.