Take-over Request Research Articles

Persona-PhysioSync AV: Personalized Interaction through Personality and Physiology Monitoring in Autonomous Vehicles.

The emergence of autonomous vehicles (AVs) marks a transformative leap in transportation technology. Central to the success of AVs is ensuring user safety, but this endeavor is accompanied by the challenge of establishing trust and acceptance of this novel technology. The traditional "one size fits all" approach to AVs may limit their broader societal, economic, and cultural impact. Here, we introduce the Persona-PhysioSync AV (PPS-AV). It adopts a comprehensive approach by combining personality traits with physiological and emotional indicators to personalize the AV experience to enhance trust and comfort. A significant aspect of the PPS-AV framework is its real-time monitoring of passenger engagement and comfort levels within AVs. It considers a passenger's personality traits and their interaction with physiological and emotional responses. The framework can alert passengers when their engagement drops to critical levels or when they exhibit low situational awareness, ensuring they regain attentiveness promptly, especially during Take-Over Request (TOR) events. This approach fosters a heightened sense of Human-Vehicle Interaction (HVI), thereby building trust in AV technology. While the PPS-AV framework currently provides a foundational level of state diagnosis, future developments are expected to include interaction protocols that utilize interfaces like haptic alerts, visual cues, and auditory signals. In summary, the PPS-AV framework is a pivotal tool for the future of autonomous transportation. By prioritizing safety, comfort, and trust, it aims to make AVs not just a mode of transport but a personalized and trusted experience for passengers, accelerating the adoption and societal integration of autonomous vehicles.

Behavioral and physiological responses to takeovers in different scenarios during conditionally automated driving

A variety of takeover scenarios will happen in conditionally automated driving. Previous studies presented mixed results regarding the effects of scenarios on takeover performance. According to drivers' strategies for takeover requests, this study selected eight representative takeover scenarios and categorized them into lane keeping and lane changing scenarios. To investigate the effects of scenario type and road environment (highway vs. urban) on drivers' takeover performance and physiological responses, a driving simulation study was conducted as a mixed design with 40 participants (average age = 22.8 years). The results showed that in lane changing scenarios, with the same sensing capability, drivers on highways had deteriorated takeover performance in the form of harsher takeover maneuvers and higher collision risk, as well as higher arousal and stress, compared to urban areas. However, such effects disappeared or even reversed in lane keeping scenarios on the curves, where drivers on highways had smoother takeover maneuvers and lower arousal and stress. These findings will help us understand the vital roles scenario type and road environment play during takeover transitions. Our findings have implications for the design of advanced driver-assistance systems and will improve driving safety in conditionally automated driving.

Getting back in the loop: Does autonomous driving duration affect driver's takeover performance?

The level 3 autonomous driving function allows the driver to perform non-driving-related tasks such as watching movies or reading while the system manages the driving task. However, when a difficult situation arises, the driver is requested to return to the loop of control. This switching from driver to passenger then back to driver may modify the driving paradigm, potentially causing an out-of-the-loop state. We tested the hypothesis of a linear (progressive) impact of various autonomous driving durations: the longer the level 3 autonomous function is used, the poorer the driver's takeover performance. Fifty-two participants were divided into 4 groups, each group being assigned a specific period of autonomous driving (5, 15, 45, or 60 min), followed by a takeover request with a time budget of 8.3 s. Takeover performance was assessed over two successive drives via reaction times and manual driving metrics (trajectories). The initial hypothesis (linearity) was not confirmed: there was a nonlinear relationship between autonomous driving duration and takeover performance, with one duration (15 min) appearing safer overall and mixed performance within groups. Repetition induced a major change in performance during the second drive, indicating rapid adaptation to the situation. The non-driving-related task appears critical in several respects (dynamics, content, driver interest) to proper use of level 3 automation. All this supports previous research prompting reservations about the prospect of car driving becoming like train travel.

Predicting the duration of reduced driver performance during the automated driving takeover process

This study carried out a simulator test to determine and predict the duration of reduced driver performance during the automated driving takeover process. Vehicle trajectory and driver behavior data were collected in critical and non-critical takeover scenarios. The earth mover’s distance was then adopted to identify the data with the optimal combination of indicators by comparing it to the reference data. The Gaussian mixture model was employed to classify the driving state as either stable or unstable, and the duration of reduced driver performance was derived for each participant based on these results. Subsequently, a generalized linear mixed model was developed to predict the duration of reduced driver performance and examine the impact of various factors on it. Results uncovered a recovery of the reduced driver performance state after drivers took over the automated vehicle. In the non-critical and critical takeover scenarios, the mean duration of reduced driver performance was 17.48 and 27.25 s, respectively. Additionally, the developed model demonstrated good overall prediction accuracy, with the duration of reduced driver performance showing a positive correlation with the lead vehicle’s speed, duration of automated driving, and takeover request lead time. Furthermore, timid drivers exhibited a longer recovery duration than aggressive drivers. These research findings offer valuable insights into understanding the recovery of reduced driver performance during the takeover process, serving as a theoretical foundation for designing safer automated driving systems.

Effects of driving style on takeover performance during automated driving: Under the influence of warning system factors

Driving style has been proposed to be a critical factor in automated driving. However, the role of driving style in the process of taking over during automated driving needs further investigation. The main purpose of this study was to investigate the influence of driving style on takeover performance under the influence of warning system factors. In addition, this study also explored whether the impact of driving style on reaction time varies over time and the role of driving style on a comprehensive takeover quality indicator. Two driving simulation experiments with different takeover request (TOR) designs were conducted. In experiment 1, content warning information was provided in the TOR with different warning stage designs; in experiment 2, countdown warning information was provided in the TOR with different warning stage designs. Sixty-four participants (32 for experiment 1 and 32 for experiment 2) were classified into two groups based on their driving style (i.e., aggressive, or defensive) using the Chinese version of the Multidimensional Driving Style Inventory (the brief MDSI-C). The results suggested that drivers’ driving style had significant effects on takeover performance, but the effects were influenced by warning system designs. Specifically, defensive participants performed better takeover performance, i.e., shorter reaction time and cautious vehicle control behaviors, than aggressive participants in most warning conditions. The content and countdown warning information and warning stage design affected the roles of driving style on takeover performance: 1) compared to the one-stage warning design, the two-stage warning design significantly shortened the reaction time of the participants with different driving styles, 2) compared to the countdown warning information design, the design of content warning information can shorten the reaction time of aggressive participants and lengthen the reaction time of defensive participants in the two-stage warning conditions, and 3) compared to the content warning information design, countdown warning information can improve the safe takeover performance of defensive participants. This study provides a better understanding of the role of driving style on takeover performance, and driving style should be considered when designing warning systems for autonomous vehicles.

Exploring the Timing of Disengagement From Nondriving Related Tasks in Scheduled Takeovers With Pre-Alerts: An Analysis of Takeover-Related Measures.

This study aimed to investigate drivers' disengagement from nondriving related tasks (NDRT) during scheduled takeovers and to evaluate its impact on takeover performance. During scheduled takeovers, drivers typically have sufficient time to prepare. However, inadequate disengagement from NDRTs can introduce safety risks. Participants experienced scheduled takeovers using a driving simulator, undergoing two conditions, with and without an NDRT. We assessed their takeover performance and monitored their NDRT disengagement from visual, cognitive, and physical perspectives. The study examined three NDRT disengagement timings (DTs): DT1 (disengaged before the takeover request), DT2 (disengaged after the request but before taking over), and DT3 (not disengaged). The impact of NDRT on takeover performance varied depending on DTs. Specifically, DT1 demonstrated no adverse effects; DT2 impaired takeover time, while DT3 impaired both takeover time and quality. Additionally, participants who displayed DT1 exhibited longer eye-off-NDRT duration and a higher eye-off-NDRT count during the prewarning stage compared to those with DT2 and DT3. Drivers can benefit from earlier disengagement from NDRTs, demonstrating resilience to the adverse effects of NDRTs on takeover performance. The disengagement of cognition is often delayed compared to that of eyes and hands, potentially leading to DT3. Moreover, visual disengagement from NDRTs during the prewarning stage could distinguish DT1 from the other two. Our study emphasizes considering NDRT disengagement in designing systems for scheduled takeovers. Measures should be taken to promote early disengagement, facilitate cognitive disengagement, and employ visual disengagement during the prewarning period as predictive indicators of DTs.

Acceptance and Usability of a Soft Robotic, Haptic Feedback Seat for Autonomy Level Transitions in Highly Automated Vehicles.

Fully autonomous vehicles, capable of completing entire end-to-end journeys without the interference of a human driver, will be one of the biggest transforming technologies of the next decades. As the journey towards fully autonomous vehicles progresses, there will be an increase in the number of highly automated vehicles on the roads, requiring the human driver to take back control in situations, which cannot be handled by the vehicle autonomously. These human-robot take-over requests can lead to safety risks, in particular in scenarios when the driver fails to understand the take-over request and, hence, lacks situational awareness. This paper presents the acceptance and usability assessment of a haptic feedback driver seat capable of informing the driver of a take-over request through static mechano-tactile haptic feedback. The seat is equipped with an embedded array of soft pneumatic actuators, that have been fully modelled and characterised. The evaluation process of the haptic feedback seat engaged 21 participants who experienced both auditory and haptic feedback from the seat in a number of simulation experiments within a driving simulator. The vehicular technology was assessed through well-established methods to understand the acceptance (usefulness and satisfaction) and usability of the haptic feedback driver seat.

Impact of Personality on Takeover Time and Maneuvers Shortly After Takeover Request

Impact of Personality on Takeover Time and Maneuvers Shortly After Takeover Request

Navigating the Handover: Reviewing Takeover Requests in Level 3 Autonomous Vehicles

Navigating the Handover: Reviewing Takeover Requests in Level 3 Autonomous Vehicles

자율주행 차량의 제어권 인수요구 정보양상과 도로 형태에 따른 운전자의 제어권 인수시간과 정신적 작업부하 차이

자율주행 차량의 제어권 인수요구 정보양상과 도로 형태에 따른 운전자의 제어권 인수시간과 정신적 작업부하 차이

Anthropomorphic Design and Self-Reported Behavioral Trust: The Case of a Virtual Assistant in a Highly Automated Car

The latest advances in car automation present new challenges in vehicle–driver interactions. Indeed, acceptance and adoption of high levels of automation (when full control of the driving task is given to the automated system) are conditioned by human factors such as user trust. In this work, we study the impact of anthropomorphic design on user trust in the context of a highly automated car. A virtual assistant was designed using two levels of anthropomorphic design: “voice-only” and “voice with visual appearance”. The visual appearance was a three-dimensional model, integrated as a hologram in the cockpit of a driving simulator. In a driving simulator study, we compared the three interfaces: two versions of the virtual assistant interface and the baseline interface with no anthropomorphic attributes. We measured trust versus perceived anthropomorphism. We also studied the evolution of trust throughout a range of driving scenarios. We finally analyzed participants’ reaction time to takeover request events. We found a significant correlation between perceived anthropomorphism and trust. However, the three interfaces tested did not significantly differentiate in terms of perceived anthropomorphism while trust converged over time across all our measurements. Finally, we found that the anthropomorphic assistant positively impacts reaction time for one takeover request scenario. We discuss methodological issues and implication for design and further research.

Effect of a looming visual cue on situation awareness and perceived urgency in response to a takeover request

This study aimed to investigate the effect of a looming visual cue on situation awareness and perceived urgency in response to a takeover request (TOR), and to explore the underlying mechanisms of this effect through three experiments. In Experiment 1, the optimal size and speed of a red disk were determined, which were effective in capturing looming motion and conveying the urgency of the situation. The results indicated that both looming speed and size ratio had significant effects on situation awareness and perceived urgency. In Experiment 2, the effects of looming stimuli were compared with dimming stimuli, and the results showed that the looming visual cue was more effective in promoting perceived urgency and situation awareness. The results also indicated that the looming visual cue attracted more visual attention than the dimming visual cue, in line with previous studies. Experiment 3 utilized a driving simulator to test the effectiveness of the looming visual cue in promoting fast and appropriate responses to TORs in complex driving scenarios. The results showed that the looming visual cue was more effective in promoting perceived urgency and enhancing situation awareness, especially in highly complex driving situations. Overall, the findings suggest that the looming visual cue is a powerful tool for promoting fast and appropriate responses to TORs and enhancing situation awareness, particularly in complex driving scenarios. These results have important implications for designing effective TOR systems and improving driver safety on the road.

The monitoring requests on young driver’s fatigue and take-over performance in prolonged conditional automated driving

Introduction: L3 automated vehicles can perform all dynamic driving tasks unless a take-over occurs due to operational limits. This issue is potentially important for young drivers who are vulnerable road users since they have skill deficits and easily evolve into aberrant driving. However, drivers lacking active involvement may be fatigued and drowsy. Previous research indicated that performing a voluntary non-driving-related task (NDRT) could keep drivers alert, but there was no difference in take-over performance with or without NDRT. Providing a monitoring request (MR) before a possible take-over request (TOR) exhibited better take-over performance in temporary automated driving. Therefore, the study aimed to investigate the effects of MR and voluntary NDRT on young drivers’ fatigue and performance. Method: Twenty-five young drivers experienced 60 min automated driving on a highway with low traffic density and a TOR prompted due to a collision event. A within-subjects was designed that comprised three conditions: NONE, TOR-only, and MR + TOR. Drivers were allowed to perform a self-paced phone NDRT during automated driving. Results: The PERCLOS and blink frequency data showed that playing phones could keep drivers vigilant. The take-over performance on whether taking phone had no difference, but with MRs condition exhibited better take-over performance including the shorter reaction time and the longer TTC. Subjective evaluations also showed the advantages of MRs with more safety, trust, acceptance, and lower workload. Conclusions: Taking MRs had a positive effect on relieving fatigue and improving take-over performance. Furthermore, MRs could potentially improve the safety and acceptance of automated driving. Practical Applications: The MR design can be used in the automotive industry to ensure the safest interfaces between fatigue drivers and automation systems.

Design friction in autonomous drive—exploring transitions between autonomous and manual drive in non-urgent situations

In the ongoing turn to automation, the growing trend towards the design of conditionally and highly automated vehicles (C/HAV) is evident. In a CAV, the driver no longer needs to partake in the driving. However, the vehicle might send a takeover request (TOR) when the CAV’s system reaches its operational boundaries, i.e. a call for a transition from autonomous to manual drive. Previous research on TORs has focused on the context of urgent situations, e.g. hazards and unpredictable events. Furthermore, it has been noted that drivers’ situation awareness (SA) deteriorates after being in autonomous drive. However, less is known about TORs in non-urgent situations. Motivated by this need, the study explores how design friction can serve as a guiding concept for transferring control between autonomous and manual drive in non-urgent situations to increase situation awareness. Design friction is defined as elements of interactions that steer attention and guides the driver to take informed decisions. The work resulted in prototypes that leveraged design friction as part of a takeover sequence. The proposed design was empirically evaluated in a fixed-base medium-fidelity driving simulator. The results indicated that the level of friction might have been too extensive, as some annoyance was expressed. However, participants claimed to feel calm and aware of their surroundings at the moment of regaining control of the vehicle. This suggests that design friction is a promising tool for guiding concept design to enhance transitions from autonomous to manual drive.

Driver-initiated take-overs during critical evasion maneuvers in automated driving

The aim of the current study is to investigate predictors and consequences of driver-initiated take-overs during automated evasion maneuvers. Literature on control transitions in automated driving has mainly focused on system-initiated take-overs. However, drivers may also initiate take-overs without take-over requests. To date, such driver-initiated take-overs have rarely been investigated. Our study addresses this research gap.In a driving simulator study with 61 participants, we investigated whether the criticality of highly dynamic evasion maneuvers and trust in automation affect the probability of driver-initiated take-overs. Criticality was manipulated via time headway (THW) and traction usage (TU). Trust was varied by manipulating automation reliability before the experimental trials. Consequences of driver-initiated take-overs in terms of collisions and lane departures were assessed.The results indicate that THW, TU, and trust affect the probability of driver-initiated take-overs. Moreover, the time it takes the automation to respond to an obstacle ahead by starting an evasion maneuver may be another relevant factor in predicting take-overs. After a take-over, drivers produced a number of unnecessary lane departures and collisions. These were independent of THW and TU.The study demonstrates that drivers are more likely to take over vehicle control during automated evasion maneuvers without take-over requests when criticality increases and trust in automation decreases. Such take-overs may be hazardous for traffic safety. Our findings help to design automated vehicles that avoid unnecessary take-overs in critical driving situations or de-escalate their consequences effectively, thus increasing traffic safety.

Regular looks out the window do not maintain situation awareness in highly automated driving

The loss of situation awareness (SA) caused by the out-of-the-loop phenomenon is a major problem for successful take-overs after highly automated driving. Despite recent advances in driving automation, a human backup for the automated system is currently indispensable. For uneventful situations, the automation works and drivers can perform engaging non-driving-related tasks (NDRTs). However, they need to be ready to respond at all times in case of a take-over request (TOR). For smooth and safe transition from automated to manual control, SA has to be re-established within seconds. We investigated whether regular out-the-window looks can facilitate this process. This driving simulation study is the first to investigate whether such situation awareness maintenance can improve take-over performance after a TOR. In a repeated-measures design, 89 participants experienced 5-minutes of highly automated driving with a NDRT followed by a critical take-over situation. Every 30 s, participants were asked either to monitor the traffic scene for 3 s (monitoring condition) or to merely interrupt the NDRT for 3 s (control condition). We ensured that incidental traffic monitoring was impossible during the NDRT. In the control condition, participants saw a blank screen during the 3-seconds intervals. Contrary to common intuition, the results show neither improvements in objective take-over performance nor in perceived SA, driver availability, criticality, stress, risk assessment, or adequacy of the TOR. We conclude that brief interruptions of the NDRT to look at the ongoing traffic are unable to maintain the driver's SA during highly automated driving. Future studies should therefore focus on other measures to optimize the recovery of SA.

Driver Response to Take-Over Requests in Real Traffic

Driver Response to Take-Over Requests in Real Traffic

The Effect of AR-HUD Takeover Assistance Types on Driver Situation Awareness in Highly Automated Driving: A 360-Degree Panorama Experiment

Human-machine co-driving presents a significant hurdle in automated driving system. The takeover process in automated driving system involves complex human factors, failure to takeover the vehicle and control driving behavior during the takeover process may lead to severe traffic safety hazards. An augmented reality head-up display (AR-HUD) takeover assistance information can provide real-time assistance information to the driving environment, enhancing drivers’ situation awareness (SA) and takeover decisions in highly automated driving system. This study investigated the impact of different AR-HUD types of takeover assistance information display. Three AR-HUD types, corresponding to the three pre-takeover behavioral processes (perception, understanding, and prediction), were evaluated: PSR (assistance in perceiving the source of risk), AS (assistance in analyzing situations), and MD (assistance in making decisions). The baseline (without assistance information) was used as the control group. In a driving simulation experiment using 360° panoramic video, seventy-nine participants performed SA assessment and visual tracking tasks. Questionnaire and eye-tracking data indicated that the type of AR-HUD displayed positively influenced drivers’ SA and takeover decisions, with AS being the most effective in enhancing SA and improving takeover performance. Additionally, this study compared the differences between the three types of AR-HUD and the baseline under two takeover request lead times (TORlt) of 5 seconds and 7 seconds. It was found that drivers’ SA was lower when TORlt was shorter (with the corresponding AR-HUD display also being shorter). This study provides insight concerning the impact of various types of AR-HUD takeover assistance information display and TORlt on driving safety. The findings support the further optimization of AR-HUD takeover assistance information design.

How much situation awareness does the driver have when driving autonomously? A study based on driver attention allocation

During the operation of the L3 automated driving system, since there is no need to supervise the vehicle at all times, the driver is often disengaged from the driving task and engages in a variety of non-driving-related tasks (NDRTs). When the autonomous driving system (ADS) encounters an unexpected situation and issues a takeover request (TOR), whether the driver can recover the situation awareness (SA) in time is the key to ensure the safety of the takeover. In this study, the theory of attention resource allocation is introduced to more accurately model the dynamic process of the SA recovery. Moreover, the attention allocation model is further developed, and the affecting factors of attention allocation are quantified. An experiment with 90 participants using driving simulator was conducted for different road scenarios to verify the proposed SA model. The model proposed in this study can accurately predict the SA value of the driver under different road scenarios and the time required for the driver to recover to the maximum SA value, which provides a reference for the scientific design of dynamic takeover lead time. The results also show that as the radius of curvature of the road decreased, the level of SA recovery would become progressively worse.

Analyzing Driver Strategy for Scheduled Takeovers Under the Effect of Multi-stage Takeover Request Design

For scheduled takeover events in conditional driving automation, a longer lead time of takeover request (ToR) up to 60 seconds was recommended for drivers’ better situation awareness and subjective evaluations. However, the long ToR lead time could impair drivers’ estimation of the urgency of situation, which potentially results in risky takeover behavior. Four multi-stage ToR designs varying in warning modality that communicate the urgency within a long lead time were thus proposed to mitigate the negative effects. The present study involved a driving simulator experiment with 32 participants performing 12 takeover tasks for exiting a freeway with the aim to analyze driver strategy for scheduled takeover events and evaluate the multi-stage ToR designs. Eight driver strategies for attention management and takeover were identified. Using multi-stage ToR designs advanced drivers’ takeover actions by decreasing the likelihood of alternating attention between non-driving activities and road and reducing the duration of takeover preparation.