Driver Situation Awareness Research Articles

Beyond gaze fixation: Modeling peripheral vision in relation to speed, Tesla Autopilot, cognitive load, and age in highway driving

ObjectiveThe study aims to model driver perception across the visual field in dynamic, real-world highway driving. BackgroundPeripheral vision acquires information across the visual field and guides a driver’s information search. Studies in naturalistic settings are lacking however, with most research having been conducted in controlled simulation environments with limited eccentricities and driving dynamics. MethodsWe analyzed data from 24 participants who drove a Tesla Model S with Autopilot on the highway. While driving, participants completed the peripheral detection task (PDT) using LEDs and the N-back task to generate cognitive load. The I-DT (identification by dispersion threshold) algorithm sampled naturalistic gaze fixations during PDTs to cover a broader and continuous spectrum of eccentricity. A generalized Bayesian regression model predicted LED detection probability during the PDT—as a surrogate for peripheral vision—in relation to eccentricity, vehicle speed, driving mode, cognitive load, and age. ResultsThe model predicted that LED detection probability was high and stable through near-peripheral vision but it declined rapidly beyond 20°-30° eccentricity, showing a narrower useful field over a broader visual field (maximum 70°) during highway driving. Reduced speed (while following another vehicle), cognitive load, and older age were the main factors that degraded the mid-peripheral vision (20°-50°), while using Autopilot had little effect. ConclusionsDrivers can reliably detect objects through near-peripheral vision, but their peripheral detection degrades gradually due to further eccentricity, foveal demand during low-speed vehicle following, cognitive load, and age. ApplicationsThe findings encourage the development of further multivariate computational models to estimate peripheral vision and assess driver situation awareness for crash prevention.

The effect of the dominance of an in-vehicle agent’s voice on driver situation awareness, emotion regulation, and trust: A simulated lab study of manual and automated driving

As in-vehicle voice agents increase in popularity, related research is extending to how voice messages can affect the driver’s cognitive and emotional states. Accordingly, we investigated how in-vehicle agent (IVA) voice dominance and driving automation can affect the driver’s situation awareness (SA), emotion regulation (ER), and trust. To this end, a lab-based experiment was conducted with a medium-fidelity driving simulator using actor-recorded voice agents. Twenty-two female and nineteen male driver-licensed participants were recruited to drive simulated vehicles with voice agents and evaluated. The results demonstrated that compared with the dominant voice, the agent with a submissive voice significantly increased ER in both manual and automated driving. Furthermore, the submissive voice significantly increased trust in automated driving compared with the dominant agent. Cross and synergistic interaction effects exist between voice dominance and driving automation in SA and ER, respectively. This study revealed that both the content of the messages of the IVAs and their voice characteristics are essential for modulating the driver’s SA, ER, and trust in driving. It is expected that larger-scale future studies with simulation or on a real road would increase the validity of this study.

The effects of takeover request lead time on drivers’ situation awareness for manually exiting from freeways: A web-based study on level 3 automated vehicles

Conditional automation systems allow drivers to turn their attention away from the driving task in certain scenarios but still require drivers to gain situation awareness (SA) upon a takeover request (ToR) and resume manual control when the system is unable to handle the upcoming situation. Unlike time-critical takeover situations in which drivers must respond within a relatively short time frame, the ToRs for non-critical events such as exiting from a freeway can be scheduled way ahead of time. It is unknown how the ToR lead time affects driver SA for resuming manual control and when to send the ToR is most appropriate in non-critical takeover events. The present study conducted a web-based, supervised experiment with 31 participants using conditional automation systems in freeway existing scenarios while playing a mobile game. Each participant experienced 12 trials with different ToR lead times (6, 8, 10, 12, 14, 16, 18, 20, 25, 30, 45, and 60 s) for exiting from freeways in a randomized order. Driver SA was measured by using a freeze probe technique in each trial when the participant pressed the spacebar on the laptop to simulate the takeover action. Results revealed a positive effect of longer ToR lead times on driver SA for resuming control to exit from freeways and the effect leveled off at the lead time of 16–30 s. The participants tended to postpone their takeover actions further when they were given a longer ToR lead time and it did not level off up to 60 s. Nevertheless, not all drivers waited till the last moment to take over AVs even though they did not get sufficient SA. The ToR lead time of 16–30 s was recommended for better SA; and it could be narrowed down to 25–30 s if considering the subjective evaluations on takeover readiness, workload, and trust. The findings provide implications for the future design of conditional automation systems used for freeway driving.

Advancing Knowledge on Situation Comprehension in Dynamic Traffic Situations by Studying Eye Movements to Empty Spatial Locations.

This study used the looking-at-nothing phenomenon to explore situation awareness (SA) and the effects of working memory (WM) load in driving situations. While driving, people develop a mental representation of the environment. Since errors in retrieving information from this representation can have fatal consequences, it is essential for road safety to investigate this process. During retrieval, people tend to fixate spatial positions of visually encoded information, even if it is no longer available at that location. Previous research has shown that this "looking-at-nothing" behavior can be used to trace retrieval processes. In a video-based laboratory experiment with 2 (WM) x 3 (SA level) within-subjects design, participants (N = 33) viewed a reduced screen and evaluated auditory statements relating to different SA levels on previously seen dynamic traffic scenarios while eye movements were recorded. When retrieving information, subjects more frequently fixated emptied spatial locations associated with the information relevant for the probed SA level. The retrieval of anticipations (SA level 3) in contrast to the other SA level information resulted in more frequent gaze transitions that corresponded to the spatial dynamics of future driving behavior. The results support the idea that people build a visual-spatial mental image of a driving situation. Different gaze patterns when retrieving level-specific information indicate divergent retrieval processes. Potential applications include developing new methodologies to assess the mental representation and SA of drivers objectively.

USING ARTIFICIAL POTENTIAL FIELDS TO MODEL DRIVER SITUATIONAL AWARENESS

Recently, the use of artificial potential fields, known as risk fields, has been proposed for modeling human driver decision making. Such potential fields map from vehicle states and control inputs to a numerical risk measure such that the probability of choosing a control decreases as the risk associated increases. In this paper, we show that such a model can be used in a natural manner to also capture aspects of the driver's situational awareness, assuming that the risk fields govern their underlying behavior. We demonstrate our ideas on a specific obstacle avoidance scenario wherein obstacles to be avoided are placed in front of a driver at predicable intervals. Using data collected on a pilot experiment involving six different drivers using a high-fidelity driving simulator, we demonstrate the ability of our approach to capture the likelihood that the driver has perceived/reacted to the obstacle. Our approach works for scenarios when the driver collides with the obstacle as well as scenarios involving successful collision avoidance.

Software-in-the-loop simulation of the forerunner UAV system

The forerunner UAV means a camera equipped drone flying in front of the advancing first responder units to increase driver situational awareness with an aerial view of the traffic situation and notification about imminent dangers. This article presents the software-in-the-loop (SIL) simulation of the concept including UNREAL4-Carla as the virtual reality environment with a firetruck driven through a game controller, the Matlab simulation of the DJI M600 forerunner hexacopter with UDP communication between firetruck and M600 and the real-time AI processing of synthetic images to detect ground vehicles and pedestrians. The target of SIL development is threefold. First, to test M600 autopilot and AI-based object detection in close to realistic conditions before the real flights. Second, to make an exhaustive feasibility study of the whole forerunner concept with several simulated situations. Third, to generate the required large amount of image data for AI object detection tuning. After introducing all parts of the SIL simulation the article presents an illustrative example evaluating the tracking of the ground vehicle with the M600 and the inference system results.

Assessing the potential impacts of connected vehicle systems on Driver’s situation awareness and driving performance

Vehicles equipped with connected vehicle technologies are able to communicate with each other and with infrastructures. Compared to Advanced Driving Assistance Systems (ADAS) using camera systems and sensor technologies, the Connected Vehicle Systems (CVS) leverage the wireless communication networks to detect hazards with a greater range, alert drivers of hazards much earlier, and therefore enhance driving safety. However, drivers’ reliance on the CVS to detect critical situations could negatively affect them maintaining situation awareness (SA) in noncritical situations when no warning is issued by the CVS. The present study conducted a driving simulator experiment with 40 participants to investigate the effect of connected vehicle systems on driver SA in normal, noncritical driving scenarios after they were exposed to the CVS with different designs of collision warning lead time (3 s, 6 s, and no warnings). After drivers experienced the CVS-supported warnings with the assigned design of lead time in critical situations, driver SA was measured in normal driving conditions using the freeze probe technique. Results revealed that drivers who experienced the CVS with early warnings (6 s) showed lower SA for normal driving events compared to those who experienced the CVS with late warnings (3 s) or no warnings. Although early warnings of CVS brought more safety benefits to drivers in critical situations, the degraded driver SA due to drivers’ reliance on such warning systems could endanger drivers when a system failure occurred. These findings highlight the importance of balancing the effects of warning lead time on driver SA and driving performance in designing connected vehicle systems.

Vehicle Trajectory Estimation Based on Fusion of Visual Motion Features and Deep Learning.

Driver situation awareness is critical for safety. In this paper, we propose a fast, accurate method for obtaining real-time situation awareness using a single type of sensor: monocular cameras. The system tracks the host vehicle’s trajectory using sparse optical flow and tracks vehicles in the surrounding environment using convolutional neural networks. Optical flow is used to measure the linear and angular velocity of the host vehicle. The convolutional neural networks are used to measure target vehicles’ positions relative to the host vehicle using image-based detections. Finally, the system fuses host and target vehicle trajectories in the world coordinate system using the velocity of the host vehicle and the target vehicles’ relative positions with the aid of an Extended Kalman Filter (EKF). We implement and test our model quantitatively in simulation and qualitatively on real-world test video. The results show that the algorithm is superior to state-of-the-art sequential state estimation methods such as visual SLAM in performing accurate global localization and trajectory estimation for host and target vehicles.

Identifying objective behavioural measures of expert driver situation awareness

Efforts to reduce road crash rates depend on a clear understanding of the factors that contribute to driver risk. Not all drivers are at equivalent risk. It is critical to identify the factors that contribute to the development of expertise in the driving environment. The rapid development of a driver’s situation awareness (SA) is central to the safe performance of the driving task. Therefore, SA must be clearly operationalised in order to better assess its role in the development of expertise. This study employs an existing scheme based on the Perceptual Cycle Model (PCM) used for post hoc incident analysis and adapts it to the driving context. We attempted to correlate performance on coded verbalisations indicative of SA with non-invasive objective gaze metrics. Gaze metrics and the verbal counts were shown to differentiate between both expert and experienced (non-expert) drivers, but these measures failed to correlate with one another. Findings indicate differences in the way expert and experienced drivers update their schema of the driving task, with equivalent effort required to do so. The novel adaptation demonstrated in this paper allows for a domain-specific assessment of SA which reliably differentiates between drivers of varying expertise levels. Although selected gaze metrics were shown to be inadequate predictors of SA, additional analysis demonstrated key differences in gaze content. Combined, these findings enhance an understanding of expert SA development contributing to reduced crash risk.

Novel Auditory Displays in Highly Automated Vehicles: Sonification Improves Driver Situation Awareness, Perceived Workload, and Overall Experience

Highly automated driving systems are expected to require the design of new user-vehicle interactions. Sonification can be used to provide contextualized alarms and cues that can increase situation awareness and user experience. In this study, we examined user perceptions of potential use cases for level 4 automated vehicles in online focus group interviews (N=12). Also, in a driving simulator study, we evaluated (1) visual-only display; (2) non-speech with visual display; and (3) speech with visual display with 20 young drivers. Results indicated participants’ interest in the use cases and insight on desired functions in highly automated vehicles. Both audiovisual display conditions resulted in higher situation awareness for drivers than the visual-only condition. Some differences were found between the non-speech and speech conditions suggesting benefits of sonification for both driving and non-driving related auditory use cases. This study will provide guidance on sonification design for highly automated vehicles.

The Relationship Between Takeover Request Lead Time and Drivers’ Situation Awareness for Freeway Exiting in Conditionally Automated Driving

Conditionally automated vehicles require the out-of-the-loop driver to intervene when the system is unable to handle forthcoming situations, such as freeway exiting. The takeover request (ToR) for exiting a freeway can be scheduled in advance. Upon a ToR, the driver needs to gain situation awareness (SA) and resume manual control. This study examined how the ToR lead time affects driver SA for resuming control and when to send the ToR is most appropriate for freeway exiting. A web-based, supervised experiment was conducted with 31 participants. Each participant experienced 12 levels of ToR lead time (6, 8, 10, 12, 14, 16, 18, 20, 25, 30, 45, and 60 s). The results showed positive effects of longer ToR lead times (16–60 s) on driver SA for resuming control to exit from freeways in comparison to shorter ToR lead times (6–14 s), and the effects level off at 16–30 s.

Situation Awareness Measurement in Remotely Controlled Cars.

This study reviews the current information concerning the measurement of the situation awareness (SA) of the teleoperated drivers of remotely controlled cars. The teleoperated drivers who drive these cars are in a remote location, and they control the cars through a communication interface. The objective methods with probes are beneficial in measuring SA on a closed circuit without real traffic. Questions specifically should address the information provided on the road by haptic sensations, such as the slope of the road and the vehicle's speed. Methods for measuring SA that involve probes and interruptions obviously are not suitable for use on public roads. A stable environment for the display and control of the communication interface is suitable for an eye tracker in measuring SA. These features also facilitate the use of subjective observer-rating methods. Both of these methods are suitable for driving on real roads because they are not intrusive. SA research in a real-road environment also should demonstrate how the SA of other drivers is affected by seeing a car without a driver. Given the remote character of driving, cultural differences in cognition may have a significant influence on the SA of the teleoperated driver.

Roadside evaluation of distracted driving – Driver limitations in recognizing traffic light transitions (REDD – LIGHT)

IntroductionDistracted driving is dangerous, widespread and has become one of the top concerns regarding traffic safety. The goals of the REDD – LIGHT study were to naturalistically determine the prevalence of distracted drivers at a red traffic light, as well as to determine the prevalence of specific distractions and their impact on the drivers’ reaction to traffic light transition. MethodsWe observed the drivers of the first two passenger motor vehicles stopped at a red traffic light. The observational variables were divided into three general groups: demographic variables, distractions (in-vehicle, outer-vehicle, and mobile phones), and reaction to traffic light transition. Vehicles falling behind the full length of a car or failing to proceed for more than 2 s at the traffic light transition from red to green were classified as delayed. ResultsOf the 1008 drivers in this study, 608 (60.3%) were observed to be distracted at the red light. In-vehicle distractions were the most common type of distraction (44.8%, n=452), while mobile phone-related distractions were 7.4% (n=75). A total of 126 (12.5%) drivers were delayed at the light transition, and 111 (88.1%) of these drivers were distracted. We identified seven specific distractions (talking on a handheld device, manipulating a handheld device, in-vehicle devices, eating/drinking, reaching, non-specific and specific outer-vehicle distractions) as significantly associated with the delay to the light transition. ConclusionThree out of five drivers engaged in distracted driving when stopped at a traffic light. In addition, approximately nine out of ten delayed drivers were distracted, clearly demonstrating the negative effects of distractions on the driver's situational awareness. These figures are astonishingly high and massively concerning, considering that the data was acquired from a single red light stop along the driver's trip.

Driver Situation Awareness and Perceived Sleepiness during Truck Platoon Driving – Insights from Eye-tracking Data

ABSTRACT Truck platoon driving technology uses vehicle-to-vehicle communication to allow one truck to follow another in an automated fashion. The first vehicle is operated manually, the second vehicle is driven semi-automatically once platoon-mode is activated. In this mode, the driver merely has to monitor traffic. Semi-automated driving in passenger cars has been shown to increase driver sleepiness and reduce situation awareness. The aim of the present study was to gain first insights whether this also applies to semi-automated platoon driving and whether platoon-specific situations pose special visual demands. In a first on-road experiment, ten professional truck drivers experienced a two-vehicle platooning system on a German highway as platoon follower or leader. In addition, all drivers conducted reference drives with a single truck. Driver situation awareness was measured with eye-tracking recordings, perceived sleepiness with subjective ratings. The results showed that the lead vehicle drivers kept their eyes less time on the road ahead as compared to normal truck driving. In particular in situations that required decoupling, drivers (in the lead vehicle as well as in the following vehicle) spent about 40% of fixations on the HMI. That is, situation awareness was reduced, amounting to potentially risky behavior, as the platoon goes blindfolded when both drivers attend to the display. Drivers did not report higher perceived sleepiness in semi-automated platoon drives than in the manual reference drives. Adequate solutions to reduce the time spent looking away from the road are required. Head-up displays should be investigated for this purpose, as they can simplify driver communication and present platoon-specific information while the eyes remain on the road.

Shaping driver-vehicle interaction in autonomous vehicles: How the new in-vehicle systems match the human needs

Autonomous vehicle (AV) technology has brought a shift in the traditional role of the driver. This paper applies a user-centred design approach to designing a new AV interior to better support drivers. Three empirical studies were conducted, involving a total of 92 drivers (with 44 in Study 1, 12 in Study 2, and 36 in Study 3) to explore user needs and requirements in an AV. In Study 1, safety and comfort, together with a variety of non-driving activities, were identified as the principal concerns about future autonomous vehicles. Based on these findings, Study 2 proposes a new rotatable seating position for AVs, with an in-vehicle information display to facilitate users' activities and situational awareness while driving. Study 3 consists of a series of laboratory simulator evaluation studies, and this indicated that drivers in the proposed design condition had better situational awareness in an AV when dealing with take-over situations. Such findings suggest the possibility of applying rear-facing seats in autonomous vehicles to support in-vehicle non-driving activities. Some specific implications of designs to enhance a driver's situational awareness have also been suggested.

Measuring Driver Perception: Combining Eye-Tracking and Automated Road Scene Perception.

ObjectiveTo investigate how well gaze behavior can indicate driver awareness of individual road users when related to the vehicle’s road scene perception.BackgroundAn appropriate method is required to identify how driver gaze reveals awareness of other road users.MethodWe developed a recognition-based method for labeling of driver situation awareness (SA) in a vehicle with road-scene perception and eye tracking. Thirteen drivers performed 91 left turns on complex urban intersections and identified images of encountered road users among distractor images.ResultsDrivers fixated within 2° for 72.8% of relevant and 27.8% of irrelevant road users and were able to recognize 36.1% of the relevant and 19.4% of irrelevant road users one min after leaving the intersection. Gaze behavior could predict road user relevance but not the outcome of the recognition task. Unexpectedly, 18% of road users observed beyond 10° were recognized.ConclusionsDespite suboptimal psychometric properties leading to low recognition rates, our recognition task could identify awareness of individual road users during left turn maneuvers. Perception occurred at gaze angles well beyond 2°, which means that fixation locations are insufficient for awareness monitoring.ApplicationFindings can be used in driver attention and awareness modelling, and design of gaze-based driver support systems.

Associating Vehicles Automation With Drivers Functional State Assessment Systems: A Challenge for Road Safety in the Future

In the near future, vehicles will gradually gain more autonomous functionalities. Drivers’ activity will be less about driving than about monitoring intelligent systems to which driving action will be delegated. Road safety, therefore, remains dependent on the human factor and we should identify the limits beyond which driver’s functional state (DFS) may no longer be able to ensure safety. Depending on the level of automation, estimating the DFS may have different targets, e.g., assessing driver’s situation awareness in lower levels of automation and his ability to respond to emerging hazard or assessing driver’s ability to monitor the vehicle performing operational tasks in higher levels of automation. Unfitted DFS (e.g., drowsiness) may impact the driver ability respond to taking over abilities. This paper reviews the most appropriate psychophysiological indices in naturalistic driving while considering the DFS through exogenous sensors, providing the more efficient trade-off between reliability and intrusiveness. The DFS also originates from kinematic data of the vehicle, thus providing information that indirectly relates to drivers behavior. The whole data should be synchronously processed, providing a diagnosis on the DFS, and bringing it to the attention of the decision maker in real time. Next, making the information available can be permanent or intermittent (or even undelivered), and may also depend on the automation level. Such interface can include recommendations for decision support or simply give neutral instruction. Mapping of relevant psychophysiological and behavioral indicators for DFS will enable practitioners and researchers provide reliable estimates, fitted to the level of automation.

Thinking Ahead of the Tractor: Driver Safety and Situation Awareness

ABSTRACTObjective: Situation awareness has been identified as a key skill in maintaining safety in high-risk, dynamic industries. However, there is a lack of research evaluating situation awareness requirements and error types in agricultural operators. The aim of this study was to explore situation awareness among farmers in the United Kingdom (UK) when operating heavy agricultural machinery.Method: The study used an online qualitative survey to collect data in three main areas: situation awareness requirements, factors impacting situation awareness, and the safety context for UK farmers. A total of 57 farmers completed the survey, responding to seven open-ended questions. Data were analyzed using directed content analysis.Results: The results indicate the importance of situation awareness for safe and efficient work practice in farming. Situation awareness requirements span the machinery system, including awareness of internal cab systems and the status of implements attached to the external machine. Farmers needed to map and monitor the surrounding conditions in addition to considering personal requirements. Data also indicated the impact of a range of factors, including system-based elements such as blind spots, individual factors such as level of fatigue, and issues related to a lack of knowledge and failure to think ahead on the maintenance of situation awareness.Conclusion: This research highlights the situation awareness requirements for operating agricultural machinery in complex and dynamic environments. By taking a human factors approach, utilizing design and practical interventions, it may be possible to both support and enhance farmer situation awareness and, therefore, reduce errors and adverse incidents.

Design of Conditional Driving Automation Variables to Improve Takeover Performance

Abstract Conditional driving automation occasionally decouples drivers from driving to participate in non-driving related tasks. On the other hand, situation awareness of drivers in some driving situations that can not be managed by the automated system is required to avoid accidents. In recent years, it has been observed that surrounding traffic conditions, complexity of the driving scenario, secondary tasks, speed of ego vehicle, and takeover request experience affect takeover performance. However, the scope of influence of the variables is not known in detail. This contribution discusses an approach for systematically designing these variables. The approach utilizes known dimensions of complex dynamical systems and fundamentals of multitasking to design driving scenarios and non-driving related tasks. The effect of the variables are systemically varied to generate different driving situations to better understand their scope and interaction.

Catch My Drift: Elevating Situation Awareness for Highly Automated Driving with an Explanatory Windshield Display User Interface

Broad access to automated cars (ACs) that can reliably and unconditionally drive in all environments is still some years away. Urban areas pose a particular challenge to ACs, since even perfectly reliable systems may be forced to execute sudden reactive driving maneuvers in hard-to-predict hazardous situations. This may negatively surprise the driver, possibly causing discomfort, anxiety or loss of trust, which might be a risk for the acceptance of the technology in general. To counter this, we suggest an explanatory windshield display interface with augmented reality (AR) elements to support driver situation awareness (SA). It provides the driver with information about the car’s perceptive capabilities and driving decisions. We created a prototype in a human-centered approach and implemented the interface in a mixed-reality driving simulation. We conducted a user study to assess its influence on driver SA. We collected objective SA scores and self-ratings, both of which yielded a significant improvement with our interface in good (medium effect) and in bad (large effect) visibility conditions. We conclude that explanatory AR interfaces could be a viable measure against unwarranted driver discomfort and loss of trust in critical urban situations by elevating SA.