Abstract
Effective shared autonomy requires a clear understanding of driver's behavior, which is governed by multiple psychophysiological and environmental variables. Disentangling this intricate web of interactions requires understanding the driver's state and behaviors in different real-world scenarios, longitudinally. Naturalistic Driving Studies (NDS) have shown to be an effective approach to understanding the driver's state and behavior in real-world scenarios. However, due to the lack of technological and computing capabilities, former NDS only focused on vision-based approaches, ignoring important psychophysiological factors such as cognition and emotion. The main objective of this paper is to introduce HARMONY, a human-centered multimodal naturalistic driving study, where driver's behaviors and states are monitored through (1) in-cabin and outside video streams (2) physiological signals including driver's heart rate and hand acceleration (IMU data), (3) ambient noise, light, and the vehicle's GPS location, and (4) music logs, including song features such as tempo. HARMONY is the first study that collects long-term naturalistic facial, physiological, and environmental data simultaneously. This paper summarizes HARMONY's goals, framework design, data collection and analysis, and the on-going and future research efforts. Through a presented case study, we first demonstrate the importance of longitudinal driver state sensing through using Kernel Density Estimation Methods. Second, we leverage the application of Bayesian Change Point detection methods to demonstrate how we can identify driver behaviors and responses to the environmental conditions by fusing psychophysiological information with features extracted from video streams.
Highlights
Autonomous Vehicles (AV) are improving at a very fast rate, it is predicted that through shared autonomy, humans will be involved in driving decision making for the foreseeable future [1], [2]
BACKGROUND we first review the different categories of previous driver-in-the-loop studies; section II-A provides an overview of conducted studies through driving simulators, section II-B focuses on on-road controlled studies, and section II-C evaluates the existing Field Operational Tests (FOT) and Naturalistic Driving Studies (NDS)
WORK The vehicle industry is advancing very quickly and new technologies are introduced to automate different aspects of driving. These improvements are intending to enhance the driver’s safety and comfort, they still lack an understanding of driver states and behaviors
Summary
Autonomous Vehicles (AV) are improving at a very fast rate, it is predicted that through shared autonomy, humans will be involved in driving decision making for the foreseeable future [1], [2]. Previous studies have pointed out that external factors such as weather conditions [18], and road geometry and design [19] impact driver’s state and behavior Most of these Naturalistic Driving Studies (NDS) rely on features collected from video cameras capturing in-cabin and outdoor conditions. The term psychophysiological refers to psychological states such as emotional responses (e.g., anger, frustration, and happiness), cognitive load, and distraction that can be measured through changes in human physiology responses (e.g., heart rate, skin temperature, and skin conductance) [25] They highlighted the data required for validating their model does not currently exist, and they had to rely on previous literature for retrieving probability conditions of different driver states under various environmental conditions [24]. We discuss the current limitations and the future road map of HARMONY
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
