Driver State Research Articles

Cortical dynamics of automated driving across the spectrum of cognitive aging: A proof-of-concept study

BackgroundAutomated vehicle (AV) technology may potentially improve road safety, independence, and mobility in older drivers with cognitive impairment (CI). Physiological sensors will be useful to provide feedback of the driver state to the AV controller system. The aim of this study was to compare changes in cortical activity recorded using electro-encephalography (EEG) during conditional automation between older adults with normal cognition (NC) and CI. MethodsFourteen older adults (age 72.43 (6.00); 10 (71%) women; nine with NC and five with CI) completed two driving simulator scenarios. Scenario 1 included conditional automated driving with an emergency take-over request (TOR) at the end of the drive. Scenario 2 was identical but with an additional non-driving related task (NDRT). EEG was recorded in three driver tasks: resting; automation; and TOR. The interaction effect of driver task (resting state – automated driving – TOR) and cognitive status (CI – NC) on frontal theta power was calculated using linear mixed models. ResultsA significant interaction effect of driver task*cognitive status on frontal theta power was found in the scenario without NDRT (F = 5.85; p = 0.001). Post-hoc effects showed that older adults with CI showed greater changes in frontal theta power from resting state to the TOR event (β = 16.44 ± 6.35; p = 0.01) and from automated driving to the TOR event (β = 17.91 ± 6.11; p = 0.006) compared to controls. No interaction effects were found on frontal theta power in the scenario with NDRT. ConclusionsOur study provides proof-of-concept that older adults with CI show increased cortical activity while engaging with AV technology compared to neurotypical older adults.

A Feature Fusion Method for Driving Fatigue of Shield Machine Drivers Based on Multiple Physiological Signals and Auto-Encoder

The driving fatigue state of shield machine drivers directly affects the safe operation and tunneling efficiency of shield machines during metro construction. To cope with the problem that it is challenging to simulate the working conditions and operation process of shield machine drivers using driving simulation platforms and that the existing fatigue feature fusion methods usually show low recognition accuracy, shield machine drivers at Shenyang metro line 4 in China were taken as the research subjects, and a multi-modal physiological feature fusion method based on an L2-regularized stacked auto-encoder was designed. First, the ErgoLAB cloud platform was used to extract the combined energy feature (E), the reaction time, the HRV (heart rate variability) time-domain SDNN (standard deviation of normal-to-normal intervals) index, the HRV frequency-domain LF/HF (energy ratio of low frequency to high frequency) index and the pupil diameter index from EEG (electroencephalogram) signals, skin signals, pulse signals and eye movement data, respectively. Second, the physiological signal characteristics were extracted based on the WPT (wavelet packet transform) method and time–frequency analysis. Then, a method for driving fatigue feature fusion based on an auto-encoder was designed aiming at the characteristics of the L2-regularization method to solve the over-fitting problem of small sample data sets in the process of model training. The optimal hyper-parameters of the model were verified with the experimental method of the control variable, which reduces the loss of multi-modal feature data in compression fusion and the information loss rate of the fused index. The results show that the method proposed outperforms its competitors in recognition accuracy and can effectively reduce the loss rate of deep features in existing decision-making-level fusion.

Drivers’ usage of driving automation systems in different contexts: A survey in China, Germany, Spain and USA

AbstractThe purpose of the study was to investigate how drivers use assisted and automated driving systems (DAS), more specifically their usage of SAE Level 1 and Level 2 systems, in different situations. An online survey was distributed to 2500 respondents in China, Germany, Spain, and the USA. The final dataset consisted of 549 respondents, all non‐professional drivers, with access to a minimum of a Level 1 system. A subset, 159 in total, had access also to a Level 2 DAS. The survey included questions on the attitude towards, access to, and usage of Level 1 and Level 2 systems in nine different situations respectively. The data was analysed on an individual and a national level. A cluster analysis showed two main groups: frequent and non‐frequent users. On an individual level, the reported usage of Level 1 and Level 2 DAS respectively differed depending on traffic situation, weather and daylight conditions and driver state. Reports by respondents with access to both Level 1 and Level 2 systems did not reveal any statistically significant differences in usage between situations. The Spanish sample was the only one that showed a consistently different usage pattern compared to samples from China, Germany, and the USA.

Using Facial Features to Detect Driver’s Drowsiness State

Abstract: Drowsiness and fatigue are one of the main causes leading to road accidents. They can be prevented by taking effort to get enough sleep before driving, drink coffee or energy drink, or have a rest when the signs of drowsiness occur. The popular drowsiness detection method uses complex methods, such as EEG and ECG [19]. This method has high accuracy for its measurement but it need to use contact measurement and it has many limitations on driver fatigue and drowsiness monitor. Thus, it is not comfortable to be used in real time driving. This paper proposes a way to detect the drowsiness signs among drivers by measuring the eye closing rate and yawning. We provide a robust and intelligent strategy for detecting driver tiredness in this work to address this growing problem. This method involves installing a camera inside the car to record the driver's facial look. The first phase involves using computer vision algorithms to identify and track the face region in the recorded video sequence. After that, the head is removed and its lateral and frontal assent are examined for indications of driver weariness. The driver's state is finally assessed during the fusion phase, and if drowsiness is found, a warning message and an alert are given to the driver. Our tests provide strong support for the proposed theory. The parameters of the eyes and mouth detection are created within the face image. The video was change into images frames per second. From there, locating the eyes and mouth can be performed. Once the eyes are located, measuring the intensity changes in the eye area determine the eyes are open or closed. The major goal of this research is to create a non-intrusive system that can recognise driver weariness and deliver a prompt warning.

Driving Alert System based on Facial Expression Recognition

Abstract: Inattention is one of the major factors and causes of many road accidents and unforeseen crashes. As a result, it is vital to develop an automatic warning system for drivers, which will send timely warning signals to them. Based on the driver's facial expressions, this issue involves determining the driver's mental state. In applications such as driver warning systems, automated facial emotion recognition has become a recent development in the image processing world. There are existing methods for recognizing facial emotions regardless of whether the signal is noisy or imperfect data, but ultimately it lacks accuracy. It is also useless and not getting proper output in dealing with spontaneous feeling, and recognition. The proposed approach develops a driver warning system that extracts the facial expressions based on a novel efficient Local Octal Pattern (LOP) and effectively recognizes the facial expressions based on Deep Neural Networks, Convolutional Neural Networks (CNN). The LOP feature map provides as an input to CNN and guides in the selection of CNN learning data thereby improving and further enhancing the understanding and learning of CNN. It also has an ability to recognize both natural and spontaneous feeling, considered input as a image as well as video. The experimental results considering YawDD dataset indicates that the proposed system has been efficiently evaluated by considering the with metrics such as Precision, Recall and F-Score and thereby it is observed and inferred that the proposed system obtained a high recall rate of 96.09% in comparison with the other state-of-the-art methods

Study on Driver Cross-Subject Emotion Recognition Based on Raw Multi-Channels EEG Data

In our life, emotions often have a profound impact on human behavior, especially for drivers, as negative emotions can increase the risk of traffic accidents. As such, it is imperative to accurately discern the emotional states of drivers in order to preemptively address and mitigate any negative emotions that may otherwise manifest and compromise driving behavior. In contrast to many current studies that rely on complex and deep neural network models to achieve high accuracy, this research aims to explore the potential of achieving high recognition accuracy using shallow neural networks through restructuring the structure and dimensions of the data. In this study, we propose an end-to-end convolutional neural network (CNN) model called simply ameliorated CNN (SACNN) to address the issue of low accuracy in cross-subject emotion recognition. We extracted features and converted dimensions of EEG signals from the SEED dataset from the BCMI Laboratory to construct 62-dimensional data, and obtained the optimal model configuration through ablation experiments. To further improve recognition accuracy, we selected the top 10 channels with the highest accuracy by separately training the EEG data of each of the 62 channels. The results showed that the SACNN model achieved an accuracy of 88.16% based on raw cross-subject data, and an accuracy of 91.85% based on EEG channel data from the top 10 channels. In addition, we explored the impact of the position of the BN and dropout layers on the model through experiments, and found that a targeted shallow CNN model performed better than deeper and larger perceptual field CNN models. Furthermore, we discuss herein the future issues and challenges of driver emotion recognition in promising smart city applications.

Real-Time Drowsiness Identification Based On Eye State Analysis

As per the previous year’s report concerning to road crashes indicates that the principal cause of such a fatal road accidents is because of negligence behavior as well as drowsiness of driver. This problem reveals the requirement of such a system that can recognize drowsiness state of driver and gives alert signal to the driver before the occurrence of any accidents. Therefore, this proposed work has established drowsy detection as well as accident avoidance system based on the eye blink duration. Here, first the open and close state of eye are detected based on the eye aspect ratio (EAR). Further, the blink duration or count during the changes of eye state from open to close are analyzed. Then, it identifies the state of drowsiness, when blink duration becomes more than a certain limits and sends the alert message to the driver through the alarm. Our developed system has shown the accuracy of 92.5 % approx on yawning dataset (YawDD). Key Word: Eye blink detection; Drowsiness Detection; Eye Aspect Ratio (EAR)

Multi-scale spatial–temporal attention graph convolutional networks for driver fatigue detection

With the development of deep learning, fatigue detection technology for drivers has achieved remarkable achievements. Although the image-based approach achieves good accuracy, it inevitably leads to greater model complexity, which is unsuitable for mobile terminal devices. Luckily, human skeletal data significantly reduces the impact of noise and input data volume while retaining valid information, and it can better deal with real-world driving scenarios with the benefit of robustness in complex driving situations. This paper proposes a lightweight multi-scale spatio-temporal attention graph convolutional network (MS-STAGCN) to efficiently utilize skeleton data to identify driver states by aggregating locally and globally valid face information, which achieves good performance even for lightweight design. The experimental results show that the method achieves 92.4% accuracy on the NTHU-DDD dataset, which can be applied to fatigue detection tasks of the driver in real-world driving scenarios in the future.

Silent failure detection in partial automation as a function of visual attentiveness

Objective Drivers using level 2 automation are able to disengage with the dynamic driving task, but must still monitor the roadway and environment and be ready to takeover on short notice. However, people are still willing to engage with non-driving related tasks, and the ways in which people manage this tradeoff are expected to vary depending on the operational design domain of the system and the nature of the task. Our aim is to model driver gaze behavior in level 2 partial driving automation when the driver is engaged in an email task on a cell phone. Both congested highway driving, traffic jams, and a hazard with a silent automation failure are considered in a driving simulator study conducted in the NADS-1 high-fidelity motion-based driving simulator. Methods Sequence analysis is a methodology that has grown up around social science research questions. It has developed into a powerful tool that supports intuitive visualizations, clustering analysis, covariate analyses, and Hidden Markov Models. These methods were used to create models for four different gaze behaviors and use the models to predict attention during the silent failure event. Results Predictive simulations were run with initial conditions that matched driver state just prior to the silent failure event. Actual gaze response times were observed to fall within distributions of predicted glances to the front. The three drivers with the largest glance response times were not able to take back manual control before colliding with the hazard. Conclusions The simulated glance response time distributions can be used in more sophisticated ways when combined with other data. The glance response time probability may be conditioned on other variables like time on task, time of day, prevalence of the current behavior for this driver, or other variables. Given the flexibility of sequence analysis and the methods it supports (clustering, HMMs), future studies may benefit from its application to gaze behavior and driving performance data.

Predicting Driver Errors during Automated Vehicle Takeovers

The transition of control between partially automated vehicles and drivers is an important part of the operational design domain and poses unique and important design issues. One approach for enhancing the design of the transition of control mechanisms is to predict driver behavior during a takeover by analyzing his/her state before a takeover. Although there is a wealth of existing literature on modeling the prediction of driver behavior, little is known about the prediction of takeover performance (e.g., driver error) and its underlying data structure (e.g., window sizes or the inclusion of certain features). Thus, the goal of this study is to predict driver error during a takeover event using supervised machine learning algorithms for various window sizes. Three machine learning algorithms (i.e., decision tree, random forest, and support vector machine with a radial basis kernel) were applied to granular driving performance, physiological, and glance data from a driving simulator experiment examining automated vehicle driving. The results showed that a random forest algorithm with an area under the receiver operating characteristic curve of 0.72, trained on a 3 s window before the takeover time, had the best performance with regard to classifying driver error accurately. In addition, we identified the 10 most important predictors that resulted in the best error prediction performance. The results of this study could be useful in developing algorithms for driver state that could be integrated into highly automated systems and, potentially, improve the takeover process.

The Effect of Vehicle Automation Styles on Drivers’ Emotional State

Self-driving vehicles promise many safety, mobility, and environmental benefits. However, users’ lack of trust and acceptance may threaten the success and potential of this technology. Monitoring the driver’s emotional state is one way to address this challenge. Empathetic automation can respond to the driver’s state and improve the experience and acceptance of self-driving vehicle drivers. In this study, 24 participants rode in a self-driving vehicle simulator and experienced three automation styles (aggressive, moderate, conservative) and four intersection types (with and without a stop sign, and with and without traffic.) We identified the observed drivers’ emotions from the video data and labeled the video frames using the dimensional and discrete emotion models to examine how automation behavior affects the driver’s emotional state. We used multilevel Bayesian linear regression and multilevel Dirichlet regression to model the continuous and discrete emotions, respectively. The automation driving style effect varied for each participant. The same conditions provoked positive responses for some participants, and negative for others. Furthermore, the results showed that intersection type, the position within the intersection, and their interaction affected the driver’s emotional state. This indicates that personalized driver state monitoring systems might enhance drivers’ experience in self-driving vehicles.

EEG driving fatigue detection based on log-Mel spectrogram and convolutional recurrent neural networks.

Driver fatigue detection is one of the essential tools to reduce accidents and improve traffic safety. Its main challenge lies in the problem of how to identify the driver's fatigue state accurately. Existing detection methods include yawning and blinking based on facial expressions and physiological signals. Still, lighting and the environment affect the detection results based on facial expressions. In contrast, the electroencephalographic (EEG) signal is a physiological signal that directly responds to the human mental state, thus reducing the impact on the detection results. This paper proposes a log-Mel spectrogram and Convolution Recurrent Neural Network (CRNN) model based on EEG to implement driver fatigue detection. This structure allows the advantages of the different networks to be exploited to overcome the disadvantages of using them individually. The process is as follows: first, the original EEG signal is subjected to a one-dimensional convolution method to achieve a Short Time Fourier Transform (STFT) and passed through a Mel filter bank to obtain a logarithmic Mel spectrogram, and then the resulting logarithmic Mel spectrogram is fed into a fatigue detection model to complete the fatigue detection task for the EEG signals. The fatigue detection model consists of a 6-layer convolutional neural network (CNN), bi-directional recurrent neural networks (Bi-RNNs), and a classifier. In the modeling phase, spectrogram features are transported to the 6-layer CNN to automatically learn high-level features, thereby extracting temporal features in the bi-directional RNN to obtain spectrogram-temporal information. Finally, the alert or fatigue state is obtained by a classifier consisting of a fully connected layer, a ReLU activation function, and a softmax function. Experiments were conducted on publicly available datasets in this study. The results show that the method can accurately distinguish between alert and fatigue states with high stability. In addition, the performance of four existing methods was compared with the results of the proposed method, all of which showed that the proposed method could achieve the best results so far.

An Electro-Oculogram (EOG) Sensor's Ability to Detect Driver Hypovigilance Using Machine Learning.

Driving safely is crucial to avoid death, injuries, or financial losses that can be sustained in an accident. Thus, a driver's physical state should be monitored to prevent accidents, rather than vehicle-based or behavioral measurements, and provide reliable information in this regard. Electrocardiography (ECG), electroencephalography (EEG), electrooculography (EOG), and surface electromyography (sEMG) signals are used to monitor a driver's physical state during a drive. The purpose of this study was to detect driver hypovigilance (drowsiness, fatigue, as well as visual and cognitive inattention) using signals collected from 10 drivers while they were driving. EOG signals from the driver were preprocessed to remove noise, and 17 features were extracted. ANOVA (analysis of variance) was used to select statistically significant features that were then loaded into a machine learning algorithm. We then reduced the features by using principal component analysis (PCA) and trained three classifiers: support vector machine (SVM), k-nearest neighbor (KNN), and ensemble. A maximum accuracy of 98.7% was obtained for the classification of normal and cognitive classes under the category of two-class detection. Upon considering hypovigilance states as five-class, a maximum accuracy of 90.9% was achieved. In this case, the number of detection classes increased, resulting in a reduction in the accuracy of detecting more driver states. However, with the possibility of incorrect identification and the presence of issues, the ensemble classifier's performance produced an enhanced accuracy when compared to others.

Fatigue driving detection method based on Time-Space-Frequency features of multimodal signals

Fatigue detection for drivers in public transportation is crucial. To effectively detect the driver's fatigue state, we investigated the deep learning-based fatigue detection method and proposed a multimodality signal fatigue detection method. In the proposed method, the convolutional autoencoder (CAE) is used to fuse electroencephalogram (EEG) and electrooculography (EOG) signal features, and the convolutional neural network (CNN) is used to maintain spatial locality. After that, the fusion features are input into the recurrent neural network (RNN) for fatigue recognition. We tested the proposed algorithmic framework on the SEED-VIG dataset and evaluated it using two statistical indicators, root mean square error (RMSE) and correlation coefficient (COR), achieving the mean RMSE/COR of 0.10/0.93 and 0.11/0.88 on single modality EOG and EEG features, respectively, and improved performance to 0.08/0.96 on multimodality features. In addition, this paper analyzes the effect of different signal features on recognition results, and the comparison illustrates that the performance of the model using multimodality features is better than that of signle modality features. The experimental results show that the algorithm framework proposed in this paper outperforms other recognition algorithms, which also proves the effectiveness of the algorithm applied to fatigue driving detection.

Using pupillometry and gaze-based metrics for understanding drivers’ mental workload during automated driving

This Horizon2020-funded driving simulator-based study on automated driving investigated the effect of different car-following scenarios, and takeover situations, on drivers’ mental workload, as measured by eye tracking-based metrics of pupil diameter and self-reported workload ratings. This study incorporated a mixed design format, with 16 drivers recruited for the SAE Level 2 (L2; SAE International, 2021) automation group, who were asked to monitor the driving and road environment during automation, and 16 drivers in the Level 3 (L3) automation group, who engaged in a non-driving related task (NDRT; Arrows task) during automation. Drivers in each group undertook two experimental drives, lasting about 18 min each. To manipulate perceived workload, difficulty of the driving task was controlled by incorporating a lead vehicle which maintained either a Short (0.5 s) or Long (1.5 s) Time Headway (THW) condition during automated car-following (ACF). Each ACF session was followed by a subsequent request to takeover, which happened either in the presence or absence of a lead vehicle. Results from standard deviation of pupil diameter values indicated that drivers’ mental workload levels fluctuated significantly more when monitoring the drive during L2 ACF, compared to manual car-following (MCF). Additionally, we found that drivers’ mental workload, as indicated by their mean pupil diameter, increased steeply around takeovers, and was further exacerbated by the presence of a lead vehicle during the takeovers, especially in the Short THW condition, for both groups. Pupil diameter was found to be sensitive to subtle variations in mental workload, and closely resembled the trend seen in self-reported workload ratings. Further research is warranted to assess the feasibility of using eye-tracking-based metrics along with other physiological sensors, especially in real-world settings, to understand whether they can be used as real-time indicators of drivers’ mental workload, in future driver state monitoring systems.

EEG-based driver states discrimination by noise fraction analysis and novel clustering algorithm.

Driver states are reported as one of the principal factors in driving safety. Distinguishing the driving driver state based on the artifact-free electroencephalogram (EEG) signal is an effective means, but redundant information and noise will inevitably reduce the signal-to-noise ratio of the EEG signal. This study proposes a method to automatically remove electrooculography (EOG) artifacts by noise fraction analysis. Specifically, multi-channel EEG recordings are collected after the driver experiences a long time driving and after a certain period of rest respectively. Noise fraction analysis is then applied to remove EOG artifacts by separating the multichannel EEG into components by optimizing the signal-to-noise quotient. The representation of data characteristics of the EEG after denoising is found in the Fisher ratio space. Additionally, a novel clustering algorithm is designed to identify denoising EEG by combining cluster ensemble and probability mixture model (CEPM). The EEG mapping plot is used to illustrate the effectiveness and efficiency of noise fraction analysis on the denoising of EEG signals. Adjusted rand index (ARI) and accuracy (ACC) are used to demonstrate clustering performance and precision. The results showed that the noise artifacts in the EEG were removed and the clustering accuracy of all participants was above 90%, resulting in a high driver fatigue recognition rate.

(Mis-)use of standard Autopilot and Full Self-Driving (FSD) Beta: Results from interviews with users of Tesla's FSD Beta.

Tesla's Full Self-Driving Beta (FSD) program introduces technology that extends the operational design domain of standard Autopilot from highways to urban roads. This research conducted 103 in-depth semi-structured interviews with users of Tesla's FSD Beta and standard Autopilot to evaluate the impact on user behavior and perception. It was found that drivers became complacent over time with Autopilot engaged, failing to monitor the system, and engaging in safety-critical behaviors, such as hands-free driving, enabled by weights placed on the steering wheel, mind wandering, or sleeping behind the wheel. Drivers' movement of eyes, hands, and feet became more relaxed with experience with Autopilot engaged. FSD Beta required constant supervision as unfinished technology, which increased driver stress and mental and physical workload as drivers had to be constantly prepared for unsafe system behavior (doing the wrong thing at the worst time). The hands-on wheel check was not considered as being necessarily effective in driver monitoring and guaranteeing safe use. Drivers adapt to automation over time, engaging in potentially dangerous behaviors. Some behavior seems to be a knowing violation of intended use (e.g., weighting the steering wheel), and other behavior reflects a misunderstanding or lack of experience (e.g., using Autopilot on roads not designed for). As unfinished Beta technology, FSD Beta can introduce new forms of stress and can be inherently unsafe. We recommend future research to investigate to what extent these behavioral changes affect accident risk and can be alleviated through driver state monitoring and assistance.

Driver Vigilance State Estimation Based on Multisource Data

Accurate estimation of driver vigilance state is important for improving driving safety and preventing accidents. Previous studies have extensively investigated the potential of driving performance measures (e.g., lane position, steering wheel angle, and lateral acceleration) and eye movement measures (e.g., glance frequency and duration) on the driver’s vigilance state estimation. However, the change in these measures may be caused by various factors. Therefore, comprehensively involving more direct measures (e.g., drivers’ electroencephalogram (EEG) response) is expected to increase the vigilance estimation performance. This study comprehensively uses measures across different functional domains (i.e., driving performance, eye movement, and EEG) to establish an estimation model of the driver’s vigilance state. Twenty participants were assigned a 2-h sustained-attention driving task in a simulator, in which they followed a lead car closely. According to their brake reaction times (RTs) to the brake signal of the lead vehicle, their vigilance states were divided into three levels (i.e., low, medium, and high). A logistic regression model that estimates drivers’ vigilance state level based on multisource data was established. This model is a promising approach to driver states detection and reveals the factors that should be considered in subsequent model development processes. It can be combined with driver vigilance arousal methods to improve driving safety.

Young Novice Drivers' Cognitive Distraction Detection: Comparing Support Vector Machines and Random Forest Model of Vehicle Control Behavior.

The use of mobile phones has become one of the major threats to road safety, especially in young novice drivers. To avoid crashes induced by distraction, adaptive distraction mitigation systems have been developed that can determine how to detect a driver's distraction state. A driving simulator experiment was conducted in this paper to better explore the relationship between drivers' cognitive distractions and traffic safety, and to better analyze the mechanism of distracting effects on young drivers during the driving process. A total of 36 participants were recruited and asked to complete an n-back memory task while following the lead vehicle. Drivers' vehicle control behavior was collected, and an ANOVA was conducted on both lateral driving performance and longitudinal driving performance. Indicators from three aspects, i.e., lateral indicators only, longitudinal indicators only, and combined lateral and longitudinal indicators, were inputted into both SVM and random forest models, respectively. Results demonstrated that the SVM model with parameter optimization outperformed the random forest model in all aspects, among which the genetic algorithm had the best parameter optimization effect. For both lateral and longitudinal indicators, the identification effect of lateral indicators was better than that of longitudinal indicators, probably because drivers are more inclined to control the vehicle in lateral operation when they were cognitively distracted. Overall, the comprehensive model built in this paper can effectively identify the distracted state of drivers and provide theoretical support for control strategies of driving distraction.

Effect of emotion on galvanic skin response and vehicle control data during simulated driving

PurposeThis study aims to investigate the effect of driver emotion on the physiological and vehicle control data and the possibility of predicting the driver emotion to enhance the driving experience. MethodThe driver emotions in the driving context were classified into eight categories depending on high and low levels of arousal and valence: happiness, surprise, fear, anger, depression, boredom, relief, and neutrality. Fourteen male and female volunteers with ages between 22 and 34 y participated in the test, and approximately 540 min of image, physiological, and vehicle data were collected. After inducing the participants’ target emotion through the viewing of a film and writing of passages, we asked the participants to drive on a highway through a driving simulator and self-evaluate their emotion. After the test, the participants were allowed to go home after emotion neutralization. ResultThe participants’ self-evaluated emotions correlated highly with the intended induced emotions. High arousal and negative valence emotions such as anger and low arousal and positive valence emotions such as relief exhibited a statistically significant elevation for the following indicators: galvanic skin response amplitude, longitudinal vehicle control data such as throttling and braking, and lateral vehicle control data. ConclusionsThe test results confirmed that a driver’s emotional state could be reflected in the differences between the biometric data or vehicle control data. In particular, the emotion associated with a high arousal and negative valence could be clearly distinguished from that associated with a low arousal and positive valence. Therefore, the driver's emotional state affects the traffic condition, and the detection of potentially risky emotions such as those associated with road rage and development of a suitable driving mode can help enhance the driving safety. ApplicationsDrivers’ emotions can be identified based on physiological data and vehicle control data and integrated into the system to formulate appropriate responses. For instance, anger anticipated from a driver can be alleviated by a preemptive measure, thereby enhancing the traffic safety.