Driver Cognitive Load Research Articles

Investigating Blind Spot Design Effects on Drivers’ Cognitive Load with Lane Changing: A Comparative Experiment with Multiple Types of Intelligent Vehicles

Lane changing is a frequent traffic accident scenario. To improve the driving safety in lane changing scenarios, the blind spot display of lane changing is increased through human–machine interaction (HMI) interfaces in intelligent vehicles to improve the driver’s rate of risk perception with regard to the driving environment. However, blind spot information will increase the cognitive load of drivers and lead to driving distraction. To quantify the coupling relationship between blind spot display and drivers’ cognitive load, we proposed a method to quantify the cognitive load of the driver’s interaction by improving the AttenD algorithm, collecting feature data by carrying out a variety of real-vehicle road-testing experiments on three kinds of intelligent vehicles, and then establishing a model blind spot design and driver cognitive load correlation model using Bayesian Logistic Ordinal Regression (BLOR) and Categorical Boosting (CatBoost). The results show that the blind spot image display can reduce the driver’s cognitive load more effectively as it is closer to the driver, has a larger area, and occupies a higher proportion of the center control screen, especially when it is located in the middle and upper regions of the center control screen. The improved AttenD algorithm is able to quantify the cognitive load of the driver, which can be widely used in vehicle testing, HMI interface development and evaluation. In addition, the analytical framework constructed in this paper can help us to understand the complex impact of HMI in intelligent vehicles and provide optimization criteria for lane change blind spot design.

Cognitive Workload Estimation in Conditionally Automated Vehicles Using Transformer Networks Based on Physiological Signals

Though driving automation promises to improve driving safety, drivers are still required to be ready to retake control in conditionally automated vehicles, which are defined by the Society of Automotive Engineers (SAE) as SAE L3 vehicles. Thus, drivers’ states can still affect driving safety in SAE L3 vehicles. Previous research found that a high cognitive load may impair drivers’ takeover performance. Thus, it is still necessary to estimate drivers’ cognitive load in SAE L3 vehicles. However, existing driver cognitive load estimation algorithms mostly focus on vehicles with a lower level of driving automation (e.g., SAE L0), which may not be relevant when estimating driver states in SAE L3 vehicles, given that drivers’ responsibilities are different, and several commonly used measures (e.g., driving performance) are unavailable when drivers are not continuously controlling the vehicle. Further, previous driver cognitive load estimation algorithms rarely considered the temporal information in the input features. Thus, we proposed a deep-learning algorithm to estimate driver cognitive load in SAE L3 vehicles, which integrated multiple physiological features (i.e., electrocardiogram, electrodermal activity, respiration) and considered the temporal correlation of the data using a transformer-encoder-based network. The performance of our algorithm was compared with baseline models on an open data set. Results showed that our algorithm outperformed baseline models and achieved an accuracy of 94.4% using within-subject data partition (proportionally splitting data from the same subject into the training and testing data sets) and an accuracy of 89% using across-subjects data partition (dividing the training and testing data sets based on individual subjects).

Classification of Driver Cognitive Load in Conditionally Automated Driving: Utilizing Electrocardiogram-Based Spectrogram with Lightweight Neural Network

With the development of conditionally automated driving, drivers will be allowed to perform non-driving-related tasks. Under such circumstances, continuous monitoring of driver cognitive load will play an increasingly important role in ensuring that drivers have sufficient mental resources to take over control of the vehicle should the driving automation fail. However, estimation of cognitive load is challenging because of the difficulties in identifying high-level feature representation and accounting for interindividual differences. Physiological measures are believed to be promising candidates for cognitive load estimation in partially automated vehicles. However, current estimation methods are mainly based on the manual feature extraction of time- or frequency-domain indicators from physiological signals, which may not adapt to dynamic driving conditions. With the development of deep learning, the neural network has shown good performance in automatically capturing high-level features from input data. Inspired by this, we adopted a novel approach to classify driver cognitive load based on electrocardiogram (ECG) spectrograms, in which the driver’s ECG signal was collected and transformed into a 2D spectrogram by a short-time Fourier transform. A squeeze-and-excitation network-based deep-learning framework that can capture high-level features and pays more attention to the cognition-related features of the spectrogram was proposed for classification. Experiments on a publicly available dataset demonstrated that our model achieved an accuracy of 96.76% in differentiating two levels of cognitive load for a within-subject evaluation and 71.50% accuracy with an across-subjects evaluation. The results demonstrated the feasibility of detecting drivers’ cognitive load through deep learning using ECG spectrogram alone.

Multimodal Brain–Computer Interface for In-Vehicle Driver Cognitive Load Measurement: Dataset and Baselines

Multimodal Brain–Computer Interface for In-Vehicle Driver Cognitive Load Measurement: Dataset and Baselines

A review of driver cognitive load detection using ECG signals

Detection of the driver’s cognitive load while driving is crucial to prevent the likelihood of traffic collisions and improve road safety. A physiological-based approach has gained significant attention due to its potential to provide reliable indicators for the driver’s state. The physiological signal of electrocardiography (ECG) is considered a promising biomarker for detecting the driver’s cognitive load. Despite the interest in cognitive load detection using ECG, an attempt has yet to be made to identify the relationship between ECG measures and driver cognitive load level. This paper seeks to investigate this gap in cognitive load literature. The finding demonstrates that further research is still needed on ECG-based driver’s cognitive load detection by examining and analyzing the limitations of research challenges and earlier studies. This study also addresses the performance and problems faced in the detection of a driver’s cognitive load considering ECG. With a better understanding of how cognitive load affects ECG measures, both researchers and companies can design more effective driver’s state detection systems.

Classification of Driver Cognitive Load: Exploring the Benefits of Fusing Eye-Tracking and Physiological Measures

In-vehicle infotainment systems can increase cognitive load and impair driving performance. These effects can be alleviated through interfaces that can assess cognitive load and adapt accordingly. Eye-tracking and physiological measures that are sensitive to cognitive load, such as pupil diameter, gaze dispersion, heart rate (HR), and galvanic skin response (GSR), can enable cognitive load estimation. The advancement in cost-effective and nonintrusive sensors in wearable devices provides an opportunity to enhance driver state detection by fusing eye-tracking and physiological measures. As a preliminary investigation of the added benefits of utilizing physiological data along with eye-tracking data in driver cognitive load detection, this paper explores the performance of several machine learning models in classifying three levels of cognitive load imposed on 33 drivers in a driving simulator study: no external load, lower difficulty 1-back task, and higher difficulty 2-back task. We built five machine learning models, including k-nearest neighbor, support vector machine, feedforward neural network, recurrent neural network, and random forest (RF) on (1) eye-tracking data only, (2) HR and GSR, (3) eye-tracking and HR, (4) eye-tracking and GSR, and (5) eye-tracking, HR, and GSR. Although physiological data provided 1%–15% lower classification accuracies compared with eye-tracking data, adding physiological data to eye-tracking data increased model accuracies, with an RF classifier achieving 97.8% accuracy. GSR led to a larger boost in accuracy (29.3%) over HR (17.9%), with the combination of the two factors boosting accuracy by 34.5%. Overall, utilizing both physiological and eye-tracking measures shows promise for driver state detection applications.

Unsupervised fNIRS feature extraction with CAE and ESN autoencoder for driver cognitive load classification

Objective. Understanding the cognitive load of drivers is crucial for road safety. Brain sensing has the potential to provide an objective measure of driver cognitive load. We aim to develop an advanced machine learning framework for classifying driver cognitive load using functional near-infrared spectroscopy (fNIRS). Approach. We conducted a study using fNIRS in a driving simulator with the N-back task used as a secondary task to impart structured cognitive load on drivers. To classify different driver cognitive load levels, we examined the application of convolutional autoencoder (CAE) and Echo State Network (ESN) autoencoder for extracting features from fNIRS. Main results. By using CAE, the accuracies for classifying two and four levels of driver cognitive load with the 30 s window were 73.25% and 47.21%, respectively. The proposed ESN autoencoder achieved state-of-art classification results for group-level models without window selection, with accuracies of 80.61% and 52.45% for classifying two and four levels of driver cognitive load. Significance. This work builds a foundation for using fNIRS to measure driver cognitive load in real-world applications. Also, the results suggest that the proposed ESN autoencoder can effectively extract temporal information from fNIRS data and can be useful for other fNIRS data classification tasks.

Towards Intelligent Data Analytics: A Case Study in Driver Cognitive Load Classification.

One debatable issue in traffic safety research is that the cognitive load by secondary tasks reduces primary task performance, i.e., driving. In this paper, the study adopted a version of the n-back task as a cognitively loading secondary task on the primary task, i.e., driving; where drivers drove in three different simulated driving scenarios. This paper has taken a multimodal approach to perform ‘intelligent multivariate data analytics’ based on machine learning (ML). Here, the k-nearest neighbour (k-NN), support vector machine (SVM), and random forest (RF) are used for driver cognitive load classification. Moreover, physiological measures have proven to be sophisticated in cognitive load identification, yet it suffers from confounding factors and noise. Therefore, this work uses multi-component signals, i.e., physiological measures and vehicular features to overcome that problem. Both multiclass and binary classifications have been performed to distinguish normal driving from cognitive load tasks. To identify the optimal feature set, two feature selection algorithms, i.e., sequential forward floating selection (SFFS) and random forest have been applied where out of 323 features, a subset of 42 features has been selected as the best feature subset. For the classification, RF has shown better performance with F1-score of 0.75 and 0.80 than two other algorithms. Moreover, the result shows that using multicomponent features classifiers could classify better than using features from a single source.

Cognitive load estimation using ocular parameters in automotive

In automotive, usage of electronic devices increased visual inattention of drivers while driving and might lead to accidents. It is often challenging to detect if a driver experienced a change in cognitive state requiring new technology that can best estimate driver's cognitive load. In this paper, we investigated the efficacy of various ocular parameters to estimate cognitive load and detect cognitive state of driver. We derived gaze and pupil-based metrics and evaluated their efficacy in classifying different levels of cognitive states while performing psychometric tests in varying light conditions. We validated the performance of our metrics in simulation as well as in-car environments. We compared the accuracy (from confusion matrix) of detecting cognitive state while performing secondary task using our proposed metrics and Machine Learning models. It was found that a Neural Network model combining multiple ocular metrics showed better accuracy (75%) than individual ocular metrics. Finally, we demonstrated the potential of our system to alert drivers in real-time under critical distractions.

An analysis of the suitability of a low-cost eye tracker for assessing the cognitive load of drivers

This paper presents a driving simulator study in which we investigated whether the Eye Tribe eye tracker (ET) is capable of assessing changes in the cognitive load of drivers through oculography and pupillometry. In the study, participants were asked to drive a simulated vehicle and simultaneously perform a set of secondary tasks with different cognitive complexity levels. We measured changes in eye properties, such as the pupil size, blink rate and fixation time. We also performed a measurement with a Detection Response Task (DRT) to validate the results and to prove a steady increase of cognitive load with increasing secondary task difficulty. The results showed that the ET precisely recognizes an increasing pupil diameter with increasing secondary task difficulty. In addition, the ET shows increasing blink rates, decreasing fixation time and narrowing of the attention field with increasing secondary task difficulty. The results were validated with the DRT method and the secondary task performance. We conclude that the Eye Tribe ET is a suitable device for assessing a driver's cognitive load.

On the selection of stimulus for the auditory variant of the detection response task method for driving experiments

ABSTRACTObjectives: The detection response task (DRT) is a method for measuring attentional effects of secondary tasks on a driver's cognitive load by measuring response times and hit rates to different types of stimuli as indirect indicators of increased cognitive load. ISO 17488 (International Organization for Standardization 2016) only provides guidelines for the technical implementation and measurement methods for the visual and tactile versions (use of visual and tactile stimuli) of the DRT method. This article presents a study with the goal of finding the most appropriate auditory stimulus for the implementation of an auditory version of the DRT method.Methods: This article presents the results of an experiment in which responses to 7 different auditory DRT stimuli—varying in frequency—were compared while inducing users' cognitive load with a modified n-back task. The experiment was conducted in a surrogate driving environment and in a within-subject design. Response times, hit rates, and secondary task performances were observed as indicators of increased cognitive load.Results: Significantly shorter response times were found for the white noise signal compared to single-frequency signals. However, the largest differences in response times, for trials without and with a cognitive task, were found for 4- and 8-kHz single-frequency signals. No significant differences were found for hit rates and secondary task performances between the different stimuli.Conclusions: Consistent significant differences in response times for all tested stimuli prove that the auditory DRT variant is also sensitive to changes in cognitive load. The mean increase in response times of more than 25% for 4- and 8-kHz signals for trials with a cognitive task compared to trials without one indicates that one of these signals could be used as a potential auditory stimulus for the auditory DRT variant.

A simulator evaluation of effects of assistive technologies on driver cognitive load at railway-level crossings

ABSTRACTIntelligent Transport Systems (ITS) have the potential to substantially reduce the number of crashes caused by human errors at railway-level crossings. However, such systems could overwhelm drivers, generate different types of driver errors, and have negative effects on safety at level crossing. The literature shows an increasing interest for new ITS for increasing driver situational awareness at level crossings, as well as evaluations of such new systems on compliance. To the authors' knowledge, the potential negative effects of such technologies have not yet been comprehensively evaluated. This study aimed at assessing the effect of different ITS interventions, designed to enhance driver behavior at railway crossings, on driver's cognitive loads. Fifty-eight participants took part in a driving simulator study in which three ITS devices were tested: an in-vehicle visual ITS, an in-vehicle audio ITS, and an on-road valet system. Driver cognitive load was objectively and subjectively assessed for each ITS intervention. Objective data were collected from a heart rate (HR) monitor and an eye tracker, whereas subjective data was collected with the NASA Task Load Index (NASA-TLX) questionnaire. Overall, results indicated that the three trialed technologies did not result in significant changes in cognitive load while approaching crossings.

An Ecological Interface Design Approach for Developing Integrated and Advanced In-Vehicle Information System

The objective of this study is to propose a principled ecological interface design approach for in-vehicle information system and reduce driver's cognitive overload otherwise it might cause significant deterioration of driving performance and fatal consequences. Rasmussen's abstraction hierarchy was applied to analyze functional difference between the old and new cars. Information requirement analysis based on the Skill-Rule-Knowledge framework was conducted to classify the level of driver's cognitive load when interacting with modern car functions. It was found that driving assistance functions have been increased but their interfaces are not integrated nor standardized properly yet. Sometimes information is too much, too complex or not given at the right time. This study suggests to follow four principles of design for improving current in-vehicle information system: 1) Consolidation and Parsimony; 2) Abstraction and Integration; 3) Utilization of different display characteristics ; 4) Standardization through Customization . In this way of information design and distribution, people having different levels of driving experience would easily interact with new powerful functions of the car and stay in a safe, comfortable driving environment without losing connectivity.

41.4: Invited Paper: Advances to In‐Car Human Machine Interface Systems

Given recent trends, it is inevitable that there will be more and more computers in vehicles: built‐in, brought‐in, worn, etc. Many of these devices will want to interact with us‐and we with them. If not designed and engineered appropriately, the user interfaces (UIs) of these devices may become increasingly complex and distracting to the driver. To avoid this, careful attention has to be paid to designing appropriate in‐vehicle interaction methods and human‐machine interfaces (HMIs). We suggest that at this point, updates to in‐vehicle HMIs cannot be just incremental, but have to be dramatic in order to reduce distraction and driver cognitive load. This paper will cover recent significant developments at Harman International that will be game‐changing for in‐car HMIs.

Classifying Cognitive Load and Driving Situation with Machine Learning

 Abstract—This paper classifies a driver's cognitive state in real driving situations to improve the in-vehicle information service that judges a user's cognitive load and driving situation. We measure the driver's eye movement and collect driving sensor data such as braking, acceleration, and steering angles that are used to classify the driver's state. A set of data about the driver's degree of cognitive load, regarded as a training set, is obtained from steering operation and task cognition. Given such information, we use a machine-learning method to classify the driver's cognitive load. We achieved reasonable accuracy in certain driving situations in which the driver moves abnormally for an appropriate service supporting safe driving. surrounding situation, the range of eye movement may increase because the driver's cognitive load has increased. This means that a high cognitive load and a change of eye movement are related. Eye movement is used in the field of physiological psychology for clarifying control (1). It is directly related to perception and can be considered an indication of mental load. Driving a car requires prediction of the surrounding environment and is influenced by the situation. Therefore, a user's cognitive load can be classified using these features. To do this, we measure the driver's eye movement and gather driving data such as accelerator use, braking, and steering. This paper takes a machine-learning approach to the above cognitive state identification problem in a realistic car-driving task. We set up cognitive loads according to the steering-entropy method (2) and a definition of the task cognition situation. This paper is organized as follows. Section II presents related works. Section III defines cognitive load. Section IV describes our classification model for cognitive load. Section V presents a performance evaluation. The final section provides conclusions.

Qualitative Reasoning Approach to a Driver’s Cognitive Mental Load

This paper explores applying qualitative reasoning to a driver’s mental state in real driving situations so as to develop a working load for intelligent transportation systems. The authors identify the cognitive state that determines whether a driver will be ready to operate a device in car navigation. In order to identify the driver’s cognitive state, the authors will measure eye movements during car-driving situations. Data can be acquired for the various actions of a car driver, in particular braking, acceleration, and steering angles from the experiment car. The authors constructed a driver cognitive mental load using the framework of qualitative reasoning. The response of the model was checked by qualitative simulation. The authors also verified the model using real data collected by driving an actual car. The results indicated that the model could represent the change in the cognitive mental load based on measurable data. This means that the framework of this paper will be useful for designing user interfaces for next-generation systems that actively employ user situations.

PROPOSAL OF WARNING METHOD FOR REAR-SIDE OBSTACLES USING DRIVER'S PERCEPTUAL RISK MODEL

Warning systems have been proposed to reduce driver cognitive and judgment load as driver systems for traffic safety. The system's efficacy could be decreased if the driver feels annoyance and/or mistrust with inappropriate warning timing, etc. This paper explores the possibility of personalized warning timing to cope with this problem. First, we propose a new warning method for rear-side obstacles using a driver's perceptual risk model that we derived from the analysis of the driver's deceleration behavior. Validity of the warning method will be shown by driving simulator experiments. In addition, there exists individual difference in expectation of meaning of warning. Thus, differences of the efficacy and driver's response behaviors against warning will be analyzed based on the difference of their expectation.

Evaluation of Driver's Cognitive Load Caused by Present Information using P3 Latency in Eye-fixation Related Potential

Evaluation of Driver's Cognitive Load Caused by Present Information using P3 Latency in Eye-fixation Related Potential

차량 내 통합 컨트롤 유형에 따른 운전자 수행도 비교

ABSTRACT As drivers spend more time in their cars, perception of driving a vehicle turns from utilizing a transportation means into residing in a personal space or even in moving office. Such a perception renders automobile manufacturers incorporate more vehicle functions, especially in-vehicle information systems As the number of system functions increases, the complexity of the information to be displayed also increases, so does the controls of such driver information systems. In this study, 3 types of control and 2 types of display menus were designed after a literature review and a market analysis. With these controls and display menus, the experiment was performed to look into the difference of driver performance and preference on the integrated vehicle control type. Finally, the study suggests the integrated vehicle control type to minimize driver's cognitive load, and to use various functions efficiently. The study also discusses the practical use of the final integrated vehicle control type. Keyword: Driver information system, Driver performan ce, Driver preference, Display and control design