Driver Monitoring System Research Articles

Driving performance and ocular activity following acute administration of 10 mg methylphenidate: A randomised, double-blind, placebo-controlled study.

Methylphenidate is a routinely prescribed treatment for attention-deficit/hyperactivity disorder with misuse potential owing to its perceived performance-enhancing and euphoric properties. Although clinically effective, there is limited understanding of how methylphenidate affects safety-sensitive tasks such as driving when used by healthy individuals. Explore the acute effects of 10 mg methylphenidate on driving performance and gaze behaviour. Twenty-five fully licensed, healthy adults (mean age = 33.5 ± 7.8 years, 64% male) took part in two 40-min simulated highway drives with simultaneous eye movements monitored using a proprietary automotive-grade driver monitoring system (Seeing Machines). Driving performance was assessed using the standard deviation of lateral position, standard deviation of speed and steering variability. Visual scanning efficiency was determined using ocular metrics, such as fixation duration and rate, gaze transition entropy, and stationary gaze entropy, were assessed to determine visual scanning efficiency. Methylphenidate significantly improved driving performance by reducing lane weaving and speed variation, particularly in the latter half of the drive. Although a significant reduction in fixation duration was observed, all other ocular metrics remained unchanged. Methylphenidate mitigates performance decrements typically associated with prolonged and monotonous driving. The absence of pronounced oculomotor effects suggests that a single 10 mg dose of methylphenidate has no deleterious impact on visual scanning behaviour during driving tasks with low-to-moderate cognitive demand. Future research should investigate the effects of methylphenidate under various dosing and driving conditions to better understand its impact. ACTRN12620000499987.

Neuro-Adaptive AI for Dynamic Distraction Mitigation in Autonomous Vehicle Environments

As autonomous vehicles develop, driver distraction becomes even more crucial as it affects both safety and operational efficiency. In this work, we investigate the gamut of new AI tools for combating and processing visual distraction scenarios within autonomous vehicles. This includes AI-based driver monitoring systems to determine the level of attention, visual distraction classification with deep learning models, augmented reality head-up displays for focal projection of critical information and gesture/voice-controlled interfaces are used in order to reduce visual interactions. This also includes how predictive analytics; adaptive user interfaces and personalized distraction mitigation programs will see AI improve driver focus and thus safety. These advanced systems are designed to provide a safer and more efficient driving experience in the emerging era of autonomous capabilities by leveraging the scalability of advanced driver-assistance technologies.

Proposal of a Cost-Effective and Adaptive Customized Driver Inattention Detection Model Using Time Series Analysis and Computer Vision

Advanced Driver Assistance Systems, such as Forward Collision Warning and Lane Departure Warning, play a crucial role in accident prevention by alerting drivers to potential hazards. With the advent of fully autonomous driving technology that requires no driver input, there is now a greater emphasis on monitoring the state of vehicle occupants. This is particularly important because, in emergency situations where control must suddenly be transferred to an unprepared occupant, the risk of accidents increases significantly. To mitigate this risk, new monitoring technologies are being developed to analyze driver behavior and detect states of inattention or drowsiness. In response to the emerging demands of driver monitoring technology, we have developed the Customized Driver Inattention Detection Model (CDIDM). This model employs video analysis and statistical techniques to accurately and rapidly classify information on drivers’ gazes. The CDIDM framework defines the components of inattentive or drowsy driving based on the Driver Monitoring System (DMS) safety standards set by the European New Car Assessment Programme (EuroNCAP). By defining six driving behavior-related scenarios, we have improved the accuracy of driver inattention assessment. The CDIDM estimates the driver’s gaze while simultaneously analyzing data in real-time. To minimize computational resource usage, this model incorporates a series of preprocessing steps that facilitate efficient time series data analysis, utilizing techniques such as DTW Barycenter Averaging (DBA) and K-means clustering. This results in a robust driver attention monitoring model based on time series classification.

Detection of Bus Driver Mobile Phone Usage Using Kolmogorov-Arnold Networks

This research introduces a new approach for detecting mobile phone use by drivers, exploiting the capabilities of Kolmogorov-Arnold Networks (KAN) to improve road safety and comply with regulations prohibiting phone use while driving. To address the lack of available data for this specific task, a unique dataset was constructed consisting of images of bus drivers in two scenarios: driving without phone interaction and driving while on a phone call. This dataset provides the basis for the current research. Different KAN-based networks were developed for custom action recognition tailored to the nuanced task of identifying drivers holding phones. The system’s performance was evaluated against convolutional neural network-based solutions, and differences in accuracy and robustness were observed. The aim was to propose an appropriate solution for professional Driver Monitoring Systems (DMS) in research and development and to investigate the efficiency of KAN solutions for this specific sub-task. The implications of this work extend beyond enforcement, providing a foundational technology for automating monitoring and improving safety protocols in the commercial and public transport sectors. In conclusion, this study demonstrates the efficacy of KAN network layers in neural network designs for driver monitoring applications.

A gaze-based driver distraction countermeasure: Comparing effects of multimodal alerts on driver's behavior and visual attention

This study, introduces and evaluates different countermeasures using real-time eye-tracking data. The countermeasures detect when driver gaze deviates from the road for longer than a predetermined threshold and then redirect the driver's attention back to the road. The countermeasures include bimodal and trimodal alerts using combinations of auditory, tactile, and visual modalities. These countermeasures showcase the utility of adopting eye-tracking technologies in the context of driver monitoring and advanced driver's assistance systems. They enhance safety as a safeguard for the increased use of devices such as in-vehicle infotainment systems. Results show that countermeasures effectively redirect drivers’ attention to the road, with higher on-road gaze time. Additionally, bimodal alerts that include the visual modality are less effective at redirecting participants’ gaze on-road and result in poorer driving performance.

Drivers’ situational awareness of surrounding vehicles during takeovers: Evidence from a driving simulator study

This study aimed to understand the influence of surrounding vehicle configuration, driving lane, and traffic density on drivers’ situational awareness (SA), takeover performance, and eye-tracking behaviors in conditionally automated driving. An experiment was conducted with the participation of 40 university students using a fixed-base driving simulator configured to simulate SAE Level 3 automation. During the experiment, participants were engaged in playing Tetris on a tablet as a non-driving related task in automated driving mode. Upon hearing an auditory takeover request, participants were instructed to take control of the vehicle, and then complete a scene reconstruction task to report their SA after transferring control back to the automated driving system. Our findings showed that drivers often neglected vehicles at their sides and rear during the takeover, which was associated with higher collision risks. Higher oncoming traffic density led to drivers’ worse SA of surrounding vehicles but more cautious driving behavior. Driving in the right lane generally resulted in smoother takeovers with lower collision risks. Interestingly, while SA did moderate the impacts of driving conditions on safety margins, a higher level of SA did not consistently relate to improved performance, especially in complex scenarios. This suggests the need for support systems that guide drivers to focus on safety–critical objects rather than simply amplifying SA in general. These insights have significant implications for the design of driver monitoring and support systems in automated vehicles.

IoT-Based Assessment of a Driver's Stress Level.

Driver Monitoring Systems (DMSs) play a key role in preventing hazardous events (e.g., road accidents) by providing prompt assistance when anomalies are detected while driving. Different factors, such as traffic and road conditions, might alter the psycho-physiological status of a driver by increasing stress and workload levels. This motivates the development of advanced monitoring architectures taking into account psycho-physiological aspects. In this work, we propose a novel in-vehicle Internet of Things (IoT)-oriented monitoring system to assess the stress status of the driver. In detail, the system leverages heterogeneous components and techniques to collect driver (and, possibly, vehicle) data, aiming at estimating the driver's arousal level, i.e., their psycho-physiological response to driving tasks. In particular, a wearable sensorized bodice and a thermal camera are employed to extract physiological parameters of interest (namely, the heart rate and skin temperature of the subject), which are processed and analyzed with innovative algorithms. Finally, experimental results are obtained both in simulated and real driving scenarios, demonstrating the adaptability and efficacy of the proposed system.

Enhancing bus safety: A modular driver monitoring system

Summary. In the area of road safety, the development of a Modular, Machine Vision-Based, Custom-Built Driver Monitoring System (DMS) for bus drivers has become imperative. This research presents a comprehensive system capable of detecting drowsiness, blinking patterns, and various forms of distraction, including the use of mobile phones, and one-handed driving. Leveraging the power of Mediapipe and YOLOv7 for real-time image analysis, as well as ROS2 for seamless data transfer, our system not only ensures the immediate safety of bus passengers but also offers expandable functionality, such as eye tracking and skeleton detection. Összefoglalás. A biztonságos tömegközlekedés iránti igény fokozza a járművezetőkre nehezedő nyomást a növekvő forgalmi torlódások miatt. Ezt súlyosbítja a közlekedési rendszer összetettsége és a fokozódó külső ingerek hatása, különösen városi környezetben. A közösségi közlekedésben alkalmazható, a közlekedésbiztonságot fokozó moduláris, gépi látáson alapuló, egyedi fejlesztésű járművezető-felügyeleti rendszer kifejlesztése az autóbuszvezetők monitorozására elengedhetetlen. Ez a kutatás egy olyan átfogó rendszert mutat be, amely képes érzékelni az álmosságot, a pislogási mintákat és a figyelemelterelés (disztrakció) különböző formáit, beleértve a mobiltelefon-használatot, és az egykezes vezetést. A Mediapipe és a YOLOv7 valós idejű képelemzésre, valamint a ROS2 adatátvitelre való felhasználásával rendszerünk nemcsak a busz utasainak biztonságát garantálja, hanem olyan bővíthető funkcionalitást is kínál, mint például szemkövetés és csontvázfelismerés (szkeleton). A rendszer alapvető célja az, hogy a szemmozgás, fejtartás és testtartás elemzésével pontosan azonosítja a járművezető fáradtságát, figyeli a pislogási mintákat az álmosság jeleit, és felismeri a közúti biztonságot veszélyeztető disztrakciókat. Továbbá a rendszer moduláris felépítése lehetővé teszi további érzékelők, például szemmozgás-követő rendszer, telemetriai eszközök vagy 5G-adapterek egyszerű integrálását, ami átfogó megfigyelést és adatfúziót tesz lehetővé a valós környezetbe történő adaptálás elősegítésére. A fejlesztett Járművezető Monitoring Rendszer a ROS2 keretrendszer segítségével integrált megoldást kínál a buszvezetők megfigyelésére. A rendszer alapvető képessége a fedélzeti kamerák által rögzített felvételeken a járművezető vizuális felismerése. Ezen túlmenően képes követni a vezető testén lévő kulcspontok, mint a fej, a törzs és a karok pozícióját, ami létfontosságú az ő testtartásának és mozgásának megértésében. Az egyediséget a vezetőfülke mérete és a buszvezető személygépjárműhöz mérten dinamikus mozgásképe adja. A rendszer az emberi arcot is részletesen elemzi, kiemelve a fontos arcpontokat, mint a szemek, orr és száj. Ez lehetővé teszi a tekintet irányának, arckifejezéseknek, valamint a fáradtság vagy stressz jeleinek azonosítását. Az adatok könnyebb értelmezése érdekében a rendszer egy vizuális ábrázolást is nyújt az észlelési folyamatról. Az adatok kezelése és kommunikációja a ROS2 keretrendszeren keresztül történik, amely strukturált módon rendezi az adatokat és támogatja a valós idejű feldolgozást és elemzést. Az összegyűjtött adatok tárolására a .rosbag fájlformátumot használjuk, amely lehetővé teszi az adatok hatékony rögzítését és későbbi felhasználását elemzésekhez és felülvizsgálatokhoz. A tanulmány a moduláris járművezető-felügyeleti rendszer felépítését, megvalósítását és tesztelését mutatja be, részletesen közli az alkalmazott algoritmusokat és technológiákat. A valós körülmények között végzett kísérletek eredményei bizonyítják a rendszer hatékonyságát, valamint a rendszer szélesebb közlekedési ökoszisztémákba való integrálhatóságát. A buszvezető monitorozása kapcsán kapott adatok hozzájárulhatnak a jármű és utasai biztonságának fokozásához. A járművezető figyelmének nyomon követése és a kognitív terhelés elemzése lehetőséget kínál a munkakörülmények optimalizálására és a balesetmegelőzési megoldások javítására.

CBAM VGG16: An efficient driver distraction classification using CBAM embedded VGG16 architecture

Driver monitoring systems (DMS) are crucial in autonomous driving systems (ADS) when users are concerned about driver/vehicle safety. In DMS, the significant influencing factor of driver/vehicle safety is the classification of driver distractions or activities. The driver’s distractions or activities convey meaningful information to the ADS, enhancing the driver/ vehicle safety in real-time vehicle driving. The classification of driver distraction or activity is challenging due to the unpredictable nature of human driving. This paper proposes a convolutional block attention module embedded in Visual Geometry Group (CBAM VGG16) deep learning architecture to improve the classification performance of driver distractions. The proposed CBAM VGG16 architecture is the hybrid network of the CBAM layer with conventional VGG16 network layers. Adding a CBAM layer into a traditional VGG16 architecture enhances the model’s feature extraction capacity and improves the driver distraction classification results. To validate the significant performance of our proposed CBAM VGG16 architecture, we tested our model on the American University in Cairo (AUC) distracted driver dataset version 2 (AUCD2) for cameras 1 and 2 images. Our experiment results show that the proposed CBAM VGG16 architecture achieved 98.65% classification accuracy for camera 1 and 97.85% for camera 2 AUCD2 datasets. The CBAM VGG16 architecture also compared the driver distraction classification performance with DenseNet121, Xception, MoblieNetV2, InceptionV3, and VGG16 architectures based on the proposed model’s accuracy, loss, precision, F1 score, recall, and confusion matrix. The drivers’ distraction classification results indicate that the proposed CBAM VGG16 has 3.7% classification improvements for AUCD2 camera 1 images and 5% for camera 2 images compared to the conventional VGG16 deep learning classification model. We also tested our proposed architecture with different hyperparameter values and estimated the optimal values for best driver distraction classification. The significance of data augmentation techniques for the data diversity performance of the CBAM VGG16 model is also validated in terms of overfitting scenarios. The Grad-CAM visualization of our proposed CBAM VGG16 architecture is also considered in our study, and the results show that VGG16 architecture without CBAM layers is less attentive to the essential parts of the driver distraction images. Furthermore, we tested the effective classification performance of our proposed CBAM VGG16 architecture with the number of model parameters, model size, various input image resolutions, cross-validation, Bayesian search optimization and different CBAM layers. The results indicate that CBAM layers in our proposed architecture enhance the classification performance of conventional VGG16 architecture and outperform the state-of-the-art deep learning architectures.

Training benefits driver behaviour while using automation with an attention monitoring system

Attention, or more generally, driver monitoring systems have been identified as a necessity to address overreliance on driving automation. However, research suggests that monitoring systems may not be sufficient to support safe use of advanced driver assistance systems (ADAS), also evidenced by a recent major recall of Tesla’s monitoring software. The objective of the current study was to investigate whether different training approaches improve driver behaviour while using ADAS with an attention monitoring system. A driving simulator study was conducted with three between-subject groups: no training, limitation-focused training (highlighted situations where ADAS would not work), and responsibility-focused training (highlighted the driver’s role/responsibility while using ADAS). All participants (N = 47) experienced eight events which required the ego-vehicle to slow down to avoid a collision. Anticipatory cues in the environment indicated the potential for the upcoming events. Event type (covered in training vs. not covered) and event criticality (action-necessary vs. action-not-necessary) were within-subject factors. The responsibility-focused group made fewer long glances (≥ 3 s) to a secondary task than the no training and limitation-focused groups when there were no anticipatory cues. Responsibility-focused training and no training were associated with faster takeover time at the events than limitation-focused training. There were additional benefits of responsibility-focused training for events that were covered in training (e.g., higher percent of time looking at the anticipatory cues). Overall, our results suggest that even if attention monitoring systems are implemented, there may be benefits to driver ADAS training. Responsibility-focused training may be preferable to limitation-focused training, especially for situations where minimizing training length is advantageous.

The effectiveness and efficacy of driving interventions with ADHD: a Dutch perspective

The diagnosis of attention deficit hyperactivity disorder (ADHD) has been identified as a factor associated with an increased risk of involvement in car crashes. As a result, individuals in the Netherlands who are diagnosed with ADHD are mandated to undergo a psychiatric evaluation before obtaining their driving licence, optionally complemented with a driving test. Recent research has, however, demonstrated that 96% of individuals pass this procedure and can drive unrestrictedly, suggesting that current regulations have a limited impact and that a different regulatory approach may be warranted. This paper addresses three such potential interventions, exploring the scientific basis of implementing mandatory behavioural training programmes, the mandatory installation of driver monitoring systems, and temporary driving restrictions. To initiate this exploration, an analysis is made of the body of literature pointing towards a subpar driving performance in those with ADHD, accompanied by a discussion of the underlying symptoms and behaviours contributing to these findings. Furthermore, between- and within-individual factors affecting ADHD driving performance are reviewed. This examination illuminates a consistent pattern of substandard driving performances among those with ADHD, whereby the period shortly post-licensure stands out as a focal point for regulatory approaches. Following these results, the three potential driving interventions are reviewed to discern their impact on road safety when implemented specifically for drivers with ADHD. Corresponding studies suggest that all these interventions hold promise in improving driving performances, but the analysis is limited by the scarcity of long-term, ADHD-specific research. The paper concludes by discussing the feasibility of these ADHD-specific driving interventions, weighing their potential benefits, e.g., reduced crash rates, along with their associated costs, such as potential stigma and hindrance in societal integration and participation. Additionally, avenues for future research in these domains are outlined.

Attentional warnings caused by driver monitoring systems: How often do they appear and how well are they understood?

The present study investigated the effects of a driver monitoring system that triggers attention warnings in case distraction is detected. Based on the EuroNCAP protocol, distraction could either be long glances away from the forward roadway (≥3s) or visual attention time sharing (>10 cumulative seconds within a 30 s time interval). In a series of manual driving simulator drives, 30 participants completed both driving related tasks (e.g., changing multiple lanes in dense traffic) and non-driving related tasks (e.g., infotainment operations). Results of warning frequencies revealed that visual attention time sharing warnings occurred more frequently than long distraction warnings. Moreover, there was a large number of attention warnings during driving related tasks. Results also revealed that participants’ mental models tended to be less accurate when it came to understanding of the visual attention time sharing warnings as compared to the long distraction warnings, which were understood more accurately. Based on these observations, the work discusses the applicability and design of driver monitoring warnings.

Practical aspects of measuring camera‐based indicators of alcohol intoxication in manual and automated driving

AbstractCamera‐based Driver Monitoring Systems (DMS) have the potential to exploit eye tracking correlates of alcohol intoxication to detect drunk driving. This study investigates how glance, blink, saccade, and fixation metrics are affected by alcohol, and whether possible effects remain stable across three different camera setups, as well as when the driver is out‐of‐the‐loop during level 4 automated driving (Wizard‐of‐Oz setup). Thirty‐five participants drove on a test track first sober and then with increasing intoxication levels reaching a breath alcohol concentration (BrAC) of 1‰. Linear Mixed‐Effects Regression analyses showed that with increasing intoxication levels, eye blinks became longer and slower, glances and fixations became fewer and longer, and more attention was directed to the road area, at the expense of more peripheral areas. Fixation and blink metrics were more robust to changes in automation mode, whereas glance‐based metrics were highly context dependent. Not all effects of alcohol intoxication could be measured with all eye tracking setups, where one‐camera systems showed lower data availability and higher noise levels compared to a five‐camera system. This means that lab findings based on higher quality eye tracking data might not be directly applied to production settings because of hardware limitations.

A Study on the Development of Driver Behavior Simulation Dummy for the Performance Evaluation of Driver Monitoring System

Driver monitoring system (DMS) was mainly developed to prevent accident risks by analyzing facial movements related to drowsiness and carelessness in real time such as driver’s gaze, blink, and head angle through cameras and warning the driver. Recently, the scope has been expanded to monitor passengers, and it has been linked to safety functions such as neglecting children, empty seats, or controlling airbags on seats with people under safety weight. However, evaluation research for algorithm advancement and performance optimization is relatively insufficient. In addition, the verification system is facing limitations such as personal information protection problems caused by the subject’s face data, errors in reproducing the subject’s drowsy and careless behavior, and differences in behavior according to individual differences. Therefore, as the importance of traffic safety is emphasized, an evaluation tool that can more efficiently and systematically evaluate the performance of DMS is needed. In this study, a driver behavior simulation dummy was developed that can quantitatively control the movement of the driver’s face and upper body. The driver behavior simulation dummy was developed in three stages in the order of function and specification definition, design and manufacture according to specifications, and verification through error tests for each function.

Design of a Multidimensional Intelligent Safety Driving Monitoring System Based on Machine Vision

This study introduces a multidimensional driving safety monitoring system that integrates millimeter-wave radar and an Arduino Uno-based alcohol sensor. The system aims to enhance driving safety and prevent accidents by real-time monitoring and analysis of the driving environment, driver's condition, and vehicle dynamics using machine vision technology. Utilizing millimeter-wave radar, the system precisely detects obstacles and pedestrians around the vehicle, facilitating efficient distance and speed measurements to support automatic emergency braking and collision warnings. Additionally, the Arduino Uno-based alcohol sensor monitors the driver's breath for alcohol concentration, ensuring the driver is in a suitable state for driving. Combined with machine vision technology, the system further analyzes video stream data from inside and outside the cockpit, monitoring driver behavior and external traffic conditions to enhance the identification of potential risks. The success of this research illustrates the potential of combining machine vision with advanced sensor technologies, offering new insights for the development of comprehensive driving assistance systems.

Efficient Eye State Detection for Driver Fatigue Monitoring Using Optimized YOLOv7-Tiny

This study refines the YOLOv7-tiny model through structured pruning and architectural fine-tuning, specifically for real-time eye state detection. By focusing on enhancing the model’s efficiency, particularly in environments with limited computational resources, this research contributes significantly to advancing driver monitoring systems, where timely and accurate detection of eye states such as openness or closure can prevent accidents caused by drowsiness or inattention. Structured pruning was utilized to simplify the YOLOv7-tiny model, reducing complexity and storage requirements. Subsequent fine-tuning involved adjustments to the model’s width and depth to further enhance processing speed and efficiency. The experimental outcomes reveal a pronounced reduction in storage size, of approximately 97%, accompanied by a sixfold increase in frames per second (FPS). Despite these substantial modifications, the model sustains high levels of precision, recall, and mean average precision (mAP). These improvements indicate a significant enhancement in both the speed and efficiency of the model, rendering it highly suitable for real-time applications where computational resources are limited.

Comprehensive Assessment of Artificial Intelligence Tools for Driver Monitoring and Analyzing Safety Critical Events in Vehicles.

Human factors are a primary cause of vehicle accidents. Driver monitoring systems, utilizing a range of sensors and techniques, offer an effective method to monitor and alert drivers to minimize driver error and reduce risky driving behaviors, thus helping to avoid Safety Critical Events (SCEs) and enhance overall driving safety. Artificial Intelligence (AI) tools, in particular, have been widely investigated to improve the efficiency and accuracy of driver monitoring or analysis of SCEs. To better understand the state-of-the-art practices and potential directions for AI tools in this domain, this work is an inaugural attempt to consolidate AI-related tools from academic and industry perspectives. We include an extensive review of AI models and sensors used in driver gaze analysis, driver state monitoring, and analyzing SCEs. Furthermore, researchers identified essential AI tools, both in academia and industry, utilized for camera-based driver monitoring and SCE analysis, in the market. Recommendations for future research directions are presented based on the identified tools and the discrepancies between academia and industry in previous studies. This effort provides a valuable resource for researchers and practitioners seeking a deeper understanding of leveraging AI tools to minimize driver errors, avoid SCEs, and increase driving safety.

Alcohol and Drowsiness Detection in Smart Transportation System

Abstract: Drunk driving is a major contributing factor in the rising number of unintentional deaths that occur in the modern day. Drunk driving is an extremely dangerous practice that can result in car accidents and injuries on the road. Numerous preventative actions had been implemented, including license suspension, fines, and the seizure of motor vehicle permits. Notwithstanding the numerous measures implemented, the proportion of collisions caused by intoxicated drivers continues to rise. The purpose of this module is to shield people from needless deaths brought on by intoxicated and sleep-deprived drivers in traffic accidents. This module includes a Wi-Fi module and an alcohol detection sensor (MQ-3) based on the Raspberry Pi model 3. We are incorporating an automated drowsy driver monitoring and accident prevention system as part of the design of our major project.

An efficient multi-task learning CNN for driver attention monitoring

Driver Monitoring System (DMS), usually equipped with a camera, is an emerging vehicle safety system that can monitor driver attentiveness and trigger timely alarms when signs of inattention are detected. Since a single indicator (e.g., eye blink rate) is insufficient and unreliable to analyze driver attentiveness, almost all existing solutions train several independent models to identify driver facial states, such as face landmark, head pose, yawning, eye state, etc. However, apart from neglecting the inherent correlations between these related tasks, multiple models also raise challenges for vehicle safety-critical systems (e.g., hardware resources, software compatibility, and real-time response). In this paper, we propose a multi-task learning CNN framework (DANet) to unify the relevant tasks into one model and simultaneously output various driver facial states. By sharing the common features and parameters of highly related tasks, DANet avoids repetitive computations and mitigates single task overfitting. More importantly, the model provides a comprehensive overview of facial states while maintaining low complexity. We also propose two novel designs: (1) Dual-loss Block, which decomposes the pose estimation task into pose classification and coarse-to-fine regression; (2) Head Pose Penalization, which constrains the network to predict gaze direction based on predicted head pose. Our method achieves compelling results in both speed and accuracy on a vehicle computing platform, marking a momentous step in this field.

An Intelligent Driving Monitoring System Utilizing Pedal Motion Sensor Integrated with Triboelectric‐Electromagnetic Hybrid Generator and Machine Learning

AbstractDriver's driving behavior and driving style have a crucial impact on traffic safety, capacity, and efficiency, so it is of great significance to monitor the driver's driving behavior and recognize their driving style. In this work, an intelligent driving monitoring system based on a triboelectric nanogenerator and electromagnetic generator is designed. The system consists of a self‐powered pedal motion sensor (SPMS) and an intelligent data processing unit (IDPU), which can monitor driving behavior and recognize driving style. SPMS is used for driving behavior monitoring, which mainly consists of a six‐phase triboelectric nanogenerator (S‐TENG) and a free‐rotating disk electromagnetic generator (FD‐EMG). S‐TENG can recognize information such as pedal movement direction, movement amplitude, and movement speed, and FD‐EMG can realize the function of a self‐powered driver's driving behavior warning. The IDPU includes a numerical calculation system for driving style characteristic variables and a driving style classifier. It can recognize the driving style based on the driving data collected by SPMS. The driving style classifier design is based on a combination of simulated driving experiments and machine learning techniques, and its accuracy is verified through experiments. This work has important potential applications in the field of traffic safety and intelligent driving.