A Fatigue Driving Detection Algorithm Based on Facial Multi-Feature Fusion

Kening Li,Yunbo Gong,Ziliang Ren

doi:10.1109/access.2020.2998363

Abstract

Researches on machine vision-based driver fatigue detection algorithm have improved traffic safety significantly. Generally, many algorithms do not analyze driving state from driver characteristics. It results in some inaccuracy. The paper proposes a fatigue driving detection algorithm based on facial multi-feature fusion combining driver characteristics. First, we introduce an improved YOLOv3-tiny convolutional neural network to capture the facial regions under complex driving conditions, eliminating the inaccuracy and affections caused by artificial feature extraction. Second, on the basis of the Dlib toolkit, we introduce the Eye Feature Vector(EFV) and Mouth Feature Vector(MFV), which are the evaluation parameters of the driver's eye state and mouth state, respectively. Then, the driver identity information library is constructed by offline training, including driver eye state classifier library, driver mouth state classifier library, and driver biometric library. Finally, we construct the driver identity verification model and the driver fatigue assessment model by online assessment. After passing the identity verification, calculate the driver's closed eyes time, blink frequency and yawn frequency to evaluate the driver's fatigue state. In simulated driving applications, our algorithm detects the fatigue state at a speed of over 20fps with an accuracy of 95.10%.

Highlights

With the rapid growth of the number of cars, there are more and more traffic accidents, which brings huge potential safety hazards to travel
DRIVER IDENTITY INFORMATION LIBRARY 1) DRIVER EYE STATE CLASSIFIER LIBRARY As mentioned above, traditional driver fatigue detection algorithms are mostly based on the P80 criterion, which uses a fixed threshold to judge the driver’s eye state without considering driver characteristics
EXPERIMENTS To verify the validity of the algorithm, the paper evaluated the performance of the improved YOLOv3-tiny network with the Self-built data set driving state dataset (DSD) and public data set WIDER FACE

Summary

INTRODUCTION

With the rapid growth of the number of cars, there are more and more traffic accidents, which brings huge potential safety hazards to travel. The objective method is to use the auxiliary tools to detect the driver’s physiological characteristics or monitor the vehicle information, etc., and to judge fatigue driving [9], [10]. It is a non-contact method to determine fatigue based on the driver’s facial features [19], [20] It does not cause interference and impact on the driver while driving the vehicle, and has the advantages of fast speed and strong operability. Detection methods based on physiology and behavior usually require the driver to wear or install more physiological information monitoring devices, which affects the comfort of the driver’s normal driving. The common algorithms judge fatigue by state of the driver’s eyes and mouth. Most of the existing algorithms are based on the PERCLOS, which uses the driver’s eyes state as a feature to judge fatigue. We propose the future optimization direction and prospect of the algorithm

METHODOLOGY

DRIVER’S IDENTITY VERIFICATION MODEL

DRIVER FATIGUE ASSESSMENT MODEL

Findings

CONCLUSION