Driver Eye Location and State Estimation Based on a Robust Model and Data Augmentation

Abstract

Eye state evaluation is crucial for vision-based driver fatigue detection. With the outbreak of COVID-19, many proposed models for eye location and state evaluation based on facial landmarks are unreliable due to mask coverings. In this paper, we proposed a robust facial landmark location model for eye location and state evaluation. First, we develop an existing lightweight face alignment model for eye key point locations that is robust in large poses. Then, to develop the performance of our model in a complex driving environment such as an environment with mask coverings, changing illumination, etc., we design a method to augment the training data set based on the original landmark data set without any extra cost. Finally, some facial landmarks around the eyes are extracted, and the eye aspect ratio (EAR) is introduced to evaluate the eye state based on eye key points. The experiment shows that our model achieves significantly improved landmark location performance on a driving simulation data set due to data augmentation. We tested our model on the BioID data set to measure the eye state evaluation performance, and the results showed that our model obtained satisfactory performance with an accuracy of approximately 97.7%. Further testing on the driving simulation data set shows that our model is robust in different driving scenarios with an average accuracy of approximately 93.9%.