Comparative Study of State-of-the-art Face Landmark Detectors for Eye State Classification in Subjects with Face Occlusion

Abstract

A study on drowsiness detection is being conducted based on the video dataset of 22 subjects who are wearing EEG sensors while driving on a simulator. The video dataset is not labelled, and it is in the process of being labelled by using Percentage of Eye Closure (PERCLOS) which is an indicator of drowsiness level. One of the preliminary steps in obtaining PERCLOS values, is to determine the eye state (open or closed) of subjects. A common method used by researchers to determine the eye state of the subject is using Eye Aspect Ratio (EAR) values that are obtained from face landmark detectors such as DLib's 68 face landmarks predictor. Based on the evaluation of the DLib's solution on 151,537 frames (approximately 84 minutes) of one subject, it was found that 98.66% of eyes state were classified correctly which resulted in 378 blinks to be detected (expected 212 blinks). The 468 3D face landmarks detector from MediaPipe (Google) was proposed as an alternative and it managed to classify the same subject with a classification accuracy of 99.87% with 264 blinks (only 52 extra blinks). Plenty of researchers are not aware of this solution and are uncertain of how it compares to DLib's solution. Thus, this paper compares the performance of these two solutions in terms of processing time and eye state classification metrics based on 10 diverse subjects.