Real-time classification for autonomous drowsiness detection using eye aspect ratio

Abstract

Various automated systems require human supervision in complex environments: this can be a monotonous task but still requiring a significant degree of attention. If those tasks are decisive to the process and work safety, then it is imperative that operators maintain adequate levels of alertness to execute necessary actions. Here, we developed a methodology for drowsiness detection based on eye patterns of people monitored by video streams. In contrast to physically intrusive methods based on a biological approach (e.g. electrooculogram), computer vision and machine leaning (ML) were used to create a low-cost real-time system to detect whether a user (operator) is drowsy using a simple web camera. The proposed methodology employs drowsiness rules for blink patterns from neuroscience literature, which allows for automatic alertness supervision of users reducing risks of potential human errorand then preventing accidents. Specifically, a temporal element is introduced by concatenating information from several consecutive video frames coupled with the ability of ML models in identifying different eye behavior. Here, multilayer perceptron, random forest, and support vector machines were analyzed: the latter had the overall best performance in terms of average test accuracy (94.9%) and required execution time. The proposed methodology also contains a personal feedback proposal to adapt models for each specific user providing even better results. We validated our model in DROZY – a public database for human drowsiness. Inter- and intra-subject investigations were conducted considering the Karolinska Sleepiness Scale (KSS) evaluation and the reaction time as performance indicators. In inter-subject analysis, our model did not provide any warning when a subject was awake, but an average of 16.1 warnings were emitted for drowsy subjects with 94.44% accuracy. For intra-subject analysis, our model could detect when subjects were prone to drowsiness. These are interesting and promising results regarding drowsiness detection.