Video-based observer rated sleepiness versus self-reported subjective sleepiness in real road driving

Christer Ahlstrom,Carina Fors,David Hallvig,Anna Anund

doi:10.1007/s12544-015-0188-y

Abstract

Observer-rated sleepiness (ORS) based on video recordings of the driver’s face is often used when analysing naturalistic driving data. The aim of this study is to investigate if ORS ratings agree with subjective self-reported sleepiness (SRS). Forty raters assessed 54 video-clips showing drivers with varying levels of sleepiness. The video-clips were recorded during a field experiment focusing on driver sleepiness using the same cameras that are typically used in large-scale field studies. The weak results prompted a second test. Ten human factors researchers made pairwise comparisons of videos showing the same four participants in an alert versus a very sleepy condition. The task was simply to select the video-clip where the driver was sleepy. The overall average percentage of video segments where ORS and SRS matched was 41 % in Test 1. ORS 0 (alert) and ORS 2 (very sleepy) were easier to score than ORS 1 and it was slightly harder to rate night-time drives. Inter-rater agreement was low, with average Pearson’s r correlations of 0.19 and Krippendorff’s alpha of 0.15. In Test 2, the average Pearson’s r correlations was 0.35 and Krippendorff’s alpha was 0.62. The correspondence between ORS and SRS showed an agreement of 35 %. The results indicate that ORS ratings based on real road video recordings correspond poorly with SRS and have low inter-rater agreement. Further research is necessary in order to further evaluate the usefulness of ORS as a measure of sleepiness.

Highlights

A recent trend in studies of driver sleepiness is to carry out large-scale naturalistic data collections [1,2,3]
The post-hoc test revealed that higher Observer-rated sleepiness (ORS) ratings were provided during night-time compared to daytime, and that increasing ORS ratings were accompanied by increasing self-reported sleepiness (SRS) ratings
ORS and SRS matched in 41 % of the 1-min video clips of drivers at various stages of sleepiness

Summary

Introduction

A recent trend in studies of driver sleepiness is to carry out large-scale naturalistic data collections [1,2,3]. The vehicles are instrumented with unobtrusive data acquisition systems that continuously record the behaviour of the vehicle (e.g. speed and lane position), the behaviour of surrounding road users, and the drivers’ behaviour (e.g. where they are looking). The advantage of these naturalistic driving studies, from a driver sleepiness perspective, is the possibility to study the extent to which sleepiness contributes to safety critical incidents [1, 4]. Several approaches to measure the effects of driver sleepiness have been explored in the literature. These include physiological recordings and their scoring [e.g. 7, 8], blinking activity and eye movements measured by cameras [e.g. 9], measures of driving performance [e.g. 10–12], self-reported sleepiness, SRS [e.g. 7, 11, 13, 14], and observer-rated sleepiness, ORS [e.g. 15–17]

Objectives

Methods

Results

Conclusion