Yawn analysis with mouth occlusion detection

Abstract

One of the most common signs of tiredness or fatigue is yawning. Naturally, identification of fatigued individuals would be helped if yawning is detected. Existing techniques for yawn detection are centred on measuring the mouth opening. This approach, however, may fail if the mouth is occluded by the hand, as it is frequently the case. The work presented in this paper focuses on a technique to detect yawning whilst also allowing for cases of occlusion. For measuring the mouth opening, a new technique which applies adaptive colour region is introduced. For detecting yawning whilst the mouth is occluded, local binary pattern (LBP) features are used to also identify facial distortions during yawning. In this research, the Strathclyde Facial Fatigue (SFF) database which contains genuine video footage of fatigued individuals is used for training, testing and evaluation of the system.