MFED: A Database for Masked Facial Expression

Abstract

Humans sometimes intentionally altered facial expression to mask the genuine emotions for certain purposes. Such masked facial expressions have seldom been recorded. In this study, we constructed a facial expression database in which participants were required to watch emotional video clips and make specific facial expressions which were consistent or inconsistent with the emotional context, and named it Masked Facial Expression Database (MFED). Twenty-four participants were recruited and trained to be familiar with six basic facial expressions. They were instructed to display six expressions under six basic emotional context which resulted in 36 combinations. Seven hundred and eighty three emotional video clips including 36 (6 by 6) categories of facial expressions were collected. We also utilized Facial Action Coding System (FACS) to encode these facial expressions. In addition, local binary patterns from three orthogonal planes (LBP-TOP) and support vector machine (SVM) were employed for feature extraction and subsequent leave-one-subject-out cross validation, respectively. Best performance of classification is 78.80% for masked and non-masked, 26.83% for experienced emotion evoked by video clips, and 47.77% for required (presented by instructions) expressions. For improving the performance of experienced emotion and required expression recognition, a well-designed convolutional neural network (CNN) was also utilized to capture high-level representations of facial expression images. Our clear-cut results demonstrated that masked and non-masked expressions could be moderately discriminated in this database. Experienced emotion was difficult to identify partly because it was definitely covered up by the masked expression.