A spatio-temporal wavelet-based descriptor for dynamic 3D facial expression retrieval and recognition

Abstract

Human emotions are often expressed by facial expressions and are generated by facial muscle movements. In recent years, the analysis of facial expressions has emerged as an active research area due to its various applications such as human---computer interaction, human behavior understanding, biometrics, emotion recognition, computer graphics, driver fatigue detection, and psychology. A novel analysis of dynamic 3D facial expressions using the positional information of automatically detected facial landmarks and the wavelet transformation is presented, which results in the proposed spatio-temporal descriptor. This descriptor is employed within the current paper in a retrieval scheme for dynamic 3D facial expression datasets and is thoroughly evaluated. Experiments have been conducted using the six prototypical expressions of the publicly available BU-4DFE dataset as well as the eight expressions included in the newly released publicly available BP4D-Spontaneous dataset. The obtained retrieval results outperform the retrieval results of the state-of-the-art methodologies. Furthermore, the retrieval results are exploited to achieve unsupervised dynamic 3D facial expression recognition. The aforementioned unsupervised procedure achieves better recognition accuracy compared to supervised dynamic 3D facial expression recognition state-of-the-art techniques.