A robust spatio-temporal scheme for dynamic 3D facial expression retrieval

Abstract

The problem of facial expression recognition in dynamic sequences of 3D face scans has received a significant amount of attention in the recent past whereas the problem of retrieval in this type of data has not. A novel retrieval scheme for such data is introduced in this paper. It is the first spatio-temporal retrieval scheme ever used for retrieval in dynamic sequences of 3D face scans. The proposed scheme automatically detects specific facial landmarks and uses them to create a spatio-temporal descriptor. At first, geometric as well as topological information of the 3D face scans is captured by using the detected landmarks. In the sequel, the aforementioned spatial information is filtered by using wavelet transformation, resulting to our final spatio-temporal descriptor. Our descriptor is invariant to the number of the 3D face scans of a facial expression sequence. The proposed retrieval scheme exploits the Square of Euclidean distance in order to compare descriptors corresponding to different 3D facial sequences. A detailed evaluation of the introduced retrieval scheme is presented showing that it outperforms previous state-of-the-art retrieval schemes. Experiments have been conducted using the six prototypical expressions of the standard data set $$\textit{BU}-4\textit{DFE}$$BU-4DFE. Finally, a majority voting methodology based on the retrieval results is used to achieve unsupervised dynamic 3D facial expression recognition. The achieved classification accuracy outperforms the state-of-the-art supervised dynamic 3D facial expression recognition techniques.