Online Kernel-Based Structured Output SVM for Early Expression Detection

Abstract

As a key component of human-computer intelligent interaction and many real-world applications, the real-time property of facial expression recognition is especially important. However, the recognition result of conventional video-based approaches can not be given until the entire video is finished. In this letter, we deal with early expression detection, which aims to identify the expression as early as possible before its ending. This is a relatively new and challenging problem. Max-margin early event detector (MMED) is a well-known framework, which can make early detection. However, the linearity restricts its applications. We thus introduce kernel learning to model the nonlinear structure of complex data distribution. Moreover, the model is further reformulated in an online setting to address the streaming videos. The high retraining cost and large memory requirement of MMED are thus significantly reduced. In addition, we employ AlexNet architecture to make further comparison with mid-level features. Experiments on two popular video-based expression datasets demonstrate both the effectiveness and efficiency of the proposed method.