Feature and label relation modeling for multiple-facial action unit classification and intensity estimation

Abstract

In this paper, we propose multiple facial Action Unit (AU) recognition and intensity estimation by modeling their relations in both feature and label spaces. First, a multi-task feature learning method is adopted to learn the shared features among the group of facial action units, and recognize or estimate their intensity simultaneously. Second, a Bayesian network is used to model the co-existent and mutual-exclusive semantic relations among action units. Finally, through probabilistic inference, the learned Bayesian network combines the results of the multi-task learning with the AU relations it captures to perform multiple AU recognition and AU intensity estimation. Experiments on the extended Cohn-Kanade database, the MMI database, the McMaster database and the DISFA database demonstrate the effectiveness of our method for both AU classification and AU intensity estimation.