Facial Action Unit Detection Based on Transformer and Attention Mechanism

Abstract

Facial Action Unit (AU) detection is a key step in facial expression recognition and analysis. The recent AU detection methods usually use well-designed complex networks based on CNN or RNN. Here, we propose a novel facial action unit detection network based on Transformer and Attention Mechanism named TAM-Net, which combines the attention mechanism with the Transformer structure. Firstly, since the facial AU is an atomic muscle of the face, which usually occurs in a relatively fixed region, we introduce a fixed-position attention mechanism to explicitly guide the feature learning of the region of interest. Secondly, we also propose an attention adjustment mechanism to adaptively refine the attention map based on label information. Finally, the Transformer based on self-attention has been applied to many Computer Vision tasks and has achieved good performance, so we introduce the popular Transformer structure to automatically learning the relationship between different facial AUs. Experiments on the challenging BP4D dataset show that the proposed method achieves competitive results.