Regressive Pseudo Label for Weakly Supervised Facial Landmark Detection

Abstract

The progress of the deep neural network and visual sensors promote the facial landmark detection. However, faces are easily collected from Internet of Things (IoT) devices worldwide but are hard-labeled facial landmarks in consistent style. Though many weak supervision (WS) algorithms and theories have been proposed to handle the labeling problem, most of them are tailored for classification tasks and fail in regression tasks, especially in facial landmark detection. To tackle this WS regression task, first, we propose a regressive pseudo-labeling method by analyzing the cluster assumption of facial landmark detection, where overlaps are reduced and interval areas are increased among clusters of facial parts for unlabeled faces. Moreover, auxiliary information and domain loss are utilized to adapt model to samples with different styles. Second, we design a generator–regressor network to first estimate facial boundary attentions and then to locate facial landmarks. However, when generating pseudo labels based on predictive models automatically, there are two major issues. One is that the results of the multistage network highly depend on the former-stage accuracy, and another is that jointing different annotation styles always produces ambiguity feature representation. Thus, we propose two ideas. During training, the generator and regressor are decoupled to alleviate the inner dependence of the multistage network, and the heatmap discriminator is introduced to improve the quality of the predicted facial boundary. We design a transformer structure to fuse face image and boundary attentions, so that it can further complement useful features. Based on these ideas, our methods can automatically annotate accurate pseudo facial landmarks for unlabeled faces. Extensive experiments show that our model achieves good performance on different benchmark data sets, both in accuracy and efficiency.