Uncertainty-aware hierarchical labeling for face forgery detection

Abstract

In this paper, we propose an uncertainty-aware hierarchical labeling method for face forgery detection, which aims to simultaneously explore the traces of forgery contents from a hierarchical level. Unlike existing face forgery detection methods that usually focus on local regions or entire images, our proposed approach takes advantage of hierarchical labeling as auxiliary supervised signals to capture hybrid information from pixel-level, patch-level and image-level. Moreover, we utilize Transformer architecture to extract the patch-level information to build a bridge between the pixel-level and the image-level information. However, the face forgery ground truth always carries data uncertainty due to the existence of the blending step and the compression process during face image manipulation. Thus we introduce an uncertainty learning method to formulate and leverage the data uncertainty. Extensive experimental results on five public datasets demonstrate that our approach not only achieves very competitive performance but also improves the generalization ability.