PRRNet: Pixel-Region relation network for face forgery detection

Abstract

As advanced facial manipulation technologies develop rapidly, one can easily modify an image by changing the identity or the facial expression of the target person, which threatens social security. To address this problem, face forgery detection becomes an important and challenging task. In this paper, we propose a novel network, called Pixel-Region Relation Network (PRRNet), to capture pixel-wise and region-wise relations respectively for face forgery detection. The main motivation is that a facial manipulated image is composed of two parts from different sources, and the inconsistencies between the two parts is a significant kind of evidence for manipulation detection. Specifically, PRRNet contains two serial relation modules, i.e. the Pixel-Wise Relation (PR) module and the Region-Wise Relation (RR) module. For each pixel in the feature map, the PR module captures its similarities with other pixels to exploit the local relations information. Then, the PR module employs a spatial attention mechanism to represent the manipulated region and the original region separately. With the representations of the two regions, the RR module compares them with multiple metrics to measure the inconsistency between these two regions. In particular, the final predictions are obtained totally based on whether the inconsistencies exist. PRRNet achieves the state-of-the-art detection performance on three recent proposed face forgery detection datasets. Besides, our PRRNet shows the robustness when trained and tested on different image qualities.