MC-LCR: Multimodal contrastive classification by locally correlated representations for effective face forgery detection

Abstract

As the remarkable development of facial manipulation technologies is accompanied by severe security concerns, face forgery detection has spurred recent research. Most detection methods train a binary classifier under global supervision to judge whether a face is real or fake. However, advanced manipulations only perform small-scale tampering, posing challenges to comprehensively capturing subtle and local forgery artifacts, especially in high-compression settings and cross-dataset scenarios. To address such limitations, we propose a framework, multimodal contrastive classification by locally correlated representations (MC-LCR), for effective face forgery detection. Instead of specific appearance features, MC-LCR amplifies implicit local discrepancies between authentic and forged faces from both the spatial and frequency domains. A shallow style representation block measures the pairwise correlation of shallow feature maps, encoding local style information to extract more discriminative features in the spatial domain. We observe that subtle forgery artifacts can be further exposed in the patch-wise phase and amplitude spectrum, and that they exhibit different clues. According to the complementarity of amplitude and phase information, we develop a patch-wise amplitude and phase dual attention module to capture locally correlated inconsistencies in the frequency domain. The collaboration of supervised contrastive loss with cross-entropy loss helps the network learn more discriminative and generalized representations. Through extensive experiments and comprehensive studies, we achieve state-of-the-art performance and demonstrate the robustness and generalization of our method.