Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

GLFF: Global and Local Feature Fusion for Face Forgery Detection

Published

Author(s)

Haiying Guan, Yan Ju, Shan Jia, Jialing Cai, Siwei Lyu

Abstract

With the rapid development of the deep generative models (such as Generative Adversarial Networks and Auto-encoders), AI-synthesized images of human face are now of such high qualities that humans can hardly distinguish them from pristine ones. Although existing detection methods have shown high performance in specific evaluation settings, e.g., on images with observable artifacts or from seen models, they tend to suffer serious performance degradation in real-world scenarios where testing images can be generated by more powerful generation models or combined with various post-processing operations. To address this issue, we propose a Global and Local Feature Fusion (GLFF) framework to learn rich and discriminative representations by combining multi-scale global features from the whole image with refined local features from informative patches for face forgery detection. Specifically, the global branch extracts and fuses low-level textual features and high-level semantic information to exploit the artifacts from multiple scale of the entire image. The fused features are utilized to guide the proposed patch selection module to select patches with more information for classification. These selected patches are then fed into the local branch to extract more detailed local artifacts. Finally, features extracted by two branches are combined using attention mechanism for binary classification. Due to the lack of comprehensive face forgery dataset for evaluation, we further create a challenging face forgery dataset, named DeepFakeFaceForensics (DF^3), which contains 6 state-of-the-art generation models and a variety of post-processing techniques to approach the real-world applications. Experimental results demonstrate the superiority of our method to the state-of-the-art methods on the proposed DF^3 dataset and three other open-source datasets.

Citation
arxiv

Keywords

Deepfakes, Forgery Detection, Artificial Intelligence

Citation

Guan, H. , Ju, Y. , Jia, S. , Cai, J. and Lyu, S. (2022), GLFF: Global and Local Feature Fusion for Face Forgery Detection, arxiv, [online], https://doi.org/10.48550/arXiv.2211.08615 (Accessed December 14, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created November 15, 2022, Updated February 22, 2023