Image Forgery Detection using Deep Learning and Signal Processing Methods

Abstract

In this thesis, we propose several new approaches for image forgery detection. In the first approach, we use inconsistencies in an imaging trace called lateral chromatic aberration (LCA) to detect forged image regions. Lateral chromatic aberration arises due to the wavelength dependence in the refraction angle of light in camera lenses, which causes a predictable misalignment of an image's color channels. During a splicing forgery, inconsistencies in the image's LCA are inherently introduced. To detect forgeries, we first propose a statistical model that captures the inconsistency between global and local estimates of LCA. We then use this model to pose forgery detection as a hypothesis testing problem and derive an optimal detection statistic. We conduct a series of experiments that demonstrate our proposed method significantly outperforms prior art. We additionally propose a new method to anti-forensically remove these inconsistencies to avoid detection, as well as a new anti-forensic counter method that detects this anti-forensic attack. A drawback of many existing deep-learning based forensic approaches is that they assume a closed set of classes. In our second approach, we propose a method to measure forensic similarity between two image patches that is effective on unknown classes (i.e. an open set). We show that this approach is useful for splicing localization and detection. In our forensic similarity approach, we first train a convolutional neural network (CNN) to output generalized features which encode camera model and editing information of an image patch. Then, we learn a similarity measure that maps pairs of these features to a score that quantifies whether the two image patches have the same or different forensic traces. We experimentally show that this approach can determine whether two image patches were captured by the same or different camera model, processed by the same or different manipulation, and even same or different manipulation parameter. Finally, we propose a graph-based method to more accurately perform forgery detection and localization on tampered images. To do this, we propose an abstract, graph-based representation of an image, which we call the Forensic Graph Representation. In this representation, small image patches are represented by graph vertices with edges assigned according the the forensic similarity between image patches. Localized tampering introduces unique structure into this graph, which align with a concept referred to as ``communities in graph-theory literature. These communities correspond to the tampered and unaltered regions in the image, and are each a sub-set of vertices that contain high weight edges within the community, and low weight edges across communities. As a result, forgery is performed by identifying whether multiple communities exist in this graph representation, and forgery localization is performed by partitioning the communities. We experimentally show that this approach outperforms naive implementations that do not consider this community…