Comparison of human and machine performance for copy-move image forgery detection involving similar but genuine objects

Abstract

Copy-move forgery (CMF) is considered easier to detect than general forgery mechanisms, but detecting it in the presence of multiple similar but genuine scene objects (SGOs) is non-trivial. We study the efficacy of human visual perception for copy-move image forgery detection (CMFD) involving SGOs, and compare the same with machine performance. Via an eye tracking study performed with 16 users where pairs of images (one real and the other tampered) were displayed in either parallel or serial fashion, we make the following observations: (1) Forgery detection is quicker and more accurate when images are spatially aligned and presented serially, so that the tampering is conspicuous. (2) Eye fixations focus on corresponding regions of the real and tampered images, with fewer and more localized fixations noted during serial comparison. (3) A gap is noted between CMFD performance of humans and machines, with each being more sensitive to different tampering factors. Overall, results reveal the need for systematic visual comparisons to distinguish SGOs from forged objects, as well as the promise of a human-machine collaborative framework to this end.