Forgery Detection Scheme of Deep Video Frame-rate Up-conversion Based on Dual-stream Multi-scale Spatial-temporal Representation

Abstract

Video Frame-Rate Up-Conversion (FRUC) is originally designed to produce the high frame-rate video by periodically inserting new frames between two adjacent frames. However, it can also be utilized to synthesize the faked high frame-rate videos or spliced videos for malicious intents. The existing FRUC detection methods can efficiently identify its occurrence by exploring the blurring effects or deformed structures left over from the traditional FRUC methods. But the video FRUC has been substantially improved in the past years, especially in this deep learning era. Deep learning-based video FRUC (Deep FRUC) weakens the visual traces of the traditional ones such that it is challenging for the current FRUC detectors. In this paper, we propose a forensics algorithm based on dual-stream multi-scale spatial-temporal representation for Deep FRUC. Specifically, we develop a multi-scale receptive field strategy, and attention scheme to learn hidden tampered trace and spatial-temporal representation, respectively. Besides, frame residual and noise residual attention streams are complementary learning from the spatial-temporal dimension, in which the former can capture contrast differences and tampered abnormal boundaries, while the latter can explode noise inconsistencies between original and tampered frames. Experimental results show that the proposed algorithm can effectively achieve the best detection accuracy compared with the existing FRUC forensics works under the Deep FRUC video datasets.