Video picture-in-picture detection using spatio-temporal slicing

Abstract

For video copy detection and near-duplicate retrieval applications, picture-in-picture (PiP) is one of widely-used but especially difficult transformations to be detected. Traditionally, PiPs in a video are detected by extracting edges within key frames sampled from the video. However, without taking the temporal continuity between frames into account, the performance of these frame-based methods is not that promising. In this paper, we propose a new video PiP detection method by introducing spatio-temporal slicing (STS) to establish the corresponding edge surface probability measurement. An optimization algorithm is then designed to refine vertical and horizontal edge lines by filtering noisy edges. This PiP detection method can be used to improve the performance of video copy detection particularly in the case of the most challenging PiP transformation. The experimental results on the TRECVID-CCD 2010 dataset demonstrate the effectiveness and efficiency of the proposed method.