Video summary generation by visual shielding compressed sensing coding and double-layer affinity propagation

Abstract

Video summary technology based on keyframe extraction is an effective means to rapidly access video content. Traditional video summary generation technology requires high video resolution, which poses a problem as most existing studies have no targeted solutions for videos that are subject to privacy protection. We propose a novel keyframe extraction algorithm for video data in the visual shielding domain, named visual shielding compressed sensing coding and double-layer affinity propagation (VSCS-DAP). VSCS-DAP involves three main steps. First, the video is compressed by compressed sensing technology to provide a visual shielding effect (protecting the privacy of monitored figures), while the data volume is significantly reduced. Then, pyramid histogram of oriented gradients (PHOG) features are extracted from the compressed video to be clustered by the first step affinity propagation (AP) to gain the summaries of the first stage. Finally, the PHOG and Hist fusion features are extracted from the keyframes of the first stage, and they cluster the fused PHOG-Hist features by the second step AP algorithm to obtain the final output summaries. Experimental results obtained on two common video datasets show that our method exhibits advantages including low redundancy and few missing frames, low computational complexity, strong real-time performance, and robustness to vision-shielded video.