Key Frame Extraction from Video Based on Determinant-Type of Sparse Measure and DC Programming

Abstract

Video is human’s favorite multimedia data type due to its abundant amount of information and intuitive experience compared with text, audio, and image. With rapid progress of computer and network technologies, the amount of video data increases fast, massive storage and frequent retrieval inevitably lead to huge spatio-temporal cost, and how to manage the massive video data efficiently becomes a challenging issue. Key frame extraction is considered as one of the most critical issues in video processing. In this paper, we introduce a novel key frame extraction method based on sparse modeling. Assume that each video frame signal can be expressed as a linear combination of the representative key frames and formulate the problem of finding the representatives as a sparse vector problem. We consider a sparsity measure that is based on the determinant of the Gram matrix of the signals. Based on this measure, we propose a novel key frame extraction formulation based on sparse modeling with the determinant measure of sparsity. The formulation can be expressed as the difference of two convex functions, making the objective function neither convex nor concave. Thus the formulation cannot be easily solved by standard convex optimization methods. Difference of convex (DC) programming is introduced to solve the optimization problem.