Number Of Key Frames Research Articles

RPCA-KFE: Key Frame Extraction for Video Using Robust Principal Component Analysis.

Key frame extraction algorithms consider the problem of selecting a subset of the most informative frames from a video to summarize its content. Several applications, such as video summarization, search, indexing, and prints from video, can benefit from extracted key frames of the video under consideration. Most approaches in this class of algorithms work directly with the input video data set, without considering the underlying low-rank structure of the data set. Other algorithms exploit the low-rank component only, ignoring the other key information in the video. In this paper, a novel key frame extraction framework based on robust principal component analysis (RPCA) is proposed. Furthermore, we target the challenging application of extracting key frames from unstructured consumer videos. The proposed framework is motivated by the observation that the RPCA decomposes an input data into: 1) a low-rank component that reveals the systematic information across the elements of the data set and 2) a set of sparse components each of which containing distinct information about each element in the same data set. The two information types are combined into a single l1-norm-based non-convex optimization problem to extract the desired number of key frames. Moreover, we develop a novel iterative algorithm to solve this optimization problem. The proposed RPCA-based framework does not require shot(s) detection, segmentation, or semantic understanding of the underlying video. Finally, experiments are performed on a variety of consumer and other types of videos. A comparison of the results obtained by our method with the ground truth and with related state-of-the-art algorithms clearly illustrates the viability of the proposed RPCA-based framework.

A Novel Method for Sensing Obscene Videos using Scene Change Detection

Video scene change detection has great importance of managing and analyzing large amount of videos. Traditionally this technique used for indexing, segmenting and categorizing different types of videos. Very few works addressed to classify obscene using scene change detection method. In this research we proposed a simple approach for sensing objectionable videos by observing scene changes into different video genres. Video scenes are grouped into set of key frames. After analyzing duration of each scene and counting the number of key frames of designated scene, it has been shown that obscene videos have infrequent scene changing nature. While in sports, dramas, music and action films have large number of scene changes. We used six types of video genres and the decision has been made by setting a threshold based on extracted key frames. Experimental result showed that the accuracy is 83.33% and false positive rate is 16.67%.

Compressing 3-D Human Motions via Keyframe-Based Geometry Videos

This paper presents keyframe-based geometry video (KGV), a novel framework for compressing 3-D human motion data by using geometry videos. Given a motion data encoded in a geometry video (GV) format, our method extracts the keyframes and produces a reconstruction matrix. Then it applies the video compression technique (e.g., H.264/Advanced Video Coding) to the reordered keyframes, which can significantly reduce the spatial and temporal redundancy in the KGV. We develop a rate distortion-based optimization algorithm to determine the parameters (i.e., the number of keyframes and quantization parameter) leading to optimal performance. Experimental results show that the proposed KGV framework significantly outperforms the existing GV techniques in terms of both the rate distortion performance and visual quality. Besides, the computational cost of the KGV is rather low at the decoder, making it highly desirable for power-constrained devices. Last but not least, our method can be easily extended to progressive compression with heterogeneous communication network.

A New Framework for Automatic Extraction of Key Frames Using DC Image Activity

The effective extraction of key frames from a video stream is an essential task for summarizing and representing the content of a video. Accordingly, this paper proposes a new and fast method for extracting key frames from a compressed video. In the proposed approach, after the entire video sequence has been segmented into elementary content units, called shots, key frame extraction is performed by first assigning the number of key frames to each shot, and then distributing the key frames over the shot using a probabilistic approach to locate the optimal position of the key frames. Moreover, we implement our proposed framework in Android to confirm the validity, availability and usefulness. The main advantage of the proposed method is that no time-consuming computations are needed for distributing the key frames within the shots and the procedure for key frame extraction is completely automatic. Furthermore, the set of key frames is independent of any subjective thresholds or manually set parameters.

A hybrid approach to keyframe extraction from motion capture data using curve simplification and principal component analysis

In this paper, we propose a novel method to extract keyframes from motion capture data. A hybrid approach, which combines a curve‐simplification algorithm with an initialization procedure including principal component analysis, is adopted. The developed method automatically extracts an appropriate number of keyframes at high speed without performance degradation. Experimental results prove the effectiveness of the present method. © 2014 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Shot boundary-based keyframe extraction for video summarisation

We propose a new automatic video summarisation technique using shot boundary-based keyframe extraction method. Initially, the frames are segmented into shots by identifying the transitions using a new CEM-visual content descriptor colour, edge strength, moments. From the segmented shots, motionless visual content representative frames known as keyframes are extracted. However, the number of keyframes to be extracted for each shot is determined using the shot dispersion ratio. The proposed shot boundary detection algorithm and video keyframe extraction technique are implemented and evaluated over TRECVID dataset. Compared with existing related algorithms, our algorithm yields better Fl-score of 94% for shot boundary detection. Also, our algorithm yields average fidelity value of 0.80 and average compression rate of 0.88% for video summary.

Action recognition from a different view

In this paper, we propose a novel approach to recognise human activities from a different view. Although appearance-based recognition methods have been shown to be unsuitable for action recognition for varying views, there must be some regularity among the same action sequences of different views. Self-similarity matrices appear to be relative stable across views. However, the ability to effectively realise this stability is a problem. In this paper, we extract the shape-flow descriptor as the low-level feature and then choose the same number of key frames from the action sequences. Self-similarity matrices are obtained by computing the similarity between any pair of the key frames. The diagonal features of the similarity matrices are extracted as the highlevel feature representation of the action sequence and Support Vector Machines (SVM) is employed for classification. We test our approach on the IXMAS multi-view data set. The proposed approach is simple but effective when compared with other algorithms.

Optimized selection of key frames for monocular videogrammetric surveying of civil infrastructure

Videogrammetry is an inexpensive and easy-to-use technology for spatial 3D scene recovery. When applied to large scale civil infrastructure scenes, only a small percentage of the collected video frames are required to achieve robust results. However, choosing the right frames requires careful consideration. Videotaping a built infrastructure scene results in large video files filled with blurry, noisy, or redundant frames. This is due to frame rate to camera speed ratios that are often higher than necessary; camera and lens imperfections and limitations that result in imaging noise; and occasional jerky motions of the camera that result in motion blur; all of which can significantly affect the performance of the videogrammetric pipeline. To tackle these issues, this paper proposes a novel method for automating the selection of an optimized number of informative, high quality frames. According to this method, as the first step, blurred frames are removed using the thresholds determined based on a minimum level of frame quality required to obtain robust results. Then, an optimum number of key frames are selected from the remaining frames using the selection criteria devised by the authors. Experimental results show that the proposed method outperforms existing methods in terms of improved 3D reconstruction results, while maintaining the optimum number of extracted frames needed to generate high quality 3D point clouds.

A NOVEL TECHNIQUE FOR SIZE CONSTRAINED VIDEO STORYBOARD GENERATION USING STATISTICAL RUN TEST AND SPANNING TREE

Storyboard consisting of key-frames is a popular format of video summarization as it helps in efficient indexing, browsing and partial or complete retrieval of video. In this paper, we have presented a size constrained storyboard generation scheme. Given the shots i.e. the output of the video segmentation process, the method has two major steps: extraction of appropriate key-frame(s) from each shot and finally, selection of a specified number of key-frames from the set thus obtained. The set of selected key-frames should retain the variation in visual content originally possessed by the video. The number of key-frames or representative frames in a shot may vary depending on the variation in its visual content. Thus, automatic selection of suitable number of representative frames from a shot still remains a challenge. In this work, we propose a novel scheme for detecting the sub-shots, having consistent visual content, from a shot using Wald–Wolfowitz runs test. Then from each sub-shot a frame rendering the highest fidelity is extracted as key-frame. Finally, a spanning tree based novel method is proposed to select a subset of key-frames having specific cardinality. Chronological arrangement of such frames generates the size constrained storyboard. Experimental result and comparative study show that the scheme works satisfactorily for a wide variety of shots. Moreover, the proposed technique rectifies mis-detection error, if any, incurred in video segmentation process. Similarly, though not implemented, the proposed hypothesis test has ability to rectify the false-alarm in shot detection if it is applied on pair of adjacent shots.

New Framework for Automated Extraction of Key Frames from Compressed Video

The effective extraction of key frames from a video stream is an essential task for summarizing and representing the content of a video. Accordingly, this paper proposes a new and fast method for extracting key frames from a compressed video. In the proposed approach, after the entire video sequence has been segmented into elementary content units, called shots, key frame extraction is performed by first assigning the number of key frames to each shot, and then distributing the key frames over the shot using a probabilistic approach to locate the optimal position of the key frames. The main advantage of the proposed method is that no time-consuming computations are needed for distributing the key frames within the shots and the procedure for key frame extraction is completely automatic. Furthermore, the set of key frames is independent of any subjective thresholds or manually set parameters.

Real-time feature-based image morphing for memory-efficient impostor rendering and animation on GPU

Real-time rendering of large animated crowds consisting of thousands of virtual humans is important for several applications including simulations, games, and interactive walkthroughs but cannot be performed using complex polygonal models at interactive frame rates. For that reason, methods using large numbers of precomputed image-based representations, called impostors, have been proposed. These methods take advantage of existing programmable graphics hardware to compensate for computational expense while maintaining visual fidelity. Thanks to these methods, the number of different virtual humans rendered in real time is no longer restricted by computational power but by texture memory consumed for the variety and discretization of their animations. This work proposes a resource-efficient impostor rendering methodology that employs image morphing techniques to reduce memory consumption while preserving perceptual quality, thus allowing higher diversity or resolution of the rendered crowds. Results of the experiments indicated that the proposed method, in comparison with conventional impostor rendering techniques, can obtain 38 % smoother animations or 87 % better appearance quality by reducing the number of key-frames required for preserving the animation quality via resynthesizing them with up to 92 % similarity on real time.

Shot boundary detection and key-frame extraction from neurosurgical video sequences

In this work, we present a system for video shot boundary detection and key-frame extraction from video sequences based on colour histograms. The growth of archived video material has made automated indexing and browsing tools necessary. The aim of this paper is to provide a framework for such a video indexing and browsing tool. The proposed system segments video sequences based on colour similarity, and then extracts key-frames by clustering the images. Computations of colour histogram differences and self-similarity modelling have been performed in segmentation process. We have extracted three-, four- and five-frames video summaries with unsupervised k-means clustering. We have also dynamically determined the number of key-frames representing the video content without any prior information based on dominant set clustering. Our results show that our approach is successful in both segmentation and key-frame extraction processes.

Optimization-based key frame extraction for motion capture animation

In this paper, we present a new solution for extracting key frames from motion capture data using an optimization algorithm to obtain compact and sparse key frame data that can represent the original dense human body motion capture animation. The use of the genetic algorithm helps determine the optimal solution with global exploration capability while the use of a probabilistic simplex method helps expedite the speed of convergence. By finding the chromosome that maximizes the fitness function, the algorithm provides the optimal number of key frames as well as the low reconstruction error with an ordinary interpolation technique. The reconstruction error is computed between the original motion and the reconstruction one by the weighted differences of joint positions and velocities. The resulting set of key frames is obtained by iterative application of the algorithm with initial populations generated randomly and intelligently. We also present experiments which demonstrate that the method can effectively extract key frames with a high compression ratio and reconstruct all other non key frames with high quality.

Towards Scalable Summarization of Consumer Videos Via Sparse Dictionary Selection

The rapid growth of consumer videos requires an effective and efficient content summarization method to provide a user-friendly way to manage and browse the huge amount of video data. Compared with most previous methods that focus on sports and news videos, the summarization of personal videos is more challenging because of its unconstrained content and the lack of any pre-imposed video structures. We formulate video summarization as a novel dictionary selection problem using sparsity consistency, where a dictionary of key frames is selected such that the original video can be best reconstructed from this representative dictionary. An efficient global optimization algorithm is introduced to solve the dictionary selection model with the convergence rates as <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (1/ <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) (where <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> is the iteration counter), in contrast to traditional sub-gradient descent methods of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (1/√ <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> ). Our method provides a scalable solution for both key frame extraction and video skim generation, because one can select an arbitrary number of key frames to represent the original videos. Experiments on a human labeled benchmark dataset and comparisons to the state-of-the-art methods demonstrate the advantages of our algorithm.

Key Frame Assignment for Compr essed Video Based on DC Image Activity

In this paper, we propose a new and fast method for assigning the number of key frames to each shot. At first we segment the entire video sequence into elementary content unit called shots and then the key frame allocation is performed by calculating the accumulated value of Af(activity function). The proposed algorithm is based on the amount of content variation using DC images extracted from compressed video. By assigning the number of key frames to the shot that has the largest value of content function, one key frame is assigned at a time until you run out of given all key frames. The main advantage of our proposed method is that we do not need to use time-exhaustive computations in allocating the key frames over the shot and can perform it fully automatically.

Key Frame Extraction Algorithm Based on Information Theory

Key frames play a very important role in video indexing and retrieval. In this paper, we propose a novel method to extract key frames based on information theory. We use improved Bayesian Information Criterion to determining the number of key frames, and then automatically extract key frames to represent video shot based on information bottleneck cluster method. Experimental results and the comparisons with other methods on various types of video sequences show the effectiveness of the proposed method.

통계적 분석 기반 불법 복제 비디오 영상 감식 방법

The carelessly and illegally copied contents are raising serious social problem as internet and multimedia technologies are advancing. Therefore, development of video copy detection system must be settled without delay. In this paper, we propose the hierarchical video copy detection method that estimates similarity using statistical characteristics between original video and manipulated(transformed) copy video. We rank according to luminance value of video to be robust to spacial transformation, and choose similar videos categorized as candidate segments in huge amount of database to reduce processing time and complexity. The copy videos generally insert black area in the edge of the image, so we remove rig black area and decide copy or not by using statistical characteristics of original video and copied video with center part of frame that contains important information of video. Experiment results show that the proposed method has similar keyframe accuracy to reference method, but we use less memory to save feature information than reference`s, because the number of keyframes is less 61% than that of reference`s. Also, the proposed method detects if the video is copied or not efficiently despite expansive spatial transformations such as blurring, contrast change, zoom in, zoom out, aspect ratio change, and caption insertion.

Equivalent Key Frames Selection Based on Iso-Content Principles

We present a key frames selection algorithm based on three iso-content principles (iso-content distance, iso-content error and iso-content distortion), so that the selected key frames are equidistant in video content according to the used principle. Two automatic approaches for defining the most appropriate number of key frames are proposed by exploiting supervised and unsupervised content criteria. Experimental results and the comparisons with existing methods from literature on large dataset of real-life video sequences illustrate the high performance of the proposed schemata.

Towards Extracting Semantically Meaningful Key Frames From Personal Video Clips: From Humans to Computers

Extracting key frames from video is of great interest in many applications, such as video summary, video organization, video compression, and prints from video. Key frame extraction is not a new problem but existing literature has focused primarily on sports or news video. In the personal or consumer video space, the biggest challenges for key frame selection are the unconstrained content and lack of any pre-imposed structures. First, in a psychovisual study, we conduct ground truth collection of key frames from video clips taken by digital cameras (as opposed to camcorders) using both first- and third-party judges. The goals of this study are to: 1) create a reference database of video clips reasonably representative of the consumer video space; 2) identify consensus key frames by which automated algorithms can be compared and judged for effectiveness, i.e., ground truth; and 3) uncover the criteria used by both first- and third-party human judges so these criteria can influence algorithm design. Next, we develop an automatic key frame extraction method dedicated to summarizing consumer video clips acquired from digital cameras. Analysis of spatio-temporal changes over time provides semantically meaningful information about the scene and the camera operator's general intents. In particular, camera and object motion are estimated and used to derive motion descriptors. A video clip is segmented into homogeneous parts based on major types of camera motion (e.g., pan, zoom, pause, steady). Dedicated rules are used to extract candidate key frames from each segment. In addition, confidence measures are computed for the candidates to enable ranking in semantic relevance. This method is scalable so that one can produce any desired number of key frames from the candidates. Finally, we demonstrate the effectiveness of our method by comparing the results with two alternative methods against the ground truth agreed by multiple judges.

Summarization of 3D Video by Rate-Distortion Trade-off

3D video, which consists of a sequence of mesh models, can reproduce dynamic scenes containing 3D information. To summarize 3D video, a key frame extraction method is developed using rate-distortion (R-D) trade-off. For this purpose, an effective feature vector is extracted for each frame. Shot detection is performed using the feature vectors as a preprocessing followed by key frame extraction. Simple but reasonable definitions of rate and distortion are presented. Based on an assumption of linearity, an R-D curve is generated in each shot, where the locations of the key frames are optimized. Finally, R-D trade-off can be achieved by optimizing a cost function using a Lagrange multiplier, where the number of key frames is optimized in each shot. Therefore, our system will automatically determine the best locations and the number of key frames in the sense of R-D trade-off. Our experimental results show the extracted key frames are compact and faithful to the original 3D video.