Video Shot Research Articles

Video Shot Boundary Detection Using Normalized Periodogram Distance Metric

Video shot boundary detection is the primary task for content based video management and retrieval system. This paper proposes a shot boundary detection strategy by exploiting the pros of Normalized Periodogram for efficiently representing the content of the video. A Normalized Periodogram based distance metric to detect the key frames using shot boundary, namely Distance- Left-Right (DLR), is addressed, which is computed on a sliding sub-window basis. The DLR sequence is used to detect the suspected shot boundary frames and a transition type detection procedure is adapted to these suspected frames for discriminating the abrupt and gradual transitions. The proposed shot boundary detection methodology yields Precision—95.02%, Recall—93.15% and F1 score—94.07% for cut, Precision—86.57%, Recall—86.67% and F1 score—86.61% for gradual, Precision—90.6%, Recall—90.02% and F1 score—90.3% for overall transitions. Experimental results show that the proposed approach is superior to the recently available shot boundary detection techniques because of its robustness and simplicity, and presents an effective distance metric to detect the shot boundary.

Video Partitioning and Secured Keyframe Extraction of MPEG Video

Video partitioning and keyframe extraction (KFE) are the key foundations of video analysis and Content based video retrieval. The use of keyframes reduces the amount of data that is necessary in video indexing and provides the outline for dealing with the video content. In the last few years, many algorithms of keyframe extraction concentrated on the original compressed video stream. This can increase computational complexity when decompression is required before video processing. Keyframe is the frame, which can be a prototype of the salient content and information of the shot. The keyframes extracted must summarize the significant features of the video in time sequence. Therefore, there is a commensurate need of an efficient and secured keyframe selection technique in an efficient CBVR system. We propose an algorithm for keyframe extraction of compressed video shots using adaptive threshold method. Extensive computation on 200 plus video clips is performed and results are accurate and satisfactory.

A Genetic Algorithm and Fuzzy Logic Approach for Video Shot Boundary Detection.

This paper proposed a shot boundary detection approach using Genetic Algorithm and Fuzzy Logic. In this, the membership functions of the fuzzy system are calculated using Genetic Algorithm by taking preobserved actual values for shot boundaries. The classification of the types of shot transitions is done by the fuzzy system. Experimental results show that the accuracy of the shot boundary detection increases with the increase in iterations or generations of the GA optimization process. The proposed system is compared to latest techniques and yields better result in terms of F1score parameter.

A generic framework for optimal 2D/3D key-frame extraction driven by aggregated saliency maps

This paper proposes a generic framework for extraction of key-frames from 2D or 3D video sequences, relying on a new method to compute 3D visual saliency. The framework comprises the following novel aspects that distinguish this work from previous ones: (i) the key-frame selection process is driven by an aggregated saliency map, computed from various feature maps, which in turn correspond to different visual attention models; (ii) a method for computing aggregated saliency maps in 3D video is proposed and validated using fixation density maps, obtained from ground-truth eye-tracking data; (iii) 3D video content is processed within the same framework as 2D video, by including a depth feature map into the aggregated saliency. A dynamic programming optimisation algorithm is used to find the best set of K frames that minimises the dissimilarity error (i.e., maximise similarity) between the original video shots of size N>K and those reconstructed from the key-frames. Using different performance metrics and publicly available databases, the simulation results demonstrate that the proposed framework outperforms similar state-of-art methods and achieves comparable performance as other quite different approaches. Overall, the proposed framework is validated for a wide range of visual content and has the advantage of being independent from any specific visual saliency model or similarity metrics.

Divide-and-conquer based summarization framework for extracting affective video content

Recent advances in multimedia technology have led to tremendous increases in the available volume of video data, thereby creating a major requirement for efficient systems to manage such huge data volumes. Video summarization is one of the key techniques for accessing and managing large video libraries. Video summarization can be used to extract the affective contents of a video sequence to generate a concise representation of its content. Human attention models are an efficient means of affective content extraction. Existing visual attention driven summarization frameworks have high computational cost and memory requirements, as well as a lack of efficiency in accurately perceiving human attention. To cope with these issues, we propose a divide-and-conquer based framework for an efficient summarization of big video data. We divide the original video data into shots, where an attention model is computed from each shot in parallel. Viewer's attention is based on multiple sensory perceptions, i.e., aural and visual, as well as the viewer's neuronal signals. The aural attention model is based on the Teager energy, instant amplitude, and instant frequency, whereas the visual attention model employs multi-scale contrast and motion intensity. Moreover, the neuronal attention is computed using the beta-band frequencies of neuronal signals. Next, an aggregated attention curve is generated using an intra- and inter-modality fusion mechanism. Finally, the affective content in each video shot is extracted. The fusion of multimedia and neuronal signals provides a bridge that links the digital representation of multimedia with the viewer’s perceptions. Our experimental results indicate that the proposed shot-detection based divide-and-conquer strategy mitigates the time and computational complexity. Moreover, the proposed attention model provides an accurate reflection of the user preferences and facilitates the extraction of highly affective and personalized summaries.

Symbolic Representation of Sign Language at Sentence Level

In this paper, we propose a model for recognition of sign language being used by communication impaired people in the society. A novel method of extracting features from a video sequence of signs is proposed. Key frames are selected from a given video shots of signs to reduce the computational complexity yet retaining the significant information for recognition. A set of features is extracted from each key frame to capture the trajectory of hand movements made by the signer. The same sign made by different signers and by the same signers at different instances may have variations. The concept of symbolic data particularly interval type data is used to capture such variations and to efficiently represent signs in the knowledgebase. A suitable similarity measure is explored for the purpose of matching and recognition of signs. A database of signs made by communication impaired people of Mysore region is created and extensive experiments are conducted on this database to demonstrate the performance of the proposed approach.

Mosaicing of Text Contents from Consecutive Frames in Pedestal Shot Videos

Mosaicing of Text Contents from Consecutive Frames in Pedestal Shot Videos

The effect of using interactive video on digital and technical performance level in the triple jump competition

The interactive video is considered one of the innovations of our contemporary age; it is used to present visual and audio information according to the learner’s responses. The image and sound are displayed via a display screen which is part of an integrated unit consisting of a computer, information input and storage device. The interactive video can introduce information using video shots and still frames along with texts, drawings and voices. It also displays the video shots in parts each on separate screen. Thus the display depends on the multi-screens system in order to introduce different lessons. In addition, the computer allows the learner to interact which enables her to control according to her potentials and speed of learning. (103: 17)

AN ADAPTIVE ORGANIZATION METHOD OF GEOVIDEO DATA FOR SPATIO-TEMPORAL ASSOCIATION ANALYSIS

Abstract. Public security incidents have been increasingly challenging to address with their new features, including large-scale mobility, multi-stage dynamic evolution, spatio-temporal concurrency and uncertainty in the complex urban environment, which require spatio-temporal association analysis among multiple regional video data for global cognition. However, the existing video data organizational methods that view video as a property of the spatial object or position in space dissever the spatio-temporal relationship of scattered video shots captured from multiple video channels, limit the query functions on interactive retrieval between a camera and its video clips and hinder the comprehensive management of event-related scattered video shots. GeoVideo, which maps video frames onto a geographic space, is a new approach to represent the geographic world, promote security monitoring in a spatial perspective and provide a highly feasible solution to this problem. This paper analyzes the large-scale personnel mobility in public safety events and proposes a multi-level, event-related organization method with massive GeoVideo data by spatio-temporal trajectory. This paper designs a unified object identify(ID) structure to implicitly store the spatio-temporal relationship of scattered video clips and support the distributed storage management of massive cases. Finally, the validity and feasibility of this method are demonstrated through suspect tracking experiments.

A Novel 2D-to-3D Video Conversion Method Using Time-Coherent Depth Maps.

In this paper, we propose a novel 2D-to-3D video conversion method for 3D entertainment applications. 3D entertainment is getting more and more popular and can be found in many contexts, such as TV and home gaming equipment. 3D image sensors are a new method to produce stereoscopic video content conveniently and at a low cost, and can thus meet the urgent demand for 3D videos in the 3D entertaiment market. Generally, 2D image sensor and 2D-to-3D conversion chip can compose a 3D image sensor. Our study presents a novel 2D-to-3D video conversion algorithm which can be adopted in a 3D image sensor. In our algorithm, a depth map is generated by combining global depth gradient and local depth refinement for each frame of 2D video input. Global depth gradient is computed according to image type while local depth refinement is related to color information. As input 2D video content consists of a number of video shots, the proposed algorithm reuses the global depth gradient of frames within the same video shot to generate time-coherent depth maps. The experimental results prove that this novel method can adapt to different image types, reduce computational complexity and improve the temporal smoothness of generated 3D video.

Effect of Patch-Light Demonstration on Observational Learning of Golf Swing

Video display of a model has been widely used as an observational learning tool; yet, the use of patch-light techniques that can highlight key features of motion has been seldom used. The purpose of this study was to determine the effectiveness of patch-light display for use in observational learning of the golf swing. Twenty-two novice golf participants were randomly assigned to three groups: video, patchlight, and control. Each participant performed 100 golf pitch shots, and 20 shots 24 hours later (retention). After each 10 shots, participants observed a recorded model perform 10 shots either in traditional video or patch-light format, or waited for a brief time period (control). Participants did not observe the model during retention. The patch-light group was neither more accurate nor more consistent than the video group or control. We believe that the familiarity of the task may have overridden any observational learning effect.

A Quick Algorithm to Search and Detect Video Shot Changes

The cost of video processing computation is very high and shot boundary detection is one of the reasons for this cost. Shot?s scope determination is an essential part in most of the video processing applications, especially for video indexing and content based video retrieval. This paper introduces a fast algorithm for searching the shots region and detecting their location, based on the divide and conquers technique. The main advantage of this method is due to skipping large parts of the video in the search step, which results in computational load reduction, leading to detection speed increase.

A KPCA spatio-temporal differential geometric trajectory cloud classifier for recognizing human actions in a CBVR system

We describe a new algorithm for distinguishing human actions in videos, called the differential geometric trajectory cloud (DGTC) method that captures both fine and large scale structure of the covariant transformed spatio-temporal optical flow field. We show the utility of our algorithm in the context of a content based video retrieval (CBVR) system, where specific frames from a full length video (or separate video shots in a database) are identified containing a queried human action. In the DGTC method, the local geometry of the spatio-temporal covariant eigenspace curves, unique to each human action, are characterized by the Frenet–Serret basis equations, thereby specifying the local time averaged curvature and torsion, as well as providing a means for defining a mean osculating hyperplane for the entire trajectory. To classify a human action from a query, our system uses an adaptive distance metric between the covariant transformed query trajectory and each of the trajectories from all of the actions in the training set. Based upon the separation of between the query and each class, the distance uses either large or small scale information about the trajectory: for large separations, the distance is the separation between trajectory cloud centroids, while for small and intermediate separations the distance is based upon the mean hyperplane orientation obtained from the time averaged curvature and torsion of the trajectory. Our system can function in real-time and has an accuracy greater than 93% for multiple action recognition within video repositories. We also demonstrate the use of our CBVR system locating specific frame positions of trained actions in two full featured films.

Forensic and anti-forensic techniques for video shot editing in H.264/AVC

This research investigates the forensics and anti-forensics of video editing operations, including shot insertion and deletion, which are common practices of content tampering. The proposed methodology is based on the fact that excessive prediction residuals in the H.264/AVC encoding may appear in the inter-coded frames that were intra-coded. For the anti-forensics, the relationships of coding modes in consecutive frames are employed to determine whether the intra-prediction can be applied during the re-encoding process of the tampered video. The correspondences of coding parameters and the bit-rates are then examined to predict the targeted distribution of quantization indices. Some nonzero indices are modified for satisfying the targeted distribution and generating a video that does not seem an edited one. Next, two forensic methods are presented. The first method evaluates the energy of resulting residuals after the deblocking filtering to reveal the editing operations that have been applied. The second method makes use of the rate control mechanism to check the quantization parameters. Although there exist certain limitations, the methods deserve more discussions under appropriate conditions.

Object based watermarking for H.264/AVC video resistant to rst attacks

In this paper, a compressed domain video watermarking scheme is proposed which embeds the watermark in the homogeneous moving object within a shot of video sequence to resist geometric attacks such as rotation, scaling etc. Intuitively, object based watermarking results low payload and has the least impact on visual quality since the object area is generally small and highly textured. The proposed work has two main contributions, firstly, an existing compressed domain motion coherent block detection algorithm [7] is extended to detect the moving objects within a video shot and secondly, a watermarking scheme has been proposed by embedding within the moving objects to resist RST attacks. A comprehensive set of experiments has been carried out to justify the applicability of the proposed scheme over the existing literature.

Use of trajectory and spatiotemporal features for retrieval of videos with a prominent moving foreground object

This paper presents generalized spatiotemporal analysis and lookup tool (GESTALT), an unsupervised framework for content-based video retrieval. GESTALT takes a query video and retrieves “similar” videos from the database. Motion and dynamics of appearance (shape) patterns of a prominent moving foreground object are considered as the key components of the video content and captured using corresponding feature descriptors. GESTALT automatically segments the moving foreground object from the given query video shot and estimates the motion trajectory. A graph-based framework is used to explicitly capture the structural and kinematics property of the motion trajectory, while an improved version of an existing spatiotemporal feature descriptor is proposed to model the change in object shape and movement over time. A combined match cost is computed as a convex combination of the two match scores, using these two feature descriptors, which is used to rank-order the retrieved video shots. Effectiveness of GESTALT is shown using extensive experimentation, and comparative study with recent techniques exhibits its superiority.

Video communication and ‘camera actions’: The production of wide video shots in courtrooms with remote defendants

We consider here the use of videoconference for remote testimonies in the courtroom. Based on video recordings of actual hearings with remote participants, we analyze the systematic organization of camera motions in this setting, and show how they constitute interactional moves in their own right, i.e. ‘camera actions’, characteristic of ‘video communication contextures’. We focus on the production of wide shot, as a situated and timed camera-mediated accomplishment in the course of the hearing, and show: (a) how such an accomplishment is sensitive to its sequential environment; (b) how it is accountable as a way to mark the particular relevance of a group of participants with respect to the ongoing talk, and therefore oriented-to as a resource to visually highlight collective forms of speakership or recipiency; (c) how such camera motions are sequentially relevant and sequentially implicative with respect to the ongoing video interaction; (d) how such camera motion, and more generally the video communication ecologies which enable them make visible a particular member's interactional competence, that of being able to recognize the relevance of subtle changes in participation frames, routinely, unreflexively and on the fly.

Video segmentation using minimum ratio similarity measurement

Video segmentation plays an essential role in digital video processing, pattern recognition, security, video conferencing, etc. The convenience of the video is based on its content which is still impossible. One major challenging task of automatic video indexing is automatic detection of video shots. In this paper, a new algorithm is proposed to detect the shot boundary by using the minimum ratio similarity measurement between the characteristic features of two consecutive frames. Where, diverse parameters are calculated for each frame that creates a feature vector of size 40. The system performance is measured in terms of metric parameters. Also, a comparative study with alternative algorithms such as rapid cut detection, histogram-based method, etc. is done. Results suggest that the precision performance of the algorithm is independent of the nature of the video. The F-measure performance comparison shows that the proposed algorithm is the best with maximum average value and minimum standard deviation.

A total variation-based hierarchical radial video visualization method

To clearly visualize the video structure and provide a user-friendly video content review and retrieval system, we first apply total variation (TV) metric to video shot segmentation and propose a novel video visualization method in this paper. Firstly, video data visualization design principles are proposed based on the analysis of video data characteristics. Secondly, a TV metric is proposed to measure the distance between two video frames. Then, video shots are detected by TV metric, video key frames are extracted by k-means algorithm, and similar shots are retrieved based on frame dominant color. Finally, according to the proposed video visualization design principles, a hierarchical radial video visualization method is presented. Experimental results show that the proposed video visualization method can express the video content and the whole structure directly and clearly.

VIDCAR: an unsupervised CBVR framework for identifying similar videos with prominent object motion

This paper presents VIDeo Content Analysis and Retrieval (VIDCAR), an unsupervised framework for Content-Based Video Retrieval (CBVR) using representation of the dynamics in the spatio-temporal model extracted from video shots. We propose Dynamic Multi Spectro Temporal-Curvature Scale Space (DMST-CSS), an improved feature descriptor for enhancing the performance of CBVR task. Our primary contribution is in representation of the dynamics of the evolution of the MST-CSS surface. Unlike the earlier MST-CSS descriptor [22], which extracts geometric features after the evolving MST-CSS surface converges to a final formation, this DMST-CSS captures the dynamics of the evolution (formation) of the surface and is thus more robust. We have represented the dynamics of MST-CSS surface as a multivariate time series to obtain a DMST-CSS descriptor. A global kernel alignment technique has been adapted to compute a match cost between query and model DMST-CSS descriptor. In our experiments, VIDCAR was shown to have greater precision recall than the competitors on five datasets.