Video Content Indexing Research Articles

A new approach to video summary generation

This paper describes a video summary approach for an actual video file that provides a rapid way to assess its information. This is a difficult task to identify relevant or informative sections. When the original video demands knowledge of its content, we build a video summary via extracting in-depth characteristics of each original video. Video summarization deals with the generation of a condensed version of the original video by including meaningful frames or segments while eliminating redundant information provides a concise and accurate representation of the original video clips by providing the most representative summary with increased interest through the application of a matching function between the current and prior frames. The primary goal of this work is to give a strategy for recording video clips using security cameras and converting them into a short video to provide a clear analysis of the video by reducing excess and extracting the key video contents into the real video. In a lengthy video clip, viewers may not have enough time to see the full thing. The viewer may be interested in seeing a specific topic under the label "Important". It may be composed of pieces that may be shared on social media and communicate the semantics of long videos, particularly video snippets that must be submitted to Internet. To browse, will be needed a network with adequate bandwidth. That is, non-affiliated video streaming, indexing, and retrieval of video content makes it valuable to viewers.

Shot boundary detection based on orthogonal polynomial

Shot boundary detection (SBD) is a substantial step in video content analysis, indexing, retrieval, and summarization. SBD is the process of automatically partitioning video into its basic units, known as shots, through detecting transitions between shots. The design of SBD algorithms developed from simple feature comparison to rigorous probabilistic and using of complex models. Nevertheless, accelerate the detection of transitions with higher accuracy need to be improved. Extensive research has employed orthogonal polynomial (OP) and their moments in computer vision and signal processing owing to their powerful performance in analyzing signals. A new SBD algorithm based on OP has been proposed in this paper. The Features are derived from orthogonal transform domain (moments) to detect the hard transitions in video sequences. Moments are used because of their ability to represent signal (video frame) without information redundancy. These features are the moments of smoothed and gradients of video frames. The moments are computed using a developed OP which is squared Krawtchouk-Tchebichef polynomial. These moments (smoothed and gradients) are fused to form a feature vector. Finally, the support vector machine is utilized to detect hard transitions. In addition, a comparison between the proposed algorithm and other state-of-the-art algorithms is performed to reinforce the capability of the proposed work. The proposed algorithm is examined using three well-known datasets which are TRECVID2005, TRECVID2006, and TRECVID2007. The outcomes of the comparative analysis show the superior performance of the proposed algorithm against other existing algorithms.

Content-based Automatic Video Genre Identification

Video content is evolving enormously with the heavy usage of internet and social media websites. Proper searching and indexing of such video content is a major challenge. The existing video search potentially relies on the information provided by the user, such as video caption, description and subsequent comments on the video. In such case, if users provide insufficient or incorrect information about the video genre, the video may not be indexed correctly and ignored during search and retrieval. This paper proposes a mechanism to understand the contents of video and categorize it as Music Video, Talk Show, Movie/Drama, Animation and Sports. For video classification, the proposed system uses audio and visual features like audio signal energy, zero crossing rate, spectral flux from audio and shot boundary, scene count and actor motion from video. The system is tested on popular Hollywood, Bollywood and YouTube videos to give an accuracy of 96%.

Fine‐granularity semantic video annotation

PurposeA fine‐grained video content indexing, retrieval, and adaptation requires accurate metadata describing the video structure and semantics to the lowest granularity, i.e. to the object level. The authors address these requirements by proposing semantic video content annotation tool (SVCAT) for structural and high‐level semantic video annotation. SVCAT is a semi‐automatic MPEG‐7 standard compliant annotation tool, which produces metadata according to a new object‐based video content model introduced in this work. Videos are temporally segmented into shots and shots level concepts are detected automatically using ImageNet as background knowledge. These concepts are used as a guide to easily locate and select objects of interest which are then tracked automatically to generate an object level metadata. The integration of shot based concept detection with object localization and tracking drastically alleviates the task of an annotator. The paper aims to discuss these issues.Design/methodology/approachA systematic keyframes classification into ImageNet categories is used as the basis for automatic concept detection in temporal units. This is then followed by an object tracking algorithm to get exact spatial information about objects.FindingsExperimental results showed that SVCAT is able to provide accurate object level video metadata.Originality/valueThe new contribution in this paper introduces an approach of using ImageNet to get shot level annotations automatically. This approach assists video annotators significantly by minimizing the effort required to locate salient objects in the video.

Human action recognition based on graph-embedded spatio-temporal subspace

Human action recognition is an important issue in the pattern recognition field, with applications ranging from remote surveillance to the indexing of commercial video content. However, human actions are characterized by non-linear dynamics and are therefore not easily learned and recognized. Accordingly, this study proposes a silhouette-based human action recognition system in which a three-step procedure is used to construct an efficient discriminant spatio-temporal subspace for k-NN classification purposes. In the first step, an Adaptive Locality Preserving Projection (ALPP) method is proposed to obtain a low-dimensional spatial subspace in which the linearity in the local data structure is preserved. To resolve the problem of overlaps in the spatial subspace resulting from the ambiguity of the human body shape among different action classes, temporal data are extracted using a Non-base Central-Difference Action Vector (NCDAV) method. Finally, the Large Margin Nearest Neighbor (LMNN) metric learning method is applied to construct an efficient spatio-temporal subspace for classification purposes. The experimental results show that the proposed system accurately recognizes a variety of human actions in real time and outperforms most existing methods. In addition, a robustness test with noisy data indicates that our system is remarkably robust toward noise in the input images.

Scalable object-based video retrieval in HD video databases

With exponentially growing quantity of video content in various formats, including the popularisation of HD (High Definition) video and cinematographic content, the problem of efficient indexing and retrieval in video databases becomes crucial. Despite efficient methods have been designed for the frame-based queries on video with local features, object-based indexing and retrieval attract attention of research community by the seducing possibility to formulate meaningful queries on semantic objects. In the case of HD video, the principle of scalability addressed by actual compression standards is of great importance. It allows for indexing and retrieval on the lower resolution available in the compressed bit-stream. The wavelet decomposition used in the JPEG2000 standard provides this property. In this paper, we propose a scalable indexing of video content by objects. First, a method for scalable moving object extraction is designed. Using the wavelet data, it relies on the combination of robust global motion estimation with morphological colour segmentation at a low spatial resolution. It is then refined using the scalable order of data. Second, a descriptor is built only on the objects extracted. This descriptor is based on multi-scale histograms of wavelet coefficients of objects. Comparison with SIFT features extracted on segmented object masks gives promising results.

Video Content Indexing using Kullback-Leibler Distance

In huge video databases, the effective video content indexing method is required. While manual indexing is the most effective approach to this goal, it is slow and expensive. Thus automatic indexing is desirable and recently various indexing tools for video databases have been developed. For efficient video content indexing, the similarity measure is an important factor. This paper presents new similarity measures between frames and proposes a new algorithm to index video content using Kullback-Leibler distance defined between two histograms. Experimental results show that the proposed algorithm using Kullback-Leibler distance gives remarkable high accuracy ratios compared with several conventional algorithms to index video content.

ViCopT: a robust system for content-based video copy detection in large databases

This paper presents an efficient approach for copy detection in large archives containing several hundred hours of videos, called ViCopT for Video Copy Tracking. Our video content indexing method consists in computing trends of behaviors of points of interest and then to assign them a label of behavior. Two methods are proposed to assign the labels: one uses heuristic tresholds and the other one uses a clustering algorithm. Such an indexing approach has several interesting properties: it provides a rich, compact and generic description, while labels of behavior provide a high-level description of the video content. A dedicated online retrieval method for copy detection is described, compared and evaluated on a large video database (1,000 h). This evaluation is done on a framework proposed for video copy detection: ViCopT displays excellent robustness to various severe signal transformations and the ability to accurately identify copies from highly similar videos. Other evaluation focuses on the flexibility of ViCopT due to the asymmetry of the video description. This allow the system to be highly scalable and very flexible regarding the situation faced: searching for copies on the internet or monitoring TV stream.

The ARGOS campaign: Evaluation of video analysis and indexing tools

The benchmarking of various methods of video analysis and indexing has become a research problem per se. While a lot of effort nowadays is devoted to organization of evaluation campaigns both on international, such as TREC Video and national levels, an in-depth analysis of the performance of the methods still remains an open issue. Several aspects in the evaluation of methods have to be addressed: how to design a benchmarking corpus in order to cover a sufficiently wide range of applications, what are the tasks to address, what are the most relevant evaluation metrics. The Argos evaluation campaign supported by the French Techno-Vision program aimed at developing resources for a benchmarking of video content analysis and indexing methods. The paper describes the type of tasks evaluated, the way the content set has been produced, metrics and tools developed for the evaluations and some of the results obtained at the end of the first phase. Perspectives based on current works will be given to conclude this paper.

Indexing of Fictional Video Content for Event Detection and Summarisation

This paper presents an approach to movie video indexing that utilises audiovisual analysis to detect important and meaningful temporal video segments, that we term events. We consider three event classes, corresponding to dialogues, action sequences, and montages, where the latter also includes musical sequences. These three event classes are intuitive for a viewer to understand and recognise whilst accounting for over 90% of the content of most movies. To detect events we leverage traditional filmmaking principles and map these to a set of computable low-level audiovisual features. Finite state machines (FSMs) are used to detect when temporal sequences of specific features occur. A set of heuristics, again inspired by filmmaking conventions, are then applied to the output of multiple FSMs to detect the required events. A movie search system, named MovieBrowser, built upon this approach is also described. The overall approach is evaluated against a ground truth of over twenty-three hours of movie content drawn from various genres and consistently obtains high precision and recall for all event classes. A user experiment designed to evaluate the usefulness of an event-based structure for both searching and browsing movie archives is also described and the results indicate the usefulness of the proposed approach.

Indexing of Fictional Video Content for Event Detection and Summarisation

Indexing of Fictional Video Content for Event Detection and Summarisation

DAG-based visual interfaces for navigation in indexed video content

Indexing and segmenting of video content by motion, color and texture has been intensively explored leading to a commonly used representation in a storyboard. In this paper, a novel method of visualization of video content is proposed. First of all, the content is segmented into shots, and then a spatio-temporal color signature of shots, based on color distribution in the frames, is proposed. This spatio-temporal color signature serves as a basis for graph clustering and graph visualization tools. Those, integrated in a platform for visualization of huge graphs, Tulip, supply an exciting graph-based navigation interface for multimedia content. The results obtained on feature documentaries are promising.

Content-Based Movie Analysis and Indexing Based on AudioVisual Cues

A content-based movie parsing and indexing approach is presented; it analyzes both audio and visual sources and accounts for their interrelations to extract high-level semantic cues. Specifically, the goal of this work is to extract meaningful movie events and assign them semantic labels for the purpose of content indexing. Three types of key events, namely, 2-speaker dialogs, multiple-speaker dialogs, and hybrid events, are considered. Moreover, speakers present in the detected movie dialogs are further identified based on the audio source parsing. The obtained audio and visual cues are then integrated to index the movie content. Our experiments have shown that an effective integration of the audio and visual sources can lead to a higher level of video content understanding, abstraction and indexing.

MPEG-7 based description schemes for multi-level video content classification

MPEG-7 has emerged as the standard for multimedia data content description for efficiently describing multimedia content. In this context, its primary goal is to provide flexible and effective searching and retrieval of multimedia resources. Most of the earlier work on MPEG-7 description schemes (DSs) and descriptors (Ds) focuses on the description of a single multimedia document, whereas MPEG-7 can be further exploited to support more advances implementations under multimedia database systems. Therefore, it is important to reconsider issues related to high level multimedia modeling and representation, in the light of the MPEG-7 perspective. In this paper, we propose a high level multimedia representation and description scheme based on multi-level video modeling and semantic video classification. The proposed multi-level multimedia representation and DSs are expected to support more effective video content indexing and accessing operations. The presented DSs and Ds are further described by using the XML Schema language, which has been adopted as the basis of the Description Definition Language (DDL) of the MPEG-7 standard.

Video Content Representation, Indexing, and Matching in Video Information Systems

Changing between frames is one of the most obvious information in video data. This frame-by-frame time series data is essential to many application areas. However, how to extract and compare video contents in a video information system is still an important problem to be solved. In this paper, we focus on the problem of design a fast searching method in a video information system to locate video segments that match a content-based query, approximately by time series feature values. The idea is to extract video contents via low-level feature extraction and/or high level semantic retrieval mechanisms according to a specific point of view, then segment video contents into bounding boxes via a box segmentation mechanism by their time series feature values. Video content indexing is constructed by the characteristics of prominent points that accompany bounding boxes. We also propose an efficient and effective video content matching algorithm to find similar sequences. With the help of the video indexing and matching mechanisms, several high level box-to-box and low level point-to-point query types can be requested. The implementation and performance evaluation of our video information prototype system is described.