Video Database Indexing Research Articles

Hybrid Method of Video Shot Segmentation Based on YCbCr Space Color Model

In this digital world, Video analysis is the most important and useful task. Currently, tremendous tasks have been done in video analysis like compressing the videos, video retrieval process and video database indexing, etc. For all these tasks, one common step is segmenting the video shots, which are referred to as Video Shots Segmentation (VSS). Video shots segmentation is used to segment the input videos into a number of frames sequentially where the scene changes occurred, i.e. called shots. In this article, segmenting the video shots follows a hybrid procedure. Here, we have introduced the moments of colors, distance metrics and threshold techniques. All the videos follow the above mentioned steps for segmenting the video shots. But, before that, the input video is converted into a specific color model i.e. YCbCr. Then, apply the color moments to extract the feature vectors of frames, which are differentiated based on the color features of frames. In every two frames of the video, distance metrics methods are applying to compute the similarity and dissimilarity of frames. And the dissimilarity of the frames can be computed by using the threshold technique to get the shots from the video. In this paper, we are using the adaptive threshold technique to segment the videos into various shots. In this step, we will get a true number of shots. By the experimental results, this proposed methodology can be evaluated with the sequence of videos based on the performance or evaluation metrics.

EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos.

Surgical workflow recognition has numerous potential medical applications, such as the automatic indexing of surgical video databases and the optimization of real-time operating room scheduling, among others. As a result, surgical phase recognition has been studied in the context of several kinds of surgeries, such as cataract, neurological, and laparoscopic surgeries. In the literature, two types of features are typically used to perform this task: visual features and tool usage signals. However, the used visual features are mostly handcrafted. Furthermore, the tool usage signals are usually collected via a manual annotation process or by using additional equipment. In this paper, we propose a novel method for phase recognition that uses a convolutional neural network (CNN) to automatically learn features from cholecystectomy videos and that relies uniquely on visual information. In previous studies, it has been shown that the tool usage signals can provide valuable information in performing the phase recognition task. Thus, we present a novel CNN architecture, called EndoNet, that is designed to carry out the phase recognition and tool presence detection tasks in a multi-task manner. To the best of our knowledge, this is the first work proposing to use a CNN for multiple recognition tasks on laparoscopic videos. Experimental comparisons to other methods show that EndoNet yields state-of-the-art results for both tasks.

Rethinking random Hough Forests for video database indexing and pattern search

Hough Forests have demonstrated effective performance in object detection tasks, which has potential to translate to exciting opportunities in pattern search. However, current systems are incompatible with the scalability and performance requirements of an interactive visual search. In this paper, we pursue this potential by rethinking the method of Hough Forests training to devise a system that is synonymous with a database search index that can yield pattern search results in near real time. The system performs well on simple pattern detection, demonstrating the concept is sound. However, detection of patterns in complex and crowded street-scenes is more challenging. Some success is demonstrated in such videos, and we describe future work that will address some of the key questions arising from our work to date.

Video event description in scene context

Video event description is an important research topic in video analysis with a vast amount of applications, such as visual surveillance, video retrieval, video annotation, video database indexing, and interactive system. In this paper, we present a framework for automated video event description, which features fused with the context knowledge to provide accurate and reliable event description. The processing framework is designed to describe the event and recognize objects activities composed of four components: object detection, classification, tracking, and semantic event description. Our contribution is to integrate the contextual cues into these components to facilitate the semantic video event description. Furthermore, in the tracking part, a novel adaptive shape kernel based mean shift tracking algorithm is proposed to improve object tracking performance under object deformation and background clutter. In the experiments, we show attractive experimental results, highlighting the system efficiency and tracking capability by using our video event description system on a real-world video for video event understanding application.

An Improved Keyframe Extraction Method Based on HSV Colour Space

Video segmentation and keyframe extraction are the basis of Content-based Video Retrieval (CBVR), in which keyframe selection is at the very core of CBVR. At shot level, key-frame extraction provides sufficient indexing and browsing of large video databases. In this paper, we proposed two improved approaches of key-frame extraction for video summarization. In our first synthesis method based on histogram-based method and pixel-based method, videos were firstly segmented into shots according to video content, by our improved histogram-based method, with the use of histogram intersection and nonuniform partitioning and weighting. Then, the obtained results are secondly detected to optimize the results. On the other hand, we realized an improved clustering algorithm for video shot segmentation, in consideration of video characteristics. Within each shot, key-frames were determined with the calculation of image entropy of every frame in HSV colour space. Our simulation results in section 4 prove that extracted key frames with our method are compact and faithful to the original video. Moreover, according to the types of test videos, different methods for shot segmentation are highly recommended.

Understanding Video Events: A Survey of Methods for Automatic Interpretation of Semantic Occurrences in Video

Understanding video events, i.e., the translation of low-level content in video sequences into high-level semantic concepts, is a research topic that has received much interest in recent years. Important applications of this paper include smart surveillance systems, semantic video database indexing, and interactive systems. This technology can be applied to several video domains including airport terminal, parking lot, traffic, subway stations, aerial surveillance, and sign language data. In this paper, we identify the two main components of the event understanding process: abstraction and event modeling. Abstraction is the process of molding the data into informative units to be used as input to the event model. Due to space restrictions, we will limit the discussion on the topic of abstraction. See the study by Lavee et al. (Understanding video events: A survey of methods for automatic interpretation of semantic occurrences in video, Technion-Israel Inst. Technol., Haifa, Israel, Tech. Rep. CIS-2009-06, 2009) for a more complete discussion. Event modeling is devoted to describing events of interest formally and enabling recognition of these events as they occur in the video sequence. Event modeling can be further decomposed in the categories of pattern-recognition methods, state event models, and semantic event models. In this survey, we discuss this proposed taxonomy of the literature, offer a unifying terminology, and discuss popular event modeling formalisms (e.g., hidden Markov model) and their use in video event understanding using extensive examples from the literature. Finally, we consider the application domain of video event understanding in light of the proposed taxonomy, and propose future directions for research in this field.

Constructing Distributed Hippocratic Video Databases for Privacy-Preserving Online Patient Training and Counseling

Digital video now plays an important role in supporting more profitable online patient training and counseling, and integration of patient training videos from multiple competitive organizations in the health care network will result in better offerings for patients. However, privacy concerns often prevent multiple competitive organizations from sharing and integrating their patient training videos. In addition, patients with infectious or chronic diseases may not want the online patient training organizations to identify who they are or even which video clips they are interested in. Thus, there is an urgent need to develop more effective techniques to protect both video content privacy and access privacy. In this paper, we have developed a new approach to construct a distributed Hippocratic video database system for supporting more profitable online patient training and counseling. First, a new database modeling approach is developed to support concept-oriented video database organization and assign a degree of privacy of the video content for each database level automatically. Second, a new algorithm is developed to protect the video content privacy at the level of individual video clip by filtering out the privacy-sensitive human objects automatically. In order to integrate the patient training videos from multiple competitive organizations for constructing a centralized video database indexing structure, a privacy-preserving video sharing scheme is developed to support privacy-preserving distributed classifier training and prevent the statistical inferences from the videos that are shared for cross-validation of video classifiers. Our experiments on large-scale video databases have also provided very convincing results.

An Introduction to the Special Issue on Event Analysis in Videos

I NTEREST from industry and academia has increased dramatically over recent years in the challenging area of event analysis and recognition from various video sources including sports, surveillance, user-generated video, etc. Video event analysis and recognition is a critical task in many applications such as detection of sporting highlights, incident detection in surveillance video, indexing, retrieval and summarization of video databases, and human-computer interaction. This special issue aims to capture the latest advances by the research community working in the area of video event analysis. The call for papers was enthusiastically greeted by the research community and we received over seventy submissions. The special issue presents 16 articles which provide fundamental contributions in a wide range of topics in video event analysis: 1) human action and activity recognition; 2) motion trajectory analysis; 3) video content analysis and pattern mining; 4) audio-visual multi-modal analysis; and 5) video analysis applications. An overview of the organization and a brief summary of the articles selected for publication in the special issue are provided below.

Application of Laplacian Mixture Model to Image and Video Retrieval

In this paper, we study the peaky nature of wavelet coefficient distributions. The study shows that the wavelet coefficients cannot be effectively modeled by a single distribution. We then propose a new modeling scheme based on a Laplacian mixture model and apply it to the indexing and retrieval of image and video databases. In this work, the parameters of the model are first used to represent texture information in image retrieval. Then we explore its application to video retrieval. Traditionally, visual information is used for video indexing and retrieval. However, in some cases audio information is more helpful for finding clues to the video events. The proposed feature extraction scheme is based on the fundamental property of the wavelet transform. Therefore, it can also be adopted to analyze the audio contents of the video data. The experimental evaluation indicates the high discriminatory power of the proposed feature set. The dimension of the extracted feature vector is low, which is important for the retrieval efficiency of the system in terms of response time. User feedback is used to enhance the retrieval performance by modifying the system parameters according to the users' behavior. A nonlinear approach for defining the similarity between the two images is also explored in this work.

Effective Detection of Various Wipe Transitions

Automatic detection of wipes and their frame ranges is important for the purpose of reliable video parsing and video database indexing. Wipes are difficult to detect because of the complexity and variety of the transition effects. Many of the existing wipe detection algorithms could detect only a few wipe effects. The false/miss detection problem caused by motion is also very serious. In this paper, we propose a novel wipe detection algorithm that can detect most wipe effects with accurate frame ranges. We carefully model a wipe based on its nature and then use the model to filter out possible confusion caused by motion or other transition effects. More precisely, properties of independence and completeness are proposed to characterize an ideal wipe; frame ranges of potential wipes are located by finding sequences which are a close approximation to an ideal wipe. Bayes rule is applied to each potential wipe to statistically estimate an adaptive threshold for the purpose of wipe verification. Experiment results on videos with different genres show that the proposed methodology can be used to detect various wipe effects effectively

Concept-Oriented Indexing of Video Databases: Toward Semantic Sensitive Retrieval and Browsing

Digital video now plays an important role in medical education, health care, telemedicine and other medical applications. Several content-based video retrieval (CBVR) systems have been proposed in the past, but they still suffer from the following challenging problems: semantic gap, semantic video concept modeling, semantic video classification, and concept-oriented video database indexing and access. In this paper, we propose a novel framework to make some advances toward the final goal to solve these problems. Specifically, the framework includes: 1) a semantic-sensitive video content representation framework by using principal video shots to enhance the quality of features; 2) semantic video concept interpretation by using flexible mixture model to bridge the semantic gap; 3) a novel semantic video-classifier training framework by integrating feature selection, parameter estimation, and model selection seamlessly in a single algorithm; and 4) a concept-oriented video database organization technique through a certain domain-dependent concept hierarchy to enable semantic-sensitive video retrieval and browsing.

ClassView : Hierarchical Video Shot Classification, Indexing, and Accessing

Recent advances in digital video compression and networks have made video more accessible than ever. However, the existing content-based video retrieval systems still suffer from the following problems. 1) Semantics-sensitive video classification problem because of the semantic gap between low-level visual features and high-level semantic visual concepts; 2) Integrated video access problem because of the lack of efficient video database indexing, automatic video annotation, and concept-oriented summary organization techniques. In this paper, we have proposed a novel framework, called ClassView, to make some advances toward more efficient video database indexing and access. 1) A hierarchical semantics-sensitive video classifier is proposed to shorten the semantic gap. The hierarchical tree structure of the semantics-sensitive video classifier is derived from the domain-dependent concept hierarchy of video contents in a database. Relevance analysis is used for selecting the discriminating visual features with suitable importances. The Expectation-Maximization (EM) algorithm is also used to determine the classification rule for each visual concept node in the classifier. 2) A hierarchical video database indexing and summary presentation technique is proposed to support more effective video access over a large-scale database. The hierarchical tree structure of our video database indexing scheme is determined by the domain-dependent concept hierarchy which is also used for video classification. The presentation of visual summary is also integrated with the inherent hierarchical video database indexing tree structure. Integrating video access with efficient database indexing tree structure has provided great opportunity for supporting more powerful video search engines.

A hierarchical access control model for video database systems

Content-based video database access control is becoming very important, but it depends on the progresses of the following related research issues: (a) efficient video analysis for supporting semantic visual concept representation; (b) effective video database indexing structure; (c) the development of suitable video database models; and (d) the development of access control models tailored to the characteristics of video data. In this paper, we propose a novel approach to support multilevel access control in video databases. Our access control technique combines a video database indexing mechanism with a hierarchical organization of visual concepts (i.e., video database indexing units), so that different classes of users can access different video elements or even the same video element with different quality levels according to their permissions. These video elements, which, in our access control mechanism, are used for specifying the authorization objects, can be a semantic cluster, a subcluster, a video scene, a video shot, a video frame, or even a salient object (i.e., region of interest). In the paper, we first introduce our techniques for obtaining these multilevel video access units. We also propose a hierarchical video database indexing technique to support our multilevel video access control mechanism. Then, we present an innovative access control model which is able to support flexible multilevel access control to video elements. Moreover, the application of our multilevel video database modeling, representation, and indexing for MPEG-7 is discussed.

Model-Based Video Classification toward Hierarchical Representation, Indexing and Access

In this paper, we develop a content-based video classification approach to support semantic categorization, high-dimensional indexing and multi-level access. Our contributions are in four points: (a) We first present a hierarchical video database model that captures the structures and semantics of video contents in databases. One advantage of this hierarchical video database model is that it can provide a framework for automatic mapping from high-level concepts to low-level representative features. (b) We second propose a set of useful techniques for exploiting the basic units (e.g., shots or objects) to access the videos in database. (c) We third suggest a learning-based semantic classification technique to exploit the structures and semantics of video contents in database. (d) We further develop a cluster-based indexing structure to both speed-up query-by-example and organize databases for supporting more effective browsing. The applications of this proposed multi-level video database representation and indexing structures for MPEG-7 are also discussed.

An Illumination Effect Descriptor for Video Sequences

In the context of indexing of video data bases, the future standard MPEG-7 will provide descriptors for motion, shape, texture, and color to identify video scenes. In this paper, an illumination effect descriptor is presented that has been proposed to MPEG-7. Effects caused by scene illumination can be a useful feature for video indexing. The proposed descriptor addresses temporal changes of object shading, of cast shadows, and of the global illumination intensity by one scalar. It is based on temporal changes of image luminance along motion trajectories. Further, an automatic, noise adaptive method for extraction of the illumination effect descriptor from a video sequence is presented. Performance analysis and sample retrieval experiments show that extracted descriptor values are sufficiently precise and sensible to distinguish video sequences with different scene illumination.

Hierarchical video indexing and retrieval for subband-coded video

We present a multiresolution approach for video indexing and feature matching of subband-coded video databases. Four different scene-change detectors were tested; scene-change detection is applied only on the lowest subband for computational efficiency. Two kinds of scene changes, abrupt and smoothly accumulated, mark the beginning of new scene segments. The index for each scene segment is the pair of the histograms of two representative frames, the first and the last frame of the scene. Using the approach of query by example, the index-matching algorithm takes a multiresolution approach by hierarchically comparing histograms at different resolutions. The search algorithm for the match between example query and its target scene segment starts from the coarsest resolution and moves to the next finer resolution until the finest resolution is reached. Experimental results are presented, and the proposed indexing technique appears to be promising for its speed and its inherent hierarchical search procedure.

Apparent 3-D camera velocity-extraction and applications

In this paper, we describe a robust method for the extraction of the apparent 3-D camera velocity and 3-D scene structure information. Our method performs the extraction of the apparent 3-D camera velocity in a fully automated way without any knowledge about 3-D scene content information as used in current methods. This has the advantage that it can be used to fully automate the generation of natural-looking virtual/augmented environments, as well as in video-database browsing. First, we describe our method for the robust extraction of 3-D parameters. This method is a combination of the eight-point method in structure-from-motion with a statistical technique to automatically select feature points in the image, irrespective of 3-D content information. Second, we discuss two applications which use the results of the 3-D parameter extraction. The first application is the generation of sprite layers using 3-D camera velocity information to represent an eight-parameter perspective image-to-sprite mapping plus 3-D scene depth information for the sprite layering. The second application is the use of 3-D camera velocity for the indexing of large video databases according to a set of seven independent types of camera motion. In this application, we discuss a formal video description structure-the camera-motion descriptor-which was successfully included in the working draft of the MPEG-7 video standard.

Wipe scene change detector for use with video compression algorithms and MPEG-7

The use of wipes in video production to smooth the transition between two consecutive scenes complicates subsequent video compression, video editing, and video database indexing. It is important to detect the wipe region in order to improve the quality of the compressed video or to allow automatic parsing of the video for the purpose of editing and database indexing. A model is developed for the wipe region and is used to derive the statistical characteristics of the frames in the wipe region. A wipe detector that exploits the linear change in the means and the variances of the frames in the wipe region is developed. The developed wipe detector is implemented and tested with video sequences containing wipes of different types and lengths. Simulation results indicate that wipe regions are well represented by the developed wipe model, and that the developed wipe detector is robust to wipe type and length.