Video Retrieval Method Research Articles

English

In the last few years, e-lecturing has become more and more popular because video provide rich source of information. The amount of lecture video data on the internet is growing exponentially. Thus, a more efficient method for video retrieval in internet or within large lecture video archives is urgently needed. This paper presents a text based video retrieval and Video search system using Optimal Character Recognition (OCR). First, we convert the video into key-frames and extract the Text using OCR. Following step is to produce a summary presenting key points of the video, by making use of meradata of text and audio extracted from the Video. This summary will then be used for grouping and Indexing of videos. In this paper, we discuss various lecture video segmentation approaches. As there is strong needs for segmenting lecture videos into topic units in order to organize the videos for browsing and to provide search capability.

Reducing Semantic Gap in Video Retrieval with Fusion: A Survey

Abstract: Multimedia provides a rich content of information and huge amount of data's are available in the field of video retrieval. Now day's enormous videos are available on web and online to accessible from internet or retrieve videos from smart phones, digital cellular assistants. There is the drastic growth in the amount of multimedia field of improving data storage, acquisition and communication technologies, which are all supported by major improvements in processing of video and audio. Researches focused on more efforts in video retrieval that contain certain visual information rather than image of their interest. Such a search is facilitated by Content Based Video Retrieval (CBVR) methods. Specifically segmentation of video is the most prominent step as the retrieved results are based on the segmentation boundaries. The shot boundary detection can be performed using various different techniques like Motion/hybrid DCT, edge tracking, histogram, HSV Model, Motion vector and Block matching methods. This paper mainly presents a study of different methods/algorithm that has been proposed in literature for video retrieval to reduce the semantic gap between low and high level features. Semantic gap between these two feature level is improving by its efficiency with the help of advanced algorithms and techniques using machine learning with fusions.

REVIEW ON CONTENT BASED VIDEO LECTURE RETRIEVAL

Recent advances in multimedia technologies allow the capture and storage of video data with relatively inexpensive computers. Furthermore, the new possibilities offered by the information highways have made a large amount of video data publicly available. However, without appropriate search techniques all these data are hardly usable. Users are not satisfied with the video retrieval systems that provide analogue VCR functionality. For example, a user analyses a soccer video will ask for specific events such as goals. Content-based search and retrieval of video data becomes a challenging and important problem. Therefore, the need for tools that can be manipulate the video content in the same way as traditional databases manage numeric and textual data is significant. Therefore, a more efficient method for video retrieval in WWW or within large lecture video archives is urgently needed. This project presents an approach for automated video indexing and video search in large lecture video archives. First of all, we apply automatic video segmentation and key-frame detection to offer a visual guideline for the video content navigation. Subsequently, we extract textual metadata by applying video Optical Character Recognition (OCR) technology on key-frames and Automatic Speech Recognition on lecture audio tracks.

BM25 With Exponential IDF for Instance Search

This paper deals with a novel concept of an exponential IDF in the BM25 formulation and compares the search accuracy with that of the BM25 with the original IDF in a content-based video retrieval (CBVR) task. Our video retrieval method is based on a bag of keypoints (local visual features) and the exponential IDF estimates the keypoint importance weights more accurately than the original IDF. The exponential IDF is capable of suppressing the keypoints from frequently occurring background objects in videos, and we found that this effect is essential for achieving improved search accuracy in CBVR. Our proposed method is especially designed to tackle instance video search, one of the CBVR tasks, and we demonstrate its effectiveness in significantly enhancing the instance search accuracy using the TRECVID2012 video retrieval dataset.

Content Based Lecture Video Retrieval Using Speech and Video Text Information

In the last decade e-lecturing has become more and more popular. The amount of lecture video data on the World Wide Web (WWW) is growing rapidly. Therefore, a more efficient method for video retrieval in WWW or within large lecture video archives is urgently needed. This paper presents an approach for automated video indexing and video search in large lecture video archives. First of all, we apply automatic video segmentation and key-frame detection to offer a visual guideline for the video content navigation. Subsequently, we extract textual metadata by applying video Optical Character Recognition (OCR) technology on key-frames and Automatic Speech Recognition (ASR) on lecture audio tracks. The OCR and ASR transcript as well as detected slide text line types are adopted for keyword extraction, by which both video- and segment-level keywords are extracted for content-based video browsing and search. The performance and the effectiveness of proposed indexing functionalities is proven by evaluation.

키 프레임 특징들에 적응적 가중치 부여를 이용한 검색 성능 개선

Video retrieval and indexing are performed by comparing feature similarities between key frames in shot after detecting a scene change and extracting key frames from the shot. Typical image features such as color, shape, and texture are used in content-based video and image retrieval. Many approaches for integrating these features have been studied. However, the issue of these approaches is how to appropriately assign weighting of key frame features at query time. Therefore, we propose a new video retrieval method using adaptively weighted image features. We performed computer simulations in test databases which consist of various kinds of key frames. The experimental results show that the proposed method has better performance than previous works in respect to several performance evaluations such as precision vs. recall, retrieval efficiency, and ranking measure.

Surveillance Video Retrieval Based on Moving Objects Detection

In this article, we proposed a surveillance video retrieval method based on detection of moving objects and researched on key problems in surveillance video, such as moving objects detection, video segmentation, selection and description of key frames, and measurement of feature similarities. We detected moving objects via movement detection algorithm based on codebook, and realized automatic segmentation of surveillance video via calculating present Frame Movement Amount based on the foreground and background information in movement detection. After the above process, we proposed a video retrieval method based on the combination of SIFT and color feature.

A sample-based hierarchical adaptive K-means clustering method for large-scale video retrieval

Finding useful patterns in large datasets has attracted considerable interest recently, and one of the most widely studied problems in this area is the identification of clusters in a multi-dimensional dataset. This paper introduces a sample-based hierarchical adaptive K-means (SHAKM) clustering algorithm for large-scale video retrieval. To handle large databases efficiently, SHAKM employs a multilevel random sampling strategy. Furthermore, SHAKM utilises the adaptive K-means clustering algorithm to determine the correct number of clusters and to construct an unbalanced cluster tree. Furthermore, SHAKM uses the fast labelling scheme to assign each pattern in the dataset to the closest cluster. To evaluate the proposed method, several datasets are used to illustrate its effectiveness. The results show that SHAKM is fast and effective on very large datasets. Furthermore, the results demonstrate that the proposed method can be used efficiently and successfully for a project on content-based video copy detection.

Entropy Supported Video Indexing for Content based Video Retrieval

The increased in availability and usage of on-line digital video has created a need of automated video content analysis techniques, including indexing and retrieving. Automation of indexing significantly reduces the processing cost while by minimizing tedious work. Traditional video retrieval methods based on video metadata, fail to meet technical challenges due to large and rapid growth of multimedia data, demanding effective retrieval systems. One of the most popular solutions for indexing is extracting the features of video key frames for developing a Content Based Video Retrieval (CBVR) system. CBVR works more effectively as these deals with content of video rather than video metadata. Various features like color, texture, shape can be integrated and used for video indexing and retrieval. Implemented CBVR system is experimented based on integration of texture, color and edge features for video retrieval. Entropy is a texture descriptor used for key frame extraction and video indexing. However entropy, color (RGB) and edge detection algorithms are used for video retrieval. These features are combined in various ways like entropyedge, entropycolor for result refinement. Dataset is created with the videos from different domains like e-learning, nature, construction etc. By the combination of these features in different ways, we achieved comparative results. Obtained result shows that combining of two or many features gives better retrieval.

Content-Based Analysis Improves Audiovisual Archive Retrieval

Content-based video retrieval is maturing to the point where it can be used in real-world retrieval practices. One such practice is the audiovisual archive, whose users increasingly require fine-grained access to broadcast television content. In this paper, we take into account the information needs and retrieval data already present in the audiovisual archive, and demonstrate that retrieval performance can be significantly improved when content-based methods are applied to search. To the best of our knowledge, this is the first time that the practice of an audiovisual archive has been taken into account for quantitative retrieval evaluation. To arrive at our main result, we propose an evaluation methodology tailored to the specific needs and circumstances of the audiovisual archive, which are typically missed by existing evaluation initiatives. We utilize logged searches, content purchases, session information, and simulators to create realistic query sets and relevance judgments. To reflect the retrieval practice of both the archive and the video retrieval community as closely as possible, our experiments with three video search engines incorporate archive-created catalog entries as well as state-of-the-art multimedia content analysis results. A detailed query-level analysis indicates that individual content-based retrieval methods such as transcript-based retrieval and concept-based retrieval yield approximately equal performance gains. When combined, we find that content-based video retrieval incorporated into the archive's practice results in significant performance increases for shot retrieval and for retrieving entire television programs. The time has come for audiovisual archives to start accommodating content-based video retrieval methods into their daily practice.

Intelligent video processing using data mining techniques

Due to the rapidly increasing video data on the Web, much research effort has been devoted to develop video retrieval methods which can efficiently retrieve videos of interest. Considering the limited man-power, it is much expected to develop retrieval methods which use features automatically extracted from videos. However, since features only represent physical contents (e.g. color, edge, motion, etc.), retrieval methods require knowledge of how to use/integrate features for retrieving videos relevant to queries. To obtain such knowledge, this thesis concentrates on 'video data mining' where videos are analyzed using data mining techniques which extract previously unknown, interesting patterns in underlying data. Thereby, patterns for retrieving relevant shots to queries are extracted as explicit knowledge.

Tu1405 Toward a Structured Training System for Probe-Based Confocal Laser Endomicroscopy (pCLE) on Barrett's Esophagus (BE): A Video Retrieval Approach to Estimate Diagnosis Difficulty

pCLE (Cellvizio, Mauna Kea Technologies) allows the endoscopist to image the epithelial surface in vivo, at microscopic level and in real-time (12 frames per second) during an ongoing endoscopy. Early diagnosis of epithelial cancers with pCLE may be perceived as a challenging task for many new endoscopists. There is a crucial need to provide objective methods to diagnose neoplasia, estimate confidence levels, and to shorten the learning curve. Our long-term objective is to develop a modular training system for pCLE diagnosis, by adapting the difficulty level according to the endoscopist's expertise. This study aims at providing an automated estimation of the diagnosis difficulty. As the understanding of pCLE video diagnosis is driven by perceived visual similarity, we propose a content-based video retrieval approach toward this goal. Our database contains annotated pCLE videos of BE that were provided by the multicentric study NCT00795184. It includes 76 patients and 123 videos (62 benign, 61 neoplastic) split into 862 stable video sub-sequences. 20 of these videos (9 benign, 11 neoplastic) were graded offline by 21 endoscopists, including 9 pCLE experts and 12 non-experts, who individually established a blinded pCLE diagnosis for each lesion. A single expert GI pathologist reviewed all the biopsies acquired on the imaging spots and provided a reference diagnosis. The percentage of false pCLE diagnosis established on a video among the endoscopists is our ”ground truth” for the diagnosis difficulty of the video. We first applied to the video database a video retrieval method that we developed especially for this task. We then used the retrieval results to extract a relevant difficulty criterion that measures contextual discrepancies between the video query and its most visually similar videos. Our video retrieval method, objectively evaluated using k-nearest neighbor classification, outperforms several state-of-the-art methods on the BE database (acc. 85.4%, sens. 90.2%, spec. 80.7%). Our estimated diagnosis difficulty has a correlation of 0.78 (p-value < 0.0002) with the ground truth difficulty measured for all the endoscopists, 0.63 (p-value < 0.003) with those measured for the experts only, and 0.80 (p-value < 0.0001) with those measured for the non-experts only. Our experiments demonstrate that there is a noticeable relationship between our retrieval-based difficulty estimation and the difficulty experienced by the endoscopists. The complete video database with estimated difficulty could thus be used to identify lesions for which an optical diagnosis will be difficult, and to develop a training simulator that features difficulty level selection. Finally, a clinical validation will be required to assess whether such a structured training system will eventually help shorten the pCLE learning curve.

Extracting representative motion flows for effective video retrieval

In this paper, we propose a novel motion-based video retrieval approach to find desired videos from video databases through trajectory matching. The main component of our approach is to extract representative motion features from the video, which could be broken down to the following three steps. First, we extract the motion vectors from each frame of videos and utilize Harris corner points to compensate the effect of the camera motion. Second, we find interesting motion flows from frames using sliding window mechanism and a clustering algorithm. Third, we merge the generated motion flows and select representative ones to capture the motion features of videos. Furthermore, we design a symbolic based trajectory matching method for effective video retrieval. The experimental results show that our algorithm is capable to effectively extract motion flows with high accuracy and outperforms existing approaches for video retrieval.

A smart atlas for endomicroscopy using automated video retrieval

To support the challenging task of early epithelial cancer diagnosis from in vivo endomicroscopy, we propose a content-based video retrieval method that uses an expert-annotated database. Motivated by the recent successes of non-medical content-based image retrieval, we first adjust the standard Bag-of-Visual-Words method to handle single endomicroscopic images. A local dense multi-scale description is proposed to keep the proper level of invariance, in our case to translations, in-plane rotations and affine transformations of the intensities. Since single images may have an insufficient field-of-view to make a robust diagnosis, we introduce a video-mosaicing technique that provides large field-of-view mosaic images. To remove outliers, retrieval is followed by a geometrical approach that captures a statistical description of the spatial relationships between the local features. Building on image retrieval, we then focus on efficient video retrieval. Our approach avoids the time-consuming parts of the video-mosaicing by relying on coarse registration results only to account for spatial overlap between images taken at different times. To evaluate the retrieval, we perform a simple nearest neighbors classification with leave-one-patient-out cross-validation. From the results of binary and multi-class classification, we show that our approach outperforms, with statistical significance, several state-of-the art methods. We obtain a binary classification accuracy of 94.2%, which is quite close to clinical expectations.

Episode-Constrained Cross-Validation in Video Concept Retrieval

Whereas video tells a narrative by a composition of shots, current video retrieval methods focus mainly on single shots. In retrieval performance estimation, similar shots in a narrative may result in performance overestimation. We propose an episode-based version of cross-validation leading up to 14% classification improvement over shot-based cross-validation.

A Novel Video Retrieval Method Based on Web Community Extraction Using Features of Video Materials

A novel video retrieval method based on Web community extraction using audio and visual features and textual features of video materials is proposed in this paper. In this proposed method, canonical correlation analysis is applied to these three features calculated from video materials and their Web pages, and transformation of each feature into the same variate space is possible. The transformed variates are based on the relationships between visual, audio and textual features of video materials, and the similarity between video materials in the same feature space for each feature can be calculated. Next, the proposed method introduces the obtained similarities of video materials into the link relationship between their Web pages. Furthermore, by performing link analysis of the obtained weighted link relationship, this approach extracts Web communities including similar topics and provides the degree of attribution of video materials in each Web community for each feature. Therefore, by calculating similarities of the degrees of attribution between the Web communities extracted from the three kinds of features, the desired ones are automatically selected. Consequently, by monitoring the degrees of attribution of the obtained Web communities, the proposed method can perform effective video retrieval. Some experimental results obtained by applying the proposed method to video materials obtained from actual Web pages are shown to verify the effectiveness of the proposed method.

Video retrieval based on object discovery

State-of-the-art video retrieval methods use global image statistics to provide low level descriptors or use object recognizers to provide high level features. Using global image statistics can be hindered by lack of explicitly characterizing the object of interest hence prone to retrieving irrelevant results, while using object recognizers can suffer from having to train a large number of object recognizers for different types of objects. We present a novel framework for content-based video retrieval. We use an unsupervised learning method to automatically discover and locate the object of interest in a video clip. This unsupervised learning algorithm alleviates the need for training a large number of object recognizers. Regional image characteristics are extracted from the object of interest to form a set of descriptors for each video. A novel ensemble-based matching algorithm compares the similarity between two videos based on the set of descriptors each video contains. Videos containing large pose, size, and lighting variations are used to validate our approach.

Efficient video retrieval using index structure

AbstractVideo retrieval remains a challenging problem since most of traditional query algorithms are ineffectual and time‐consuming. In this article, we proposed a new video retrieval method, which segments the video stream by visual similarity between neighboring frames, and adopt the high‐dimensional index structure to organize segments. Furthermore, a new similarity measure is brought forward to improve the query accuracy by synthetically taking into account the visual similarity and temporal order among video segments. Based on the similarity measure, we propose a novel video clip retrieval algorithm which achieves high query efficiency by using restricted sliding window to construct candidate video clips. Experimental results show that the proposed video retrieval method is efficient and effective. © 2008 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 18, 113–123, 2008

Adding Semantics to Detectors for Video Retrieval

In this paper, we propose an automatic video retrieval method based on high-level concept detectors. Research in video analysis has reached the point where over 100 concept detectors can be learned in a generic fashion, albeit with mixed performance. Such a set of detectors is very small still compared to ontologies aiming to capture the full vocabulary a user has. We aim to throw a bridge between the two fields by building a multimedia thesaurus, i.e., a set of machine learned concept detectors that is enriched with semantic descriptions and semantic structure obtained from WordNet. Given a multimodal user query, we identify three strategies to select a relevant detector from this thesaurus, namely: text matching, ontology querying, and semantic visual querying. We evaluate the methods against the automatic search task of the TRECVID 2005 video retrieval benchmark, using a news video archive of 85 h in combination with a thesaurus of 363 machine learned concept detectors. We assess the influence of thesaurus size on video search performance, evaluate and compare the multimodal selection strategies for concept detectors, and finally discuss their combined potential using oracle fusion. The set of queries in the TRECVID 2005 corpus is too small for us to be definite in our conclusions, but the results suggest promising new lines of research.

Eminent scene change detection algorithm and new indexing method for video retrieval on MPEG sequences

We propose an eminent scene change detection (SCD) algorithm, and also present a new video-indexing method for fast content- based browsing and retrieval in video databases. The first step in our approach is to extract representative frames by using the proposed SCD algorithm from the Motion Picture Experts Group (MPEG) video sequence. Thereafter, we perform the video indexing by applying the rosette pattern (RP) to the extracted representative frames, and retrieve them. A principal cause of these processing procedures is that the performance of video indexing and retrieval system depends on the SCD. And how accurately SCD performs as well as a preprocessing are some of the considerations that we must take into account. Therefore, we classify every possible scene change that may happen within the video into three types: abrupt, gradual, and flashlight. And we detect every kind of scene change. Our SCD algorithm is better than the conventional ones in terms of SCD performance and we demonstrate its superiority by experiments. We also extract representative frames that represent series of detected scenes by the proposed accumulative histogram intersection measure (AHIM). Finally, we apply the RP to extracted representative frames for indexing, we can remarkably reduce the number of pixels (NOP) required to index and excellently retrieve the video scene.