Interactive Video Retrieval Research Articles

Influences of users' familiarity with visual search topics on interactive video digital libraries

AbstractUser‐centered analysis can benefit the development of interactive video digital libraries. Findings from this study support the idea that having additional understanding about the intended users of video digital libraries can help researchers match system designs with the envisioned use of prototype systems. This study examines one user‐centered factor specifically, familiarity with visual search topics, to explore if and how this may be associated with other factors within an interactive video retrieval context. Twenty‐eight users from the field of science education were recruited to complete six visual search topics using a prototype system to retrieve video clips from a collection of NASA Science Education Programs. Analysis revealed that topic familiarity was associated with other factors that were examined throughout this study, including user‐assessed and experimenter‐assessed topic completion ratios, opinions of the prototype system, and interaction behaviors. Such results can have a variety of implications for developing video digital libraries, especially those designed to support queries and interactions of knowledgeable users from a defined domain.

Towards hierarchical context: unfolding visual community potential for interactive video retrieval

Community structure as an interesting property of network has attracted wide attention from many research fields. In this paper, we exploit the visual community structure in visual-temporal correlation network and utilize it to improve interactive video retrieval. Firstly, we propose a hierarchical community-based feedback algorithm. By re-ranking the video shots through diffusion processes respectively on the inter-community and intra-community level, the feedback algorithm can make full use of the limited user feedback. Furthermore, since it avoids entire graph computation, the feedback algorithm can make quick responses to user feedback, which is particularly important for the large video collections. Secondly, we propose a community-based visualization interface called VideoMap. By organizing the video shots following the community structure, the VideoMap presents a comprehensive and informative view of the whole dataset to facilitate users' access. Moreover, the VideoMap can help users to quickly locate the potential relevant regions and make active annotation according to the distribution of labeled samples on the VideoMap. Experiments on TRECVID 2009 search dataset demonstrate the efficiency of the feedback algorithm and the effectiveness of the visualization interface.

Browsing Video Along Multiple Threads

This paper describes a novel method for browsing a large video collection. It links various forms of related video fragments together as threads. These threads are based on query results, the timeline as well as visual and semantic similarity. We design two interfaces which use threads as the basis for browsing. One interface shows a minimal set of threads, and the other as many as fit on the screen. To evaluate both interfaces we perform a regular user study, a study based on user simulation, and we participated in the interactive video retrieval task of the TRECVID benchmark. The results indicate that the use of threads in interactive video retrieval is beneficial. Furthermore, we found that in general the query result and the timeline are the most important threads, but having several additional threads improves the performance as it encourages people to explore new dimensions.

A Content-Driven System Architecture for Tackling Automatic Cataloging of Animated Movie Databases

This article discusses content-based access to video information in large video databases and particularly, to retrieve animated movies. The authors examine temporal segmentation, and propose cut, fade and dissolve detection methods adapted to the constraints of this domain. Further, the authors discuss a fuzzy linguistic approach for deriving automatic symbolic/semantic content annotation in terms of color techniques and action content. The proposed content descriptions are then used with several data mining techniques SVM, k-means to automatically retrieve the animation genre and to classify animated movies according to some color techniques. The authors integrate all the previous techniques to constitute a prototype client-server architecture for a 3D virtual environment for interactive video retrieval.

Analyzing user interaction with the ViewFinder video retrieval system

AbstractThis study investigates interactive video retrieval. The basis for this study is that user‐ and search task‐centric research in video information retrieval can assist efforts for developing effective user interfaces and help complement the existing corpus of video retrieval research by providing evidence for the benefits of evaluating systems using such an approach. Accordingly, the results were collected and analyzed from the perspective of certain users and search tasks (i.e., information needs). The methodology of this study employed specially designed interactive search experiments to examine a number of different factors in a video retrieval context, including those that correspond to search tasks of a particular domain, interface features and functions, system effectiveness, and user interactions. The results indicated that the use and effectiveness of certain interface features and functions were dependent on the type of search task, while others were more consistent across the full experiment. Also included is a review of prior research pertaining to visual search tasks, systems development, and user interaction. ViewFinder, the prototype system used to carry out the interactive search experiments of this study, is fully described.

Simulated evaluation of faceted browsing based on feature selection

In this paper we explore the limitations of facet based browsing which uses sub-needs of an information need for querying and organising the search process in video retrieval. The underlying assumption of this approach is that the search effectiveness will be enhanced if such an approach is employed for interactive video retrieval using textual and visual features. We explore the performance bounds of a faceted system by carrying out a simulated user evaluation on TRECVid data sets, and also on the logs of a prior user experiment with the system. We first present a methodology to reduce the dimensionality of features by selecting the most important ones. Then, we discuss the simulated evaluation strategies employed in our evaluation and the effect on the use of both textual and visual features. Facets created by users are simulated by clustering video shots using textual and visual features. The experimental results of our study demonstrate that the faceted browser can potentially improve the search effectiveness.

A New Interactive Video Retrieval Framework Using Semantic Matching

摘要: 近年来基于内容的视频检索技术受到人们越来越多的关注. 本文提出了一套基于语义匹配的交互式视频检索框架, 其贡献主要为以下三方面: 1)定义新型的视频高层特征---语义直方图用以描述视频的高层语义信息; 2)使用主导集聚类算法建立基于非监督学习的检索机制, 用以降低在线计算复杂度和提高检索效率; 3)提出新型的相关反馈机制---基于语义的分支反馈, 该机制采用分支反馈结构和分支更新策略实现检索性能的提升. 实验结果表明了本框架的有效性. 关键词: 语义匹配直方图 / 基于非监督学习的检索机制 / 基于语义的分支反馈

Query representation by structured concept threads with application to interactive video retrieval

In this paper, we provide a new formulation for video queries as structured combination of concept threads, contributing to the general query-by-concept paradigm. Occupying a low-dimensional region in the concept space, concept thread defines a ranked list of video documents ordered by their combined concept predictions. This localized representation incorporates the previous concept based formulation as a special case and extends the restricted AND concept combination logic to a two-level concept inference network. We apply this new formulation to interactive video retrieval and utilize abundant feedback information to mine the latent semantic concept threads for answering complex query semantics. Simulative experiments which are conducted on two years’ TRECVID data sets with two sets of concept lexicons demonstrate the advantage of the proposed formulation. The proposed query formulation offers some 60% improvements over the simple browsing search baseline in nearly real time. It has clear advantages over c-tf-idf and achieves better results over the state-of-the-art online ordinal reranking approach. Meanwhile, it not only alleviates user’s workload significantly but also is robust to user mislabeling errors.

A Learned Lexicon-Driven Paradigm for Interactive Video Retrieval

Effective video retrieval is the result of interplay between interactive query selection, advanced visualization of results, and a goal-oriented human user. Traditional interactive video retrieval approaches emphasize paradigms, such as query-by-keyword and query-by-example, to aid the user in the search for relevant footage. However, recent results in automatic indexing indicate that query-by-concept is becoming a viable resource for interactive retrieval also. We propose in this paper a new video retrieval paradigm. The core of the paradigm is formed by first detecting a large lexicon of semantic concepts. From there, we combine query-by-concept, query-by-example, query-by-keyword, and user interaction into the MediaMill semantic video search engine. To measure the impact of increasing lexicon size on interactive video retrieval performance, we performed two experiments against the 2004 and 2005 NIST TRECVID benchmarks, using lexicons containing 32 and 101 concepts, respectively. The results suggest that from all factors that play a role in interactive retrieval, a large lexicon of semantic concepts matters most. Indeed, by exploiting large lexicons, many video search questions are solvable without using query-by-keyword and query-by-example. In addition, we show that the lexicon-driven search engine outperforms all state-of-the-art video retrieval systems in both TRECVID 2004 and 2005

Assessing user behaviour in news video retrieval

The results of a study are presented, in which people queried a news archive using an interactive video retrieval system. 242 search sessions by 39 participants on 24 topics were assessed. Before, during and after the study, participants filled in questionnaires about their expectations of a search. The questionnaire data, logged user actions on the system, queries formulated by users, and a quality measure of each search were studied. The results of the study show that topics concerning 'specific' people or objects were better retrieved than topics concerning 'general' objects and scenes. Users were able to estimate the overall quality of a search but did not know when the optimal result was reached within the search process. Analysis of the results at various stages in the retrieval process suggests that retrieval based on transcriptions of the speech in video data adds more to the average precision of the result than content-based image retrieval based on low-level visual features. The latter is particularly useful in providing the user with an overview of the dataset and thus an indication of the success of a search. Based on the results, implications for the design of user interfaces of video retrieval systems are discussed.

Interactive retrieval of video using pre-computed shot-shot similarities

A probabilistic framework for content-based interactive video retrieval is described. The developed indexing of video fragments originates from the probability of the user's positive judgment about key-frames of video shots. Initial estimates of the probabilities are obtained from low-level feature representation. Only statistically significant estimates are picked out, the rest are replaced by an appropriate constant allowing efficient access at search time without loss of search quality and leading to improvement in most experiments. With time, these probability estimates are updated from the relevance judgment of users performing searches, resulting in further substantial increases in mean average precision.

Content-based access to video objects: Temporal Segmentation, visual summarization, and feature extraction

The classical approach to content-based video access has been ‘frame-based’, consisting of shot boundary detection, followed by selection of key frames that characterize the visual content of each shot, and then clustering of the camera shots to form story units. However, in an object-based multimedia environment, content-based random access to individual video objects becomes a desirable feature. To this effect, this paper introduces an ‘object-based’ approach to temporal video partitioning and content-based indexing, where the basic indexing unit is ‘lifespan of a video object’, rather than a ‘camera shot’ or a ‘story unit’. We propose to represent each video object by an adaptive 2D triangular mesh. A mesh-based object tracking scheme is then employed to compute the motion trajectories of all mesh node points until the object exits the field of view. A new similarity measure that is based on motion discontinuities and shape changes of the tracked object is defined to detect content changes, resulting in temporal lifespan segments. A set of ‘key snapshots’ which constitute a visual summary of the lifespan of the object is automatically selected. These key snapshots are then used to animate objects of interest using tracked motion trajectories for a moving visual representation. The proposed scheme provides such functionalities as object-based search/browsing for interactive video retrieval, surveillance video analysis, and object-based content manipulation/editing for studio postprocessing and desktop multimedia authoring. The approach is applicable to any video data where the initial appearance of object(s) can be specified, and the object motion can be modeled by a piecewise affine transformation. The system is demonstrated using different types of video: virtual studio productions (composited video), surveillance video, and TV broadcast video.

Homenet: A Broadband Voice/Data/Video Network on CATV Systems

Homenet is a broadband distributed communication system that supports data, real-time digitized voice, and analog video on a single cable in a CATV type of network. The distance limitation problem encountered in local area networking schemes is eliminated by dividing the large CATV net into smaller “homenets.” This feature makes the network suitable for a large number of users located in a relatively wide geographic scope. This paper describes the implementation of a small experimental version of this system in hardware. More attention is given to the protocol processing hardware, which implements a protocol based on collision detection called Movable Slot Time Division Multiplexing (MSTDM). The MSTDM protocol guarantees the continuity of voice signals received at the user station. Problems such as clock synchronization and confusion of data and voice packets are addressed, and solutions are given. Presently, an experimental network composed of five user nodes in two different frequency nets is operational. An interactive video retrieval service implemented in the network is described as an example of the type of user services (other than data/voice/one-way video) that can be offered at the main head end of the system.