Video Retrieval System Research Articles

Content-based retrieval system for surgery videos

There has been tremendous growth of laparoscopic surgical video archives in the past few years. Finding the surgical videos manually from the database is very troublesome and time consuming. As a consequence, efficient methods for retrieval of such videos from a database are needed. In this paper, a content-based video retrieval (CBVR) system is presented using average intensity, colour and shape feature. In the proposed retrieval system, average intensity, colour and shape feature are used. The ordinal measure introduced by Bhat and Nayar, which is based on ranks rather than of their values is used in this work. This is important in the laparoscopic videos because adjacent frames can be distinguished on the basis of ranking of their average intensities. Experimental results reveal that mean average precision of the proposed technique is better than the existing motion intensity and direction descriptor (MIDD) approach. This reflects the effectiveness of proposed method.

A cloud-based face video retrieval system with deep learning

Face video retrieval is an attractive research topic in computer vision. However, it remains challenges to overcome because of the significant variation in pose changes, illumination conditions, occlusions, and facial expressions. In video content analysis, face recognition has been playing a vital role. Besides, deep neural networks are being actively studied, and deep learning models have been widely used for object detection, especially for face recognition. Therefore, this study proposes a cloud-based face video retrieval system with deep learning. First, a dataset is collected and pre-processed. To produce a useful dataset for the CNN models, blurry images are removed, and face alignment is implemented on the remaining images. Then the final dataset is constructed and used to pre-train the CNN models (VGGFace, ArcFace, and FaceNet) for face recognition. We compare the results of these three models and choose the most efficient one to develop the system. To implement a query, users can type in the name of a person. If the system detects a new person, it performs enrolling that person. Finally, the result is a list of images and time associated with those images. In addition, a system prototype is implemented to verify the feasibility of the proposed system. Experimental results demonstrate that this system outperforms in terms of recognition accuracy and computational time.

Using diverse set of features to design a content-based video retrieval system optimised by gravitational search algorithm

Using diverse set of features to design a content-based video retrieval system optimised by gravitational search algorithm

A Review of Content-Based Video Retrieval Techniques for Person Identification

The rise of technology spurs the advancement in the surveillance field. Many commercial spaces reduced the patrol guard in favor of Closed-Circuit Television (CCTV) installation and even some countries already used surveillance drone which has greater mobility. In recent years, the CCTV Footage have also been used for crime investigation by law enforcement such as in Boston Bombing 2013 incident. However, this led us into producing huge unmanageable footage collection, the common issue of Big Data era. While there is more information to identify a potential suspect, the massive size of data needed to go over manually is a very laborious task. Therefore, some researchers proposed using Content-Based Video Retrieval (CBVR) method to enable to query a specific feature of an object or a human. Due to the limitations like visibility and quality of video footage, only certain features are selected for recognition based on Chicago Police Department guidelines. This paper presents the comprehensive reviews on CBVR techniques used for clothing, gender and ethnic recognition of the person of interest and how can it be applied in crime investigation. From the findings, the three recognition types can be combined to create a Content-Based Video Retrieval system for person identification.

Content Based Video Retrieval by Using Distributed Real-Time System Based on Storm

Time processing is a challenging issue for content-based video retrieval systems, especially when the process of indexing, classifying and retrieving desired and relevant videos is from a huge database. A CBVR system called bounded coordinate of motion histogram (BCMH) has been implemented as a case study. The BCMH offline step requires a long time to complete the learning phase, and the online step falls short in addressing the real-time video processing. To overcome these drawbacks, this article presents a batch-oriented computing based on Apache Hadoop to improve the time processing for the offline step, and a real-time oriented computing based on Apache Storm topologies to achieve a real-time response for the online step. The proposed approach is tested on the HOLLYWOOD2 dataset and the obtained results demonstrate reliability and efficiency of the proposed method.

The development of a video retrieval system using a clinician-led approach

Patient video taken at home can provide valuable insights into the recovery progress during a programme of physical therapy, but is very time consuming for clinician review. Our work focussed on (i) enabling any patient to share information about progress at home, simply by sharing video and (ii) building intelligent systems to support Physical Therapists (PTs) in reviewing this video data and extracting the necessary detail. This paper reports the development of the system, appropriate for future clinical use without reliance on a technical team, and the clinician involvement in that development. We contribute an interactive content-based video retrieval system that significantly reduces the time taken for clinicians to review videos, using human head movement as an example. The system supports query-by-movement (clinicians move their own body to define search queries) and retrieves the essential fine-grained movements needed for clinical interpretation. This is done by comparing sequences of image-based pose estimates (here head rotations) through a distance metric (here Fréchet distance) and presenting a ranked list of similar movements to clinicians for review. In contrast to existing intelligent systems for retrospective review of human movement, the system supports a flexible analysis where clinicians can look for any movement that interests them. Evaluation by a group of PTs with expertise in training movement control showed that 96% of all relevant movements were identified with time savings of as much as 99.1% compared to reviewing target videos in full. The novelty of this contribution includes retrospective progress monitoring that preserves context through video, and content-based video retrieval that supports both fine-grained human actions and query-by-movement. Future research, including large clinician-led studies, will refine the technical aspects and explore the benefits in terms of patient outcomes, PT time, and financial savings over the course of a programme of therapy. It is anticipated that this clinician-led approach will mitigate the reported slow clinical uptake of technology with resulting patient benefit.

Content based video retrieval system using principal object analysis

Video retrieval is a searching problem on videos or clips based on the content of video clips which relates to the input image or video. Some recent approaches have been in challenging problem due to the diversity of video types, frame transitions and camera positions. Besides, that an appropriate measures is selected for the problem is a question. We propose a content based video retrieval system in some main steps resulting in a good performance. From a main video, we process extracting keyframes and principal objects using Segmentation of Aggregating Superpixels (SAS) algorithm. After that, Speeded Up Robust Features (SURF) are selected from those principal objects. Then, the model “Bag-of-words” in accompanied by SVM classification are applied to obtain the retrieval result. Our system is evaluated on over 300 videos in diversity from music, history, movie, sports, and natural scene to TV program show.

Video Big Data Retrieval Over Media Cloud: A Context-Aware Online Learning Approach

Online video sharing (e.g., via YouTube or YouKu) has emerged as one of the most important services in the current Internet, where billions of videos on the cloud are awaiting exploration. Hence, a personalized video retrieval system is needed to help users find interesting videos from big data content. Two of the main challenges are to process the increasing amount of video big data and resolve the accompanying “cold start” issue efficiently. Another challenge is to satisfy the users’ need for personalized retrieval results, of which the accuracy is unknown. In this paper, we formulate the personalized video big data retrieval problem as an interaction between the user and the system via a stochastic process, not just a similarity matching, accuracy (feedback) model of the retrieval; introduce users’ real-time context into the retrieval system; and propose a general framework for this problem. By using a novel contextual multiarmed bandit-based algorithm to balance the accuracy and efficiency, we propose a context-based online big-data-oriented personalized video retrieval system. This system can support datasets that are dynamically increasing in size and has the property of cross-modal retrieval. Our approach provides accurate retrieval results with sublinear regret and linear storage complexity and significantly improves the learning speed. Furthermore, by learning for a cluster of similar contexts simultaneously, we can realize sublinear storage complexity with the same regret but slightly poorer performance on the “cold start” issue compared to the previous approach. We validate our theoretical results experimentally on a tremendously large dataset; the results demonstrate that the proposed algorithms outperform existing bandit-based online learning methods in terms of accuracy and efficiency and the adaptation from the bandit framework offers additional benefits.

Protagonist and deuteragonist-based video indexing and retrieval system for movie and video song sequences

Protagonist and deuteragonist-based video indexing and retrieval system for movie and video song sequences

Protagonist and deuteragonist-based video indexing and retrieval system for movie and video song sequences

Protagonist and deuteragonist are two main characters that plays leading role in Indian Hindi movie (IHM). Currently such information is attached using textual caption which is highly unreliable. The research presented in this paper is to automatically index and retrieve the content based on protagonist and deuteragonist from the large IHM and video song sequences (VSS). Video song sequence indexes are extracted using audio-based approach from IHM in module 1. These indexes are used as an input in module 2. Faces are identified from every VSS. Colour histogram and spatiogram descriptor are extracted from faces. Similarity between two faces is computed using Bhattacharyya coefficient. Similarity-based clustering technique is performed to obtain clusters of faces. Recognition of protagonist and deuteragonist is done using surf feature points. The experimental results are carried out using Indian Hindi movies of different genres.

An MPEG-7 Compatible Video Retrieval System with Integrated Support for Complex Multimodal Queries

Early prototype multimedia database management systems used the query-by-example paradigm to respond to user queries. Users needed to formulate their queries by providing examples or sketches. The query-by-keyword paradigm, on the other hand, has emerged due to the desire to search multimedia content in terms of semantic concepts using keywords or sentences rather than low-level multimedia descriptors. After all, it's much easier to formulate some queries by using keywords. However, some queries are still easier to formulate by examples or sketches-for example, the trajectory of a moving object.

Derivations of design concepts for video search interfaces

PurposeKeywords search is intuitive, simple to use and convenient. It is also the de facto input interface for textual and multimedia retrieval. However, individuals often perform poorly when faced with exploratory search tasks that are common during learning, resulting in poor quality searches. The purpose of this paper is to examine how adolescent learners search and select videos to support self-learning. The findings allow for the identification of design concepts of video retrieval interface and features that can facilitate better exploratory searches.Design/methodology/approachParticipants were assigned two customized video search tasks. The think-aloud protocol is used to allow participants to verbalize their actions, thoughts and feeling. This approach offered rich insights to the participants’ cognitive processes and considerations when performing the search tasks.FindingsThis study identified five themes for exploratory video search behavior: selection of internet resources, query formulation/reformulation, selection of the video(s) for preview, getting acquainted with the video content, and making a decision for the search task. The analysis of these themes led to a number of design concepts, ranging from supporting exploration of topics to better interaction with metadata.Practical implicationsThe findings can inform future development of dedicated video retrieval systems interfaces that seeks to facilitate effective exploratory searches by learners.Originality/valueThis study contributes by suggesting design concepts for video retrieval system developers to support exploratory video searches.

Content Based Video Retrieval Systems with Local Features: A Survey

With the improvement of mixed media information composes and accessible transfer speed there is immense interest of video retrieving frameworks, as clients move from content based recovery frameworks to content based retrieval frameworks. Determination of removed features assume an imperative job in substance based video retrieving paying little mind to video qualities being under thought. This work assists the up and coming analysts in the field of video retrieving with getting the thought regarding distinctive procedures and strategies accessible for the video recovery. These highlights are proposed for choosing, ordering and positioning as indicated by their potential enthusiasm to the client. Great feature determination likewise permits the time and space expenses of the recovery procedure to be lessened. This overview surveys the fascinating highlights that can be separated from video information for ordering and retrieving alongside likeness estimation techniques. We likewise recognize present research issues in territory of content based video retrieving frameworks.

Video Retrieval System for Meniscal Surgery to Improve Health Care Services

Meniscal surgery is considered the most general orthopedic process that deals with the treatment of meniscus tears for human health care. It leads to a communal contusion to the cartilage that stabilizes and cushions the knee joints of human beings. Such tears can be classified into different categories based on age group, region, and occupation. Further, a large number of sportsmen and heavy weightlifters even in developed countries are affected by meniscus injuries. These patients are subjected to arthroscopic surgery, and during surgical treatment, the perseverance of meniscus is a very crucial task. Current research provides a significant ratio of meniscal tear patients around the globe, the critical expanse is considered as having strikingly risen with a mean annual of 0.066% due to surgery failure. To decumbent this ratio, an innovative training mechanism is proposed through video retrieval system in this research. This research work is focussed on developing a corpus and video retrieval system for meniscus surgery. Using the proposed system, surgeons can access guidance by watching the videos of surgeries performed by an expert and their seniors. The proposed system is comprised of four approaches to the spatiotemporal methodology to improve health care services. It entails key point, statistical modeling, PCA-scale invariant feature transform (SIFT), and PCA-Gaussian mixture model (GMM) with a combination of sparse-optical flow. The real meniscal surgery dataset is used for testing purposes and evaluation. The results conclude that using PCA-SIFT approach improves the results with an average precision of 0.78.

Learning a Multi-Concept Video Retrieval Model with Multiple Latent Variables

Effective and efficient video retrieval has become a pressing need in the “big video” era. The objective of this work is to provide a principled model for computing the ranking scores of a video in response to one or more concepts, where the concepts could be directly supplied by users or inferred by the system from the user queries. Indeed, how to deal with multi-concept queries has become a central component in modern video retrieval systems that accept text queries. However, it has been long overlooked and simply implemented by weighted averaging of the corresponding concept detectors’ scores. Our approach, which can be considered as a latent ranking SVM, integrates the advantages of various recent works in text and image retrieval, such as choosing ranking over structured prediction, modeling inter-dependencies between querying concepts, and so on. Videos consist of shots, and we use latent variables to account for the mutually complementary cues within and across shots. Concept labels of shots are scarce and noisy. We introduce a simple and effective technique to make our model robust to outliers. Our approach gives superior performance when it is tested on not only the queries seen at training but also novel queries, some of which consist of more concepts than the queries used for training.

Semantic Based Video Retrieval System: Survey

In this review paper a number of studies and researches are surveyed, in order to assist the upcoming researchers, to know about the techniques available in the field of semantic based video retrieval. The video retrieval system is used for finding the users’ desired video among a huge number of available videos on the Internet or database. This paper gives a general discussion on the overall process of the semantic video retrieval phases. In addition to its present a generic review of techniques that has been proposed to solve the semantic gap as the major scientific problem in semantic based video retrieval. The semantic gap is formed because of the difference between the low level features that are extracted from video content and user's perceptions of these features in a real world. The transformation of low level features of the video content into meaningful semantic concepts is a research topic that has received considerable attention in recent years.

Multi-modal features and correlation incorporated Naive Bayes classifier for a semantic-enriched lecture video retrieval system

ABSTRACTWith the advancement of science and technology and the development of internet sources, e-learning draws huge attention as it is capable of providing classroom lectures in the form of videos. Here, a method has been proposed for effective retrieval of the lecture videos, which employs the correlated naive Bayes (CNB) classifier. In this proposed method, Optical Character Recognition uses the tesseract classifier and GOM (Gabor Ordinal Measure) to extract the textual features and the image texture from the key frames. K-means clustering clusters the features and the classifier retrieves the relevant video. Experimentation has been done in the MATLAB and the parameters such as precision, recall, and F measure of the CNB are compared over the other methods such as K-Nearest Neighbour and naive Bayes (NB) classifiers. The CNB classifier achieves a better precision, recall, and F measure rate of 0.9366, 0.9511, and 0.9426, respectively.

Using Diverse Set of Features to Design a Content-Based Video Retrieval System Optimized by Gravitational Search Algorithm

This paper explains about the content-based video retrieval approach (CBVR) using four varieties of features and 12 distance measurements, which is optimised by gravitational search algorithm (GSA). Initially, CBVR technique extracts five kinds of features such as colour, texture, shape, image and audio features that belong to each frame. Consequently, it emerges particular distance measurements for every sort of features to compute the similarity between query frame and remaining in the database frame. In this paper, we have used GSA to find the nearly optimal combination between the features and their respective similarity measurements. At last, from the video database, the query-based videos are recovered. For experimentation, here we used two types of databases such as sports video and UCF sports action datasets. The experimental results demonstrate that the proposed CBVR method shows better performance when contrasted with other existing methods.

Deep Learning for Retrieval of Natural Flower Videos

In this paper we have proposed a flower video retrieval system using Deep learning approach for training and learning features. Network is trained in three ways; with keyframes, with segmented flowers and with gradient of the flowers as input. For a given query video, the system retrieves similar videos from the database using Multiclass Support Vector Machine (MSVM). An extensive experimentation has been conducted on a relatively huge flower video data set of our own consisting of more than 2600 flower videos belonging to 30 different classes of flowers. It is observed that the complexity of the system can be reduced to a larger extend by training the DCNN using gradient of the flowers without compromising the performance. However, among the three ways, the system trained with segmented keyframes has shown best retrieval accuracy. It has also been observed that the proposed DCNN based approach out performs traditional retrieval approach.

Depth-based human activity recognition: A comparative perspective study on feature extraction

Depth Maps-based Human Activity Recognition is the process of categorizing depth sequences with a particular activity. In this problem, some applications represent robust solutions in domains such as surveillance system, computer vision applications, and video retrieval systems. The task is challenging due to variations inside one class and distinguishes between activities of various classes and video recording settings. In this study, we introduce a detailed study of current advances in the depth maps-based image representations and feature extraction process. Moreover, we discuss the state of art datasets and subsequent classification procedure. Also, a comparative study of some of the more popular depth-map approaches has provided in greater detail. The proposed methods are evaluated on three depth-based datasets “MSR Action 3D”, “MSR Hand Gesture”, and “MSR Daily Activity 3D”. Experimental results achieved 100%, 95.83%, and 96.55% respectively. While combining depth and color features on “RGBD-HuDaAct” Dataset, achieved 89.1%.