Video Object Retrieval Research Articles

An Integrated Approach Using Optimized Naive Bayes Classifier and Optical Flow Orientation for Video Object Retrieval

An Integrated Approach Using Optimized Naive Bayes Classifier and Optical Flow Orientation for Video Object Retrieval

Weighed query-specific distance and hybrid NARX neural network for video object retrieval

Abstract The technical revolution in the field of video recording using the surveillance videos has increased the amount of the video databases that caused the need for an efficient video management system. This paper proposes a hybrid model using the nearest search algorithm (NSA) and the Levenberg–Marquardt (LM)-based non-linear autoregressive exogenous (NARX) neural network for performing the video object retrieval using the trajectories. Initially, the position of the objects in the video are retrieved using NSA and NARX individually, and they are averaged to determine the position of the object. The positions determined using the hybrid model is compared with the original database, and the trajectories of the objects are retrieved based on the minimum distance, which depends on the weighed query-specific distance. Experiments have been carried out using seven videos taken from the CAVIAR dataset, and the performance of the proposed method is compared with the existing methods. This proposed method found to be better than the existing method with respect to multiple object tracking precision (MOTP), multiple object tracking accuracy (MOTA), average tracking accuracy (ATA), precision, recall and F-measure that results a greater MOTP rate of 0.8796, precision rate of 0.8154, recall rate of 0.8408, the F-measure at a rate of 0.8371, MOTA of 0.8459 and ATA of 0.8324.

Support vector regression and extended nearest neighbor for video object retrieval

Video retrieval is one of the emerging areas in video capturing that gained various technical advances, increasing the availability of a huge mass of videos. For the text or the image query given, retrieving the relevant videos and the objects from the videos is not always an easy task. A hybrid model was developed in the previous work using the Nearest Search Algorithm (NSA) and exponential weighted moving average (EWMA), for the video object retrieval. In NSA + EWMA, the object trajectories are retrieved based on the query specific distance. This work extends the previous work by developing a novel path equalization scheme for equalizing the path length of the query and the tracked object. Initially, a hybrid model based on Support Vector Regression and NSA tracks the position of the object in the video. The proposed density measure scheme equalizes the path length of the query and the object. Then, the identified path length related to the query is given to extended nearest neighbor classifier for retrieving the video. From the simulation results, it is evident that the proposed video retrieval scheme achieved high values of 0.901, 0.860, 0.849, and 0.922 for precision, recall, F-measure, and multiple object tracking precision, respectively.

Query-Specific Distance and Hybrid Tracking Model for Video Object Retrieval

AbstractIn the area of modern intelligent systems, the retrieval process of video objects is still a challenging task because objects are usually affected by object confusion, similar appearance among objects, different posing, small size of objects, and interactions among multiple objects. In order to overcome these challenges, the video object is retrieved based on the trajectory points of the multiple-motion objects. However, if an object is in an occlusion situation, the calculation of trajectory points from the objects is considerably altered. In order to overcome the above challenges, we have proposed a technique of query-specific distance and hybrid tracking model for video object retrieval. To verify the performance of the proposed method, five videos were collected from the CAVIAR dataset. Then, the proposed tracking process was applied with these five videos and the performance was analysed based on various parameters, such as precision, recall, and f-measure. From the results, we can prove that the proposed hybrid model attained a higher f-measure of 76.7% compared to that of other existing tracking models, such as the nearest neighbourhood algorithmic model and spatial-exponential weighted moving average model.

The use of IMUs for video object retrieval in lightweight devices

We introduce a new object retrieval approach where besides cameras, Inertial Measurement Unit (IMU) sensors are used for the retrieval of 3D objects. Contrary to computationally intensive deep learning recognition and retrieval solutions we focus on lightweight methods which could be utilized in handheld devices and autonomous systems equipped with moderate computing power and memory. We use fast and robust compact image descriptors and the relative orientation of the camera to build multi-view-centered retrieval object models. As for retrieval the Hough transformation paradigm is used to evaluate the results of queries applied on several frames of a video. We analyze the performance of our lightweight approach on several test datasets and with different comparisons, including automatic tracking for the generation of queries. These experiments show the advantages of our proposed techniques since retrieval rate could be significantly increased.

Video Object Retrieval by Trajectory and Appearance

The prevalence of video recording capability, either on surveillance systems or mobile devices, has contributed to the popularity of video data. As a result, video management has become relatively more important than before and, in particular, video retrieval has been one of the main issues in this regard. Traditional video retrieval systems take texts as the inputs to look for similar information from the title, annotation or embedded textual data of a video, in a way that is very similar to the keyword search adopted by a common search engine. However, the lack of visual information specification during a search often makes the result rather inaccurate or even useless. For this reason, video retrieval systems using images or videos as the inputs have also been proposed; nevertheless, the associated ambiguity and complexity have made the implementation of such systems relatively difficult and, therefore, those systems are not as successful as desired. To address this, in this paper, we propose to perform a video retrieval of a desired object through the inputs of its trajectory and/or appearance , together with the help of a 3-D graphical user interface for more intuitive interactions, so that more satisfactory results can be achieved. We firmly believe that such a framework could serve as the foundation for behavior analysis used in many surveillance systems.

Spatio-Temporal Tube data representation and Kernel design for SVM-based video object retrieval system

In this article, we propose a new video object retrieval system. Our approach is based on a Spatio-Temporal data representation, a dedicated kernel design and a statistical learning toolbox for video object recognition and retrieval. Using state-of-the-art video object detection algorithms (for faces or cars, for example) we segment video object tracks from real movies video shots. We then extract, from these tracks, sets of spatio-temporally coherent features that we call Spatio-Temporal Tubes. To compare these complex tube objects, we design a Spatio-Temporal Tube Kernel (STTK) function. Based on this kernel similarity we present both supervised and active learning strategies embedded in Support Vector Machine framework. Additionally, we propose a multi-class classification framework dealing with unbalanced data. Our approach is successfully evaluated on two real movies databases, the french movie "L'esquive" and episodes from "Buffy, the Vampire Slayer" TV series. Our method is also tested on a car database (from real movies) and shows promising results for car identification task.

Video-object segmentation and 3D-trajectory estimation for monocular video sequences

In this paper, we describe a video-object segmentation and 3D-trajectory estimation method for the analysis of dynamic scenes from a monocular uncalibrated view. Based on the color and motion information among video frames, our proposed method segments the scene, calibrates the camera, and calculates the 3D trajectories of moving objects. It can be employed for video-object segmentation, 2D-to-3D video conversion, video-object retrieval, etc. In our method, reliable 2D feature motions are established by comparing SIFT descriptors among successive frames, and image over-segmentation is achieved using a graph-based method. Then, the 2D motions and the segmentation result iteratively refine each other in a hierarchically structured framework to achieve video-object segmentation. Finally, the 3D trajectories of the segmented moving objects are estimated based on a local constant-velocity constraint, and are refined by a Hidden Markov Model (HMM)-based algorithm. Experiments show that the proposed framework can achieve a good performance in terms of both object segmentation and 3D-trajectory estimation.

Enabling MPEG‐7 structural and semantic descriptions in retrieval applications

AbstractThe MPEG‐7 standard supports the description of both the structure and the semantics of multimedia; however, the generation and consumption of MPEG‐7 structural and semantic descriptions are outside the scope of the standard. This article presents two research prototype systems that demonstrate the generation and consumption of MPEG‐7 structural and semantic descriptions in retrieval applications. The active system for MPEG‐4 video object simulation (AMOS) is a video object segmentation and retrieval system that segments, tracks, and models objects in videos (e.g., person, car) as a set of regions with corresponding visual features and spatiotemporal relations. The region‐based model provides an effective base for similarity retrieval of video objects. The second system, the Intelligent Multimedia Knowledge Application (IMKA), uses the novel MediaNet framework for representing semantic and perceptual information about the world using multimedia. MediaNet knowledge bases can be constructed automatically from annotated collections of multimedia data and used to enhance the retrieval of multimedia.

Shape-based retrieval of video objects

The increasing availability of object-based video content requires new technologies for automatically extracting and matching of the low level features of arbitrarily shaped video. This paper proposes methods for shape retrieval of arbitrarily shaped video objects. Our methods take into account not only the still shape features but also the shape deformations that may occur in an object's lifespan. We compute the shape similarity of video objects by comparing the similarity of their representative temporal instances. We also describe motion of a video object via describing the deformations in an object's shape. Experimental results show that our proposed methods offer very good retrieval performance and match closely with the human ranking.

Two-dimensional mesh-based visual-object representation for interactive synthetic/natural digital video

This paper first provides an overview of two-dimensional (2-D) and three-dimensional mesh models for digital video processing. It then introduces 2-D mesh-based modeling of video objects as a compact representation of motion and shape for interactive, synthetic/natural video manipulation, compression, and indexing. The 2-D mesh representation and the mesh geometry and motion compression have been included in the visual tools of the upcoming MPEG-4 standard. Functionalities enabled by 2-D mesh-based visual-object representation include animation of still texture maps, transfiguration of video overlays, video morphing, and shape-and motion-based retrieval of video objects.

NeTra-V: toward an object-based video representation

We present a prototype video analysis and retrieval system, called NeTra-V, that is being developed to build an object-based video representation for functionalities such as search and retrieval of video objects. A region-based content description scheme using low-level visual descriptors is proposed. In order to obtain regions for local feature extraction, a new spatio-temporal segmentation and region-tracking scheme is employed. The segmentation algorithm uses all three visual features: color, texture, and motion in the video data. A group processing scheme similar to the one in the MPEG-2 standard is used to ensure the robustness of the segmentation. The proposed approach can handle complex scenes with large motion. After segmentation, regions are tracked through the video sequence using extracted local features. The results of tracking are sequences of coherent regions, called "subobjects". Subobjects are the fundamental elements in our low-level content description scheme, which can be used to obtain meaningful physical objects in a high-level content description scheme. Experimental results illustrating segmentation and retrieval are provided.