Video Signature Research Articles

Similarity-based ranking of videos from fixed-size one-dimensional video signature

The amount of information is multiplying, one of the popular and widely used formats is short videos. Therefore, maintaining the copyright protection of this information, preventing it from being disclosed without authorization, is a challenge. This work presents a way to rank a set of short videos based on a video profile similarity metric, finding a set of reference videos, using a self-supervised method, without the need for human tagging. The self-supervised method uses a search based on a Genetic Algorithm, of a subgroup of the most similar videos. Similarities are calculated using the SMAPE metric on video signatures vectors, generated with a fixed size, using Structural Tensor, maximum sub matrix and T-SNE.

LieToMe: An Ensemble Approach for Deception Detection from Facial Cues.

Deception detection is a relevant ability in high stakes situations such as police interrogatories or court trials, where the outcome is highly influenced by the interviewed person behavior. With the use of specific devices, e.g. polygraph or magnetic resonance, the subject is aware of being monitored and can change his behavior, thus compromising the interrogation result. For this reason, video analysis-based methods for automatic deception detection are receiving ever increasing interest. In this paper, a deception detection approach based on RGB videos, leveraging both facial features and stacked generalization ensemble, is proposed. First, a face, which is well-known to present several meaningful cues for deception detection, is identified, aligned, and masked to build video signatures. These signatures are constructed starting from five different descriptors, which allow the system to capture both static and dynamic facial characteristics. Then, video signatures are given as input to four base-level algorithms, which are subsequently fused applying the stacked generalization technique, resulting in a more robust meta-level classifier used to predict deception. By exploiting relevant cues via specific features, the proposed system achieves improved performances on a public dataset of famous court trials, with respect to other state-of-the-art methods based on facial features, highlighting the effectiveness of the proposed method.

Integration of Image and Video Signature in Graphical Password Authentication System

Integration of Image and Video Signature in Graphical Password Authentication System - written by Vaishali Ravi , Seema Khan P , Usha H Y published on 2020/06/06 download full article with reference data and citations

Ownership Identification of Multimedia Content Prediction on a Large Scale

This article prepared with three main objectives, first it is to review the large size of multimedia file protection, second for ownership identification of the multimedia content and finally for implementing this setup for cloud based setup instead of web based service architecture. Therefore many of the infrastructure developed for multimedia content to enable the security against the file handling services for managing the better ownership identification. On private and/or public clouds, the device can be deployed. Our framework has two new components: (i) method of producing 3-D image signatures, and (ii) distributed multimedia object matching engine. The signature method produces stable and accurate 3-D video signatures capturing the depth signals in those videos as well as computationally efficient processing and comparison and having limited space. The corresponding distributed engine is highly scalable and designed to support different multimedia objects. We have introduced and deployed the proposed system on two clouds such as private and amazon based clouds.

Randomized neural network based signature for dynamic texture classification

Dynamic texture analysis has been the focus of intensive research in recent years. Thus, this paper presents an innovative and highly discriminative dynamic texture analysis method, whose signature is composed of the weights of the output layer of a randomized neural network after a training procedure. This training is performed by using the pixels of slices of each orthogonal plane of the video (XY, YT, and XT) as input feature vectors and corresponding output labels. The obtained video signature provided an accuracy of 97.05%, 98.54%, 97.74% and 96.51% on the UCLA-50 classes, UCLA-9 classes, UCLA-8 classes and Dyntex++, respectively. These results, when compared to other dynamic texture analysis methods, demonstrate that our descriptors are very effective and that our proposed approach can contribute significantly to the field of dynamic texture analysis.

Graph-based video sequence matching using dominant colour graph profile (DCGP)

This paper presents a fast and effective technique for videos’ visual similarity detection and measurement using compact fixed-length signatures. The proposed technique (dominant colour graph profile DCGP) extracts and encodes the spatio-temporal information of a given video shot into a graph-based structure (tree) that fully captures this vital information. The graph structured properties are utilized to construct a fixed-length video signature of 112 decimal values per video shot. The encoded spatio-temporal information is extracted following channelling each video frame into a block-based structure, where the positions of respective blocks are tracked across video frames and encoded into multiple DCGP trees. The proposed technique provides a high matching speed (>2000 fps) and robust retrieval performance. The experiments on various standard and challenging datasets shows the framework’s robust performance, in terms of both, retrieval and computational performances.

Ranking, clustering and fusing the normalized LBP temporal facial features for face recognition in video sequences

This paper proposes a novel approach for recognizing faces in videos with high recognition rate. Initially, the feature vector based on Normalized Local Binary Patterns is obtained for the face region. A set of training and testing videos are used in this face recognition procedure. Each frame in the query video is matched with the signature of the faces in the database using Euclidean distance and a rank list is formed. Each ranked list is clustered and its reliability is analyzed for re-ranking. Multiple re-ranked lists of the query video is fused together to form a video signature. This video signature embeds diverse intra-personal variations such as poses, expressions and facilitates in matching two videos with large variations. For matching two videos, their composite ranked lists are compared using a Kendall Tau distance measure. The developed methods are deployed on the YouTube and ChokePoint videos, and they exhibit significant performance improvement owing to their novel approach when compared with the existing techniques.

Spatial Pyramid Covariance-Based Compact Video Code for Robust Face Retrieval in TV-Series.

We address the problem of face video retrieval in TV-series, which searches video clips based on the presence of specific character, given one face track of his/her. This is tremendously challenging because on one hand, faces in TV-series are captured in largely uncontrolled conditions with complex appearance variations, and on the other hand, retrieval task typically needs efficient representation with low time and space complexity. To handle this problem, we propose a compact and discriminative representation for the huge body of video data, named compact video code (CVC). Our method first models the face track by its sample (i.e., frame) covariance matrix to capture the video data variations in a statistical manner. To incorporate discriminative information and obtain more compact video signature suitable for retrieval, the high-dimensional covariance representation is further encoded as a much lower dimensional binary vector, which finally yields the proposed CVC. Specifically, each bit of the code, i.e., each dimension of the binary vector, is produced via supervised learning in a max margin framework, which aims to make a balance between the discriminability and stability of the code. Besides, we further extend the descriptive granularity of covariance matrix from traditional pixel-level to more general patch-level, and proceed to propose a novel hierarchical video representation named spatial pyramid covariance along with a fast calculation method. Face retrieval experiments on two challenging TV-series video databases, i.e., the Big Bang Theory and Prison Break, demonstrate the competitiveness of the proposed CVC over the state-of-the-art retrieval methods. In addition, as a general video matching algorithm, CVC is also evaluated in traditional video face recognition task on a standard Internet database, i.e., YouTube Celebrities, showing its quite promising performance by using an extremely compact code with only 128 bits.

Fusing the facial temporal information in videos for face recognition

Face recognition is a challenging and innovative research topic in the present sophisticated world of visual technology. In most of the existing approaches, the face recognition from the still images is affected by intra‐personal variations such as pose, illumination and expression which degrade the performance. This study proposes a novel approach for video‐based face recognition due to the availability of large intra‐personal variations. The feature vector based on the normalised semi‐local binary patterns is obtained for the face region. Each frame is matched with the signature of the faces in the database and a rank list is formed. Each ranked list is clustered and its reliability is analysed for re‐ranking. To characterise an individual in a video, multiple re‐ranked lists across the multiple video frames are fused to form a video signature. This video signature embeds diverse intra‐personal and temporal variations, which facilitates in matching two videos with large variations. For matching two videos, their video signatures are compared using Kendall‐Tau distance. The developed methods are deployed on the YouTube and ChokePoint videos, and they exhibit significant performance improvement owing to their approach when compared with the existing techniques.

Modeling spatio-temporal layout with Lie Algebrized Gaussians for action recognition

We propose a novel approach to model spatio-temporal distribution of local features for action recognition in videos. The proposed approach is based on the Lie Algebrized Gaussians (LAG) which is a feature aggregation approach and yields high-dimensional video signature. In the framework of LAG, local features extracted from a video are aggregated to train a video-specific Gaussian Mixture Model (GMM). Then the video-specific GMM is encoded as a vector based on Lie group theory and this step is also referred to as GMM vectorization. As the video-specific GMM gives a soft partition of the feature space, for each cell of the feature space (i.e. each Gaussian component), we use a GMM to model the spatio-temporal locations of the local features assigned to the Gaussian component. The location GMMs are encoded as vectors just like the local feature GMM. We term those vectors of location GMMs spatio-temporal LAG (STLAG). In addition, although the LAG and the popular Fisher Vector (FV) are derived from distinct theory perspectives, we find that they are closely related. Hence the power and l2 normalization proposed for the FV are also beneficial to the LAG. Experimental results show that STLAG is very effective to model spatio-temporal layout compared with other techniques such as spatio-temporal pyramid and feature augmentation. Using the state-of-the-art dense trajectory features, our approach achieves state-of-the-art performance on two challenging datasets: Hollywood2 and HMDB51.

Content Identification using Video Fingerprint based on Video Classification

nternet usage has drastically increased in recent years. Based on the topic a tool for video content identification and video classification is presented. Tool uses MPEG-7 standard. Thus video streams are easily available in various forms. O ne must appreciate text-based search engines for immediate access to such a vast multimedia data on web. But currently, only text-based search engines are obtainable. So a tool that will search for a unique content in the video not in the name of the video is proposed. Tool will make use of video fingerprint. Video Fingerprint is a unique identity of a video most commonly called as video signature. The video signature has numerous applications like distribution management, corporate or personal database management, rights management and monetization, metadata association, and usage monitoring. We will be making use of this Video content identification tool for implementation of Topic linking concept, a method of organizing contents based on topics, presentation videos and provides such efficient way of browsing, such as news programs. This in turn helps to categorize the video according to its possessed content and store and manage the multimedia data in an easy way. Keywordsopiclinking,VideoContentIdentification,MPEG-7,

동영상 복사본 검출을 위한 MPEG-7 Video Signature 성능분석

In recent years, we can access to video contents anywhere and at any time. Therefore distributed video is easily copied, transformed and republished. Since it brings copyright problem, similarity/duplicate detection and measurement is essential to identify the excessive content duplication. In this paper, we analysed various discernment of video which has been transformed with various ways using MPEG-7 Video Signature. MPEG-7 Video Signature, one of video copy detection algorithms, is block based abstraction. Thus we assume Video Signature is weak for spatial transform. The experiments show that MPEG-7 Video Signature is very weak for spatial transform which could occur general as we have assumed.

On Recognizing Faces in Videos Using Clustering-Based Re-Ranking and Fusion

Due to widespread applications, availability of large intra-personal variations in video and limited information content in still images, video-based face recognition has gained significant attention. Unlike still face images, videos provide abundant information that can be leveraged to address variations in pose, illumination, and expression as well as enhance the face recognition performance. This paper presents a video-based face recognition algorithm that computes a discriminative video signature as an ordered list of still face images from a large dictionary. A three-stage approach is proposed for optimizing ranked lists across multiple video frames and fusing them into a single composite ordered list to compute the video signature. This signature embeds diverse intra-personal variations and facilitates in matching two videos with large variations. For matching two videos, a discounted cumulative gain measure is utilized, which uses the ranking of images in the video signature as well as the usefulness of images in characterizing the individual in the video. The efficacy of the proposed algorithm is evaluated under different video-based face recognition scenarios such as matching still face images with videos and matching videos with videos. The efficacy of the proposed algorithm is demonstrated on the YouTube faces database and the MBGC v2 video challenge database that comprise different types of video-based face recognition challenges such as matching still face images with videos and matching videos with videos. Performance comparison with the benchmark results on both the databases and a commercial face recognition system shows the efficiency of the proposed algorithm for video-based face recognition.

A Frame-Based Video Signature Method for Very Quick Video Identification and Location

A video signature is a set of feature vectors that compactly represents and uniquely characterizes one video clip from another for fast matching. To find a short duplicated region, the video signature must be robust against common video modifications and have a high discriminability. The matching method must be fast and be successful at finding locations. In this paper, a frame-based video signature that uses the spatial information and a two-stage matching method is presented. The proposed method is pair-wise independent and is robust against common video modifications. The proposed two-stage matching method is fast and works very well in finding locations. In addition, the proposed matching structure and strategy can distinguish a case in which a part of the query video matches a part of the target video. The proposed method is verified using video modified by the VCE7 experimental conditions found in MPEG-7. The proposed video signature method achieves a robustness of 88.7% under an independence condition of 5 parts per million with over 1,000 clips being matched per second.

A Frame-Based Video Signature Method for Very Quick Video Identification and Location

A video signature is a set of feature vectors that compactly represents and uniquely characterizes one video clip from another for fast matching. To find a short duplicated region, the video signature must be robust against common video modifications and have a high discriminability. The matching method must be fast and be successful at finding locations. In this paper, a frame-based video signature that uses the spatial information and a two-stage matching method is presented. The proposed method is pair-wise independent and is robust against common video modifications. The proposed two-stage matching method is fast and works very well in finding locations. In addition, the proposed matching structure and strategy can distinguish a case in which a part of the query video matches a part of the target video. The proposed method is verified using video modified by the VCE7 experimental conditions found in MPEG-7. The proposed video signature method achieves a robustness of 88.7% under an independence condition of 5 parts per million with over 1,000 clips being matched per second.

大規模な映像識別のための改変にロバストなVideo Signature

This paper presents comparative evaluation of a robust frame-level descriptor for video identification, called a Video Signature, which was previously developed by the authors and was accepted as part of a new ISO/IEC standard “MPEG-7 Video Signature Tools”. The Video Signature is designed for video copy detection from a large-scale database, with robustness to various modifications, including caption overlay, camera capturing, compression, and color changes, working at extremely low false positive rate. It represents intensity differences between various sub-regions in a frame, which are configured at variety of scales, shapes, and locations in the frame, to provide robustness and uniqueness to the descriptor. The intensity differences are quantized into ternary values, resulting in a compact representation of 76 bytes per frame. Furthermore, a confidence value is calculated for each frame, which is used in the matching process to significantly reduce false positives. An experiment to identify videos under nine types of modifications was carried out to compare the performance of the Video Signature to that of the conventional color-based and spatial visual features. The results show that the Video Signatures improves the detection rate for all modification types compared with the conventional features, in particular significant increase by 38.8% for caption overlay and 61.8% for camera capturing have been achieved. We also present a system for structuring a large-scale video database using the Video Signature, developed for efficient management of video contents. The details of the system are explained and its feasibility as an application is shown.

Similar video detection using multiple direct-mapped cache

Recently, the importance of video detection technology has increased. In our earlier studies, we had proposed the single direct-mapped cache method based on the RGB colour space feature to handle the processing of a large amount of videos on the internet. Although our method achieved a high processing speed, the detection accuracy had room for improvement. In this paper, we propose a new method to improve the accuracy. For this purpose, the combination of MPEG-7 video signature and the multiple direct-mapped cache method is used. The experimental results show that the average detection accuracy is 95% and the processing time is 0.2 seconds for each video. These results indicate a 29% improvement in accuracy over the previous method. The possibility of handling 136 years worth of video data while keeping the same accuracy and processing speed is also one of the unique features of the proposed method.

Performance of multi camera views’ detection using MPEG-7 visual signature tools

AbstractThe paper presents experimental results for the detection of related video streams (views) of multi camera system using MPEG-7 visual signature tools. This detection can be used for identification of video streams stored in video repositories which constitute the same multi camera recording set. Two signature tools have been investigated: Image Signature descriptor and Video Signature descriptor. The motivation for the usage of the tools designed primarily for replica detection is the fact that the video material produced by multi camera system typically consists of a few related video streams which are similar to each other and, thus, the video streams produced by multi camera system can be regarded as semi-replicas as they contain predominantly the same scenes or objects recorded at the same time. Experiments show that only in some restricted scenarios of a multi camera system setup the MPEG-7 visual signature tools can be used for reliable detection of multi camera views. These reliable scenarios include systems having cameras with perpendicular optical axes and “far” scene.

Frame based Video Retrieval using Video Signatures

The World Wide Web today has grown so wide and the video-on-demand applications and video share web are becoming very popular day-by-day on the World Wide Web. An efficient video similarity search algorithm for contentbased video retrieval is important in video-on-demand based services. However, there is no satisfying video similarity search algorithm showing cent percentage performance. It is proposed here to implement a video similarity measure algorithm based on the color-features of each video represented by a compact fixed size representation known as Video Signature. This Video signature which is based on the image signature is computed on the basis of YCbCr Histogram and the sum of its weighted means. The video signatures of videos are then used to find the similar videos in-terms of visually similar frames, by using the range. This method of similarity measure is assumed to be efficient in various aspects.

The MPEG-7 Video Signature Tools for Content Identification

This paper presents the core technologies of the video signature tools recently standardized by ISO/IEC Moving Picture Experts Group (MPEG) as an amendment to the MPEG-7 Standard (ISO/IEC 15938). The video signature is a high-performance content fingerprint that is suitable for desktop scale to web-scale deployment and provides high levels of robustness to common video editing operations and high temporal localization accuracy at extremely low false alarm rates, achieving a detection rate in the order of 96% at a false alarm rate in the order of five false matches per million comparisons. The applications of the video signature are numerous and include rights management and monetization, distribution management, usage monitoring, metadata association, and corporate or personal database management. In this paper, we review the prior work in the field, explain the standardization process and status, and provide details and evaluation results for the video signature tools.