Near-duplicate Video Research Articles

A Novel DC GCN with Attention Mechanism for Accurate Near Duplicate Video Data Cleaning

There has been a steady emergence of nearly identical recordings in the last several decades, thanks to the exponential development of video data. The use of regular videos has been impacted by data quality difficulties produced by near-duplicate movies, which are becoming increasingly noticeable. While there has been progress in the field of near-duplicate video detection, there is still no automated merging method for video data characterised by high-dimensional features. As a result, it is challenging to automatically clean near-duplicate videos in advance video dataset data quality. Research on removing near-duplicate video data is still in its early stages. The precision of near-duplicate video data cleaning is severely compromised by the delicate issues of video data organization besides initial clustering centres in the current research, which arise when the previous distribution is unknown. In tackle these problems, we offer a new kind of Graph Convolutional Neural Network (GCN) that uses dense influences and a categorization attention mechanism. Deeply connected graph convolutional networks (DC-GCNs) learn about faraway nodes by making GCNs deeper. By using dense connections, the DC-GCN is able to multiplex the small-scale features of shallow layers and generate features at diverse scales. Finally, an attention mechanism is incorporated to aid in feature combination and importance determination. Sparrow Search Optimisation Algorithm (SSA) is used to pick the parameters of the given model in the most optimal way. In the end, experiments are carried out using a coal mining video dataset and a widely known dataset called CC_WEB_VIDEO. The simulation findings show that the suggested strategy performs better than certain previous studies.

Extremely compact video representation for efficient near-duplicates detection

With the influx of online video uploads, a robust method for detecting video copies under various distortions is essential for copyright protection and search efficiency. We propose a compact video representation that uses a Siamese Neural Network for near-duplicate detection. This approach achieves up to 0.998 recall and 0.853 precision, even with distorted 56x56px miniatures. Consequently, we derive extremely compact video descriptors, facilitating video fragment retrieval in large datasets. Our method pre-selects frames from a video by identifying local maximums from the interframe differences curve, preserving characteristic sequence patterns even after extreme compression. Utilizing a simpler convolution-based model in the Siamese Neural Network, we improve results by up to 8% over VGG-16, with a 1.7-fold reduction in inference time. With compact descriptors of 1.875 kB, our models outperform others by more than two times. Additionally, we introduce a new dataset with dynamically changing scenes, enhancing its suitability for near-duplicate video detection.

An Automatic Near-Duplicate Video Data Cleaning Method Based on a Consistent Feature Hash Ring

An Automatic Near-Duplicate Video Data Cleaning Method Based on a Consistent Feature Hash Ring

Deep Metric Learning for Near-Duplicate Video Retrieval Leveraging Efficient Semantic Feature Extraction

Deep Metric Learning for Near-Duplicate Video Retrieval Leveraging Efficient Semantic Feature Extraction

Video block and FABEMD features for an effective and fast method of reporting near-duplicate and mirroring videos

Near-duplicate video content has taken the large storage space in the age of big data. Without respecting the copyright ethic, social media users mirror, resize, and/or hide certain online video content and re-upload it as new data. This research aims to avoid the complex and high-dimensional matching and present an efficient approach for detecting near-duplicate videos, this detection is based on feature extraction using visual, motion, and high-level features. Fast and adaptive bidimensional empirical mode decomposition is used to preserve the relevant data to the furthest extent possible during the low/high-frequency transition and vice-versa. In addition, for a generic model, the invariant moments are added to the aforementioned features in order to reinforce them against different video transformations such as rotating and scaling. Furthermore, the video frames are divided into blocks with a fixed number of features, this set of features is represented by a signature, where its mean and standard deviation represents a single video map allowing easy similarity computation. The F1-score and accuracy are used to evaluate the results of this study; the relevant results are ranked by Top_{1} for the best result, and the five top-ranked results are presented by Top_{5}. Further, our result of Top_{1} reached over 80% on F1-score, with a difference of ±4% from the Top_{5} results, and it is over 90% on Accuracy using different datasets, such as UCF11, UCF50, and HDMB51.

Feature Re-Learning for Video Recommendation

Predicting the relevance between two given videos with respect to their visual content is a key component for content-based video recommendation and retrieval. The application is in video recommendation, video annotation, Category or near-duplicate video retrieval, video copy detection and so on. In order to estimate video relevance previous works utilize textual content of videos and lead to poor performance. The proposed method is feature re-learning for video relevance prediction. This work focus on the visual contents to predict the relevance between two videos. A given feature is projected into a new space by an affine transformation. Different from previous works this use a standard triplet ranking loss that optimize the projection process by a novel negative-enhanced triplet ranking loss. In order to generate more training data, propose a data augmentation strategy which works directly on video features. The multi-level augmentation strategy works for video features, which benefits the feature relearning. The proposed augmentation strategy can be flexibly used for frame-level or video-level features. The loss function that consider the absolute similarity of positive pairs and supervise the feature re-learning process and a new formula for video relevance computation.

Effective and Efficient Content Redundancy Detection of Web Videos

Currently, an unprecedentedly vast amount of videos are hosted on the Internet and shared by users across the world. Within these videos, a considerable portion is duplicate or near-duplicate. Consequently, building an effective yet efficient content-based redundancy detection system is of importance, as this research would be beneficial to a variety of applications. Despite the progress in this field, designing a practical detection system for web videos continues to be difficult, because of the contradictions between the accuracy and speed requirements. In this paper, we propose a novel near-duplicate video detection system, CompoundEyes, whose design philosophy deviates from the conventional feature-centered paradigm. Instead, the focus of our system has been shifted from the design of an advanced feature representation to the design of system architecture. This design methodology not only ensures a decent detection accuracy by the collaboration of the classifiers but also substantially accelerates the detection speed due to the low dimensionality of the feature representations and the exploitation of the parallelism among the components. Experiments have been conducted to demonstrate that the CompoundEyes is both accurate and fast.

Joint Multi-View Hashing for Large-Scale Near-Duplicate Video Retrieval

Multi-view hashing can well support large-scale near-duplicate video retrieval, due to its desirable advantages of mutual reinforcement of multiple features, low storage cost, and fast retrieval speed. However, there are still two limitations that impede its performance. First, existing methods only consider local structures in multiple features. They ignore the global structure that is important for near-duplicate video retrieval, and cannot fully exploit the dependence and complementarity of multiple features. Second, existing works always learn hashing functions bit by bit, which unfortunately increases the time complexity of hash function learning. In this paper, we propose a supervised hashing scheme, termed as joint multi-view hashing (JMVH), to address the aforementioned problems. It jointly preserves the global and local structures of multiple features while learning hashing functions efficiently. Specially, JMVH considers features of video as items, based on which an underlying Hamming space is learned by simultaneously preserving their local and global structures. In addition, a simple but efficient multi-bit hash function learning based on generalized eigenvalue decomposition is devised to learn multiple hash functions within a single step. It can significantly reduce the time complexity of conventional hash function learning processes that sequentially learn multiple hash functions bit by bit. The proposed JMVH is evaluated on two public databases: CC_WEB_VIDEO and UQ_VIDEO. Experimental results demonstrate that the proposed JMVH achieves more than a 5 percent improvement compared to several state-of-the-art methods which indicates the superior performance of JMVH.

Advance on large scale near-duplicate video retrieval

Emerging Internet services and applications attract increasing users to involve in diverse video-related activities, such as video searching, video downloading, video sharing and so on. As normal operations, they lead to an explosive growth of online video volume, and inevitably give rise to the massive near-duplicate contents. Near-duplicate video retrieval (NDVR) has always been a hot topic. The primary purpose of this paper is to present a comprehensive survey and an updated review of the advance on large-scale NDVR to supply guidance for researchers. Specifically, we summarize and compare the definitions of near-duplicate videos (NDVs) in the literature, analyze the relationship between NDVR and its related research topics theoretically, describe its generic framework in detail, investigate the existing state-of-the-art NDVR systems. Finally, we present the development trends and research directions of this topic.

IR Feature Embedded BOF Indexing Method for Near-Duplicate Video Retrieval

Due to the explosive increase in online videos, near-duplicate video retrieval (NDVR) has attracted much researcher attention. NDVR has very wide applications, such as copyright protection, online video monitoring, and automatic video tagging. Local features serve as elementary building blocks in many NDVR algorithms, and most of them exploit the local volume information using a bag of features (BOF) representation. However, such representation ignores potentially valuable information about the global distribution of interest points. Moreover, the discriminative power of the local descriptors is significantly reduced by the quantizer in BOF. Our motivation is that if we use the global features to classify the same or similar keyframes into the same class, it will be very useful in improving the performance of NDVR. In this paper, we present an improved radon transform (IR) feature which captures the detailed global geometrical distribution of interest points. It is calculated by using the 2D discrete Radon transform, and then applying a principal component analysis. Such IR feature is not only invariant to the geometry transformations but also robust to the noises. In addition, we propose a fusion strategy to combine the BOF representation with the global IR feature for further improving the recognition accuracy. Convincing experimental results on several publicly available datasets demonstrate that our proposed approach outperforms the state-of-the-art approaches in NDVR.

Stochastic Non-linear Hashing for Near-Duplicate Video Retrieval using Deep Feature applicable to Large-scale Datasets

Stochastic Non-linear Hashing for Near-Duplicate Video Retrieval using Deep Feature applicable to Large-scale Datasets

Near-duplicate Video Retrieval: Current Research and Future Trends - Withdrawn

Near-duplicate Video Retrieval: Current Research and Future Trends - Withdrawn

Hierarchical feature fusion hashing for near-duplicate video retrieval

In recent years, owing to the rapid growth of the number of videos on the Internet, near-duplicate video retrieval (NDVR) by video hashing has attracted huge attention. In the existing methods, the visual features of videos, including single feature and multiple visual feature fusion, are widely used in the NDVR algorithms. However, low-level visual features have some disadvantages in expressing high-level semantics, which may lead to low performance in NDVR. In this paper, we propose a video hashing method for NDVR based on hierarchical feature fusion to address this issue. In the proposed method, low-level handcrafted features from videos were first extracted; then, the intermediate-level deep features and high-level semantic features extracted from the convolutional neural network are obtained. Finally, these semantic features are combined with low-level visual features, where the global structural relationships and complementarity discovered among the hierarchical features are utilized to learn the hash code for NDVR. Extensive experiments are performed on the CC-WEB-VIDEO dataset; the proposed framework proves to have a better retrieval performance compared with the state-of-the-art approaches.

A corpus of debunked and verified user-generated videos

PurposeAs user-generated content (UGC) is entering the news cycle alongside content captured by news professionals, it is important to detect misleading content as early as possible and avoid disseminating it. The purpose of this paper is to present an annotated dataset of 380 user-generated videos (UGVs), 200 debunked and 180 verified, along with 5,195 near-duplicate reposted versions of them, and a set of automatic verification experiments aimed to serve as a baseline for future comparisons.Design/methodology/approachThe dataset was formed using a systematic process combining text search and near-duplicate video retrieval, followed by manual annotation using a set of journalism-inspired guidelines. Following the formation of the dataset, the automatic verification step was carried out using machine learning over a set of well-established features.FindingsAnalysis of the dataset shows distinctive patterns in the spread of verified vs debunked videos, and the application of state-of-the-art machine learning models shows that the dataset poses a particularly challenging problem to automatic methods.Research limitations/implicationsPractical limitations constrained the current collection to three platforms: YouTube, Facebook and Twitter. Furthermore, there exists a wealth of information that can be drawn from the dataset analysis, which goes beyond the constraints of a single paper. Extension to other platforms and further analysis will be the object of subsequent research.Practical implicationsThe dataset analysis indicates directions for future automatic video verification algorithms, and the dataset itself provides a challenging benchmark.Social implicationsHaving a carefully collected and labelled dataset of debunked and verified videos is an important resource both for developing effective disinformation-countering tools and for supporting media literacy activities.Originality/valueBesides its importance as a unique benchmark for research in automatic verification, the analysis also allows a glimpse into the dissemination patterns of UGC, and possible telltale differences between fake and real content.

Global-view hashing: harnessing global relations in near-duplicate video retrieval

Multi-view features are often used in video hashing for near-duplicate video retrieval because of their mutual assistance and complementarity. However, most methods only consider the local available information in multiple features, such as individual or pairwise structural relations, which do not fully utilize the dependent nature of multiple features. We thus propose a global-view hashing (GVH) framework to address the above-mentioned issue; our framework harnesses the global relations among samples characterized by multiple features. In the proposed framework, multiple features of all videos are jointly used to explore a common Hamming space, where the hash functions are obtained by comprehensively utilizing the relations from not only intra-view but also inter-view objects. In addition, the hash function obtained from the proposed GVH can learn multi-bit hash codes in a single iteration. Compared to existing video hashing schemes, the GVH not only globally considers the relations to obtain a more precise retrieval with short-length hash codes but also achieves multi-bit learning in a single iteration. We conduct extensive experiments on the CC_WEB_VIDEO and UQ_VIDEO datasets, and the experimental results show that our proposed method outperforms the state-of-the-art methods. As a side contribution, we will release the codes to facilitate other research.

Near-duplicate Video Detection Algorithm Based on Global GSP Feature and Local ScSIFT Feature Fusion

The main problem with near-duplicate video detection is the high computational complexity and the low efficiency. Near-duplicate video detection methods based on global feature is running fast but with low accuracy, on the contrary, methods based on local feature is accurate, but the calculation is large and time-consuming. Therefore, a near-duplicate video detection algorithm combining global GSP feature and local ScSIFT feature is proposed. Firstly, the video clips and the query ones are discretized into a set of key frame sequences, and the temporal information is recorded at the same time. Secondly, by filtering the video clips on global Gaussian-Scale pyramid feature, similar video clips are selected as the candidates to assure the high recall. Then, combined with the temporal features of the keyframes, the candidate videos are further detected by the local ScSIFT feature to obtain a higher precision. Experimental results show that the proposed algorithm can improve the accuracy of near-duplicate video detection on the basis of guaranteeing the timeliness of the algorithm.

GVoS

The exponential increase of online videos greatly enriches the life of users but also brings huge numbers of near-duplicate videos (NDVs) that seriously challenge the video websites. The video websites entail NDV-related applications such as detection of copyright violation, video monitoring, video re-ranking, and video recommendation. Since these applications adopt different features and different processing procedures due to diverse scenarios, constructing separate and special-purpose systems for them incurs considerable costs on design, implementation, and maintenance. In this article, we propose a general NDV system on Storm (GVoS)—a popular distributed real-time stream processing platform—to simultaneously support a wide variety of video applications. The generality of GVoS is achieved in two aspects. First, we extract the reusable components from various applications. Second, we conduct the communication between components via a mechanism called Stream Shared Message (SSM) that contains the video-related data. Furthermore, we present an algorithm to reduce the size of SSM in order to avoid the data explosion and decrease the network latency. The experimental results demonstrate that GVoS can achieve performance almost the same as the customized systems. Meanwhile, GVoS accomplishes remarkably higher systematic versatility and efficiently facilitates the development of various NDV-related applications.

Joint Compression of Near-Duplicate Videos

The expanding social network and multimedia technologies encourage more and more people to store and transmit information in visual format, such as image and video. However, the cost of this convenience brings about a shock to traditional video severs and exposes them under the risk of overloading. In the huge volume of online videos, there are a large amount of near-duplicate videos (NDVs). Although quite a number of research work have been proposed to detect NDVs, little research effort is made to compress these NDVs in a more effective manner than independent video compression. In this study, we make an in-depth exploration of the data redundancy of NDVs and propose a video analysis and coding framework to jointly compress NDVs. In order to employ the proposed NDV analysis and coding framework, a graph-based similar video grouping method and a number of preprocessing functions are designed to explore the correlation of visual information among NDVs and thus suit the requirement of joint video coding. Experimental results verify that the proposed NDV analysis and coding framework is able to effectively compress NDVs and thus save video data storage.

Comprehensive Feature-Based Robust Video Fingerprinting Using Tensor Model

Content-based near-duplicate video detection (NDVD) is essential for effective search and retrieval, and robust video fingerprinting is a good solution for NDVD. Most existing video fingerprinting methods use a single feature or concatenate different features to generate video fingerprints, and show good performance under single-mode modifications such as noise addition and blurring. However, when they suffer combined modifications, the performance is degraded to a certain extent because such features cannot characterize the video content completely. By contrast, the assistance and consensus among different features can improve the performance of video fingerprinting. Therefore, in the present study, we mine the assistance and consensus among different features based on a tensor model, and we present a new comprehensive feature to fully use them in the proposed video fingerprinting framework. We also analyze what the comprehensive feature really is for representing the original video. In this framework, the video is initially set as a high-order tensor that consists of different features, and the video tensor is decomposed via the Tucker model with a solution that determines the number of components. Subsequently, the comprehensive feature is generated by the low-order tensor obtained from tensor decomposition. Finally, the video fingerprint is computed using this feature. A matching strategy used for narrowing the search is also proposed based on the core tensor. The robust video fingerprinting framework is resistant not only to single-mode modifications but also to their combination.

Multimodal Video-to-Near-Scene Annotation

Traditional video annotation approaches focus on annotating keyframes/shots or whole videos with semantic keywords. However, the extraction processes of keyframes/shots might lack semantic meanings, and it is hard to use a few keywords to describe the content of a long video with multiple topics. In this work, near-scenes, which contain similar concepts, topics, or semantic meanings, are designed for better video content understanding and annotation. We propose a novel framework of hierarchical video-to-near-scene annotation not only to preserve but also to purify the semantic meanings of near-scenes. To detect near-scenes, a pattern-based prefix tree is first constructed to fast retrieve near-duplicate videos. Then, the videos containing similar near-duplicate segments and similar keywords are clustered with consideration of multimodal features including visual and textual features. To enhance the precision of near-scene detection, a pattern-to-intensity-mark (PIM) method is proposed to perform precise frame-level near-duplicate segment alignment. For each near-scene, a video-to-concept distribution model is designed to analyze the representativeness of keywords and discriminations of clusters by the proposed potential term frequency and inverse document frequency and entropy. Tags are ranked according to video-to-concept distribution scores, and the tags with the highest scores are propagated to near-scenes detected. Extensive experiments demonstrate that the proposed PIM outperforms state-of-the-art approaches compared in terms of quality segments and quality frames for near-scene detection. Furthermore, the proposed framework of hierarchical video-to-near-scene annotation can achieve high quality of near-scene annotation in terms of mean average precision.