Shot Boundary Detection Method Research Articles

Video shot boundary detection based on frames objects comparison and scale-invariant feature transform technique

The most popular source of data on the Internet is video which has a lot of information. Automating the administration, indexing, and retrieval of movies is the goal of video structure analysis, which uses content-based video indexing and retrieval. Video analysis requires the ability to recognize shot changes since video shot boundary recognition is a preliminary stage in the indexing, browsing, and retrieval of video material. A method for shot boundary detection (SBD) is suggested in this situation. This work proposes a shot boundary detection system with three stages. In the first stage, multiple images are read in temporal sequence and transformed into grayscale images. Based on correlation value comparison, the number of redundant frames in the same shots is decreased, from this point on, the amount of time and computational complexity is reduced. Then, in the second stage, a candidate transition is identified by comparing the objects of successive frames and analyzing the differences between the objects using the standard deviation metric. In the last stage, the cut transition is decided upon by matching key points using a scale-invariant feature transform (SIFT). The proposed system achieved an accuracy of 0.97 according to the F-score while minimizing time consumption.

Propose shot boundary detection methods by using visual hybrid features

<span lang="EN-US">Shot boundary detection is the fundamental technique that plays an important role in a variety of video processing tasks such as summarization, retrieval, object tracking, and so on. This technique involves segmenting a video sequence into shots, each of which is a sequence of interrelated temporal frames. This paper introduces two methods, where the first is for detecting the cut shot boundary via employing visual hybrid features, while the second method is to compare between them. This enhances the effectiveness of the performance of detecting the shot by selecting the strongest features. The first method was performed by utilizing hybrid features, which included statistics histogram of hue-saturation-value color space and grey level co-occurrence matrix. The second method was performed by utilizing hybrid features that include discrete wavelet transform and grey level co-occurrence matrix. The frame size decreased. This process had the advantage of reducing the computation time. Also used local adaptive thresholds, which enhanced the method’s performance. The tested videos were obtained from the BBC archive, which included BBC Learning English and BBC News. Experimental results have indicated that the second method has achieved (97.618%) accuracy performance, which was higher than the first and other methods using evaluation metrics.</span>

Efficient Shot Boundary Detection with Multiple Visual Representations

Due to the unlimited growth of video-capturing devices and media, searching and finding a particular video in this huge database becomes a laborious as well as expensive task. Information-rich shots are the inevitable factor of the content-based video processing (CBVP) system. Hence, shot boundary detection (SBD) becomes the basic step of all content-based video retrieval processes. The accuracy of the existing SBD methods highly suffers from false positives and false negatives due to the presence of multiple variants. An efficient SBD method with multiple invariant features is proposed in this paper. A right combination of invariant features such as edge change ratio (ECR), colour layout descriptor (CLD), and scale-invariant feature transform (SIFT) key point descriptors helped to improve the accuracy level of SBD. As the selected features are invariant to most of the variants in video frames, such as illuminance changes, motion, scaling, and rotation, a markable reduction in false detection is possible. Support vector machine (SVM) classifier is used for the classification of frames into transition frames and shot frames. This proposed method is experimented and analysed with the standard SBD dataset TRECVid 2007 videos. The experimental results are compared with some state-of-art methods, and our method shows better performance with a 97% of F1 score.

A novel method for video shot boundary detection using CNN-LSTM approach

A novel method for video shot boundary detection using CNN-LSTM approach

First person video summarization using different graph representations

First-person video summarization has emerged as an important research problem for computer vision and multimedia communities. In this paper, we show how different graph representations can be developed for accurately summarizing first-person (egocentric) videos in a computationally efficient manner. Each frame in a video is first represented as a weighted graph. A shot boundary detection method using graph based mutual information is developed. We next construct a weighted graph for each shot. A representative frame from each shot is selected using a graph centrality measure. A new way of characterizing egocentric video frames using a graph based center-surround model is shown next. Here, each representative frame is modeled as a union of a center region (graph) and a surround region (graph). By exploiting spectral measures of dissimilarity between the two (center and surround) graphs, optimal center and surround regions are determined. Optimal regions for all frames within a shot are kept the same as that of the representative frame. Center-surround differences in entropy and optical flow values along with PHOG (Pyramidal HOG) features are extracted from each frame. All frames in a video are finally represented by another weighted graph, termed as a Video Similarity Graph (VSG). The frames are clustered by applying a Minimum Spanning Tree (MST) based approach with a new measure for inadmissible edges. Frames closest to the centroid of each cluster are captured to build the summary. Experimental evaluation on two benchmark datasets indicate the advantage of the proposed formulation.

Shot based keyframe extraction using edge-LBP approach

Advancement in technology has led to tremendous increase in the online video content that requires efficient and effective content based video analysis approaches. In this regard, efficient approach for abrupt Shot Boundary Detection (SBD) and keyframe extraction has been presented. The proposed method detects abrupt shots by extracting binarized edge information from frames for texture characterisation using Local Binary Pattern (LBP) method. Further, Euclidean distance has been applied on the histogram features constructed and an adaptive threshold is used to detect abrupt shots. During keyframe extraction phase, magnitude gradient using Sobel operator has been extracted from each frame of the segmented shot. Subsequently, magnitude values are transformed into Z-score which describes the position of each pixelin terms of its distance from themean, when measured in standard deviation units of every frame. Finally, Co-efficient of variation is computed for each frame and the frame possessing the highest value is selected as a keyframe from every shot. Experiments were conducted on TRECVID 2001 dataset to analyze and validate the proposed approach. Experimental result manifest that the proposed SBD and keyframe extraction method outperforms some of the state-of-the-art algorithms with average F1-score of 98.15% and average fidelity measure of 90% respectively.

Video shot boundary detection based on multi-level features collaboration

Video shot boundary detection (SBD) is a basic work of content-based video retrieval and analysis. Various SBD methods have been proposed; however, there exist limitations in the complexity of boundary detection process. In this paper, a simple yet efficient SBD method is proposed, and the aim here is to speed up the boundary detection and simplify the detection process without loss of detection recall and accuracy. In our proposed model, we mainly use a top-down zoom rule, the image color feature, and local descriptors and combine a kind of motion area extraction algorithm to achieve shot boundary detection. Firstly, we select candidate transition segments via color histogram and the speeded-up robust features. Then, we perform cut transition detection through uneven slice matching, pixel difference, and color histogram. Finally, we perform gradual transition detection by the motion area extraction, scale-invariant feature transform, and even slice matching. The experiment is evaluated on the TRECVid2001 and the TRECVid2007 video datasets, and the experimental results show that our proposed method improves the recall, accuracy, and the detection speed, compared with some other related SBD methods.

Key Feature Extraction for Video Shot Boundary Detection using CNN

Now days as the progress of digital image technology, video files raise fast, there is a great demand for automatic video semantic study in many scenes, such as video semantic understanding, content-based analysis, video retrieval. Shot boundary detection is an elementary step for video analysis. However, recent methods are time consuming and perform badly in the gradual transition detection. In this paper we have projected a novel approach for video shot boundary detection using CNN which is based on feature extraction. We designed couple of steps to implement this method for automatic video shot boundary detection (VSBD). Primarily features are extracted using H, V&S parameters based on mean log difference along with implementation of histogram distribution function. This feature is given as an input to CNN algorithm which detects shots which is based on probability function. CNN is implemented using convolution and rectifier linear unit activation matrix which is followed after filter application and zero padding. After downsizing the matrix it is given as a input to fully connected layer which indicates shot boundaries comparing the proposed method with CNN method based on GPU the results are encouraging with substantially high values of precision Recall & F1 measures. CNN methods perform moderately better for animated videos while it excels for complex video which is observed in the results.

A Proposed Method for Video's Shot Boundary Detection based on Multiple Features

A Proposed Method for Video's Shot Boundary Detection based on Multiple Features

An Efficient and Robust Temporal Video Segmentation

Temporal video segmentation is the primary step of content based video retrieval. The whole processes of video management are coming under the focus of content based video retrieval, which includes, video indexing, video retrieval, and video summarization etc. In this paper, we proposed a computationally efficient and discriminating shot boundary detection method, which uses a local feature descriptor named local Contrast and Ordering (LCO) for feature extraction. The results of the experiments, which are conducted on the video dataset TRECVid, analyzed and compared with some existing shot boundary detection methods. The proposed method has given a promising result, even in the cases of illumination changes, rotated images etc.

Shot boundary detection based on block-wise principal component analysis

With the rapid development of digital video, shot boundary detection (SBD) has attracted much attention since it is the fundamental preprocessing for video indexing, annotation, retrieval, and other content-based operations. However, most state-of-the-art SBD methods are based on the spatial features of video image, and the overall characteristics of video shots are not fully considered. We propose a feature extraction method based on shot characteristics and a more robust SBD process. First, a video is divided into several segments, the segments containing consecutive video frames inside a shot are considered as training segments and others are called candidate segments. Afterward, using block-wise principal component analysis on the training segments, shot eigenspaces are established. The video frames in the candidate segment are then projected onto the corresponding shot eigenspace to extract the feature vectors. Finally, analysis and pattern matching for feature vectors are performed to extract the video shot boundary. Experiments on TRECVID test data demonstrate that the mean values of F1 in cut transition detection and gradual transition (GT) detection of our method are 0.901 and 0.866, respectively, obviously higher than the values of the compared methods, especially in GT detection, thus providing better accuracy in SBD.

A novel shot boundary detection system using hybrid optimization technique

This paper proposes a novel shot boundary detection method which combines the Advantages of Particle Swarm Optimization (PSO) and Gravitational Search Algorithm (GSA) to optimize the weights of the Feed-Forward Neural Network (FNN). To increase the performance of the system, the output of the hybrid technique is again analyzed by forming a Continuity matrix (ϕ). Then an Outlier along with a Continuity matrix is used for extracting a possible set of transition frames. A set of thresholds δ1 and δ2 is selected for classifying abrupt and gradual transitions from the available set of possible transition frames. Experimental results using TRECVid 2001 depicts that PSOGSA outperforms GSA and PSO in-terms of training the feed forward neural network and generating a higher overall F1 score. The proposed system also gives better performance when compared with other latest techniques in-terms of F1 score.

Two Stage Shot Boundary Detection via Feature Fusion and Spatial-Temporal Convolutional Neural Networks

Shot boundary detection is essentially to detect the position of frames where the shot changes. It has been actively studied in video analysis and management for convenience, which becomes a key technique with the rapid proliferation of rich and diverse videos. With respect to the complex characteristics of different shots in varying length and content variation property, in this paper we present a two stage method for shot boundary detection (TSSBD) which distinguishes abrupt shot by fusing color histogram and deep features, and locate gradual shot changes with C3D-based deep analysis. Abrupt shot changes are detected firstly as it occurs between two frames, which divides the complete video into segments containing gradual transitions; Over these video segments, gradual shot change detection is implemented using 3D-convolutional neural network, which classifies clips into specific gradual shot change types; Finally, an effective merging strategy is proposed to locate positions of gradual shot transitions. The experimental analysis illustrates that the proposed progressive method is capable of detecting both abrupt shot transitions and gradual shot transitions accurately.

ERP/MMR Algorithm for Classifying Topic‐Relevant and Topic‐Irrelevant Visual Shots of Documentary Videos

We propose and evaluate a video summarization method based on a topic relevance model, a maximal marginal relevance (MMR), and discriminant analysis to generate a semantically meaningful video skim. The topic relevance model uses event‐related potential (ERP) components to describe the process of topic relevance judgment. More specifically, the topic relevance model indicates that N400 and P600, which have been successfully applied to the mismatch process of a stimulus and the discourse‐internal reorganization and integration process of a stimulus, respectively, are used for the topic mismatch process of a topic‐irrelevant video shot and the topic formation process of a topic‐relevant video shot. To evaluate our proposed ERP/MMR‐based method, we compared the video skims generated by the ERP/MMR‐based, ERP‐based, and shot boundary detection (SBD) methods with ground truth skims. The results showed that at a significance level of 0.05, the ROUGE‐1 scores of the ERP/MMR method are statistically higher than those of the SBD method, and the diversity scores of the ERP/MMR method are statistically higher than those of the ERP method. This study suggested that the proposed method may be applied to the construction of a video skim without operational intervention, such as the insertion of a black screen between video shots.

Video Searching and Fingerprint Detection by Using the Image Query and PlaceNet-Based Shot Boundary Detection Method

This work presents a novel shot boundary detection (SBD) method based on the Place-centric deep network (PlaceNet), with the aim of using video shots and image queries for video searching (VS) and fingerprint detection. The SBD method has three stages. In the first stage, we employed Local Binary Pattern-Singular Value Decomposition (LBP-SVD) features for candidate shot boundaries selection. In the second stage, we used the PlaceNet to select the shot boundary by semantic labels. In the third stage, we used the Scale-Invariant Feature Transform (SIFT) descriptor to eliminate falsely detected boundaries. The experimental results show that our SBD method is effective on a series of SBD datasets. In addition, video searching experiments are conducted by using one query image instead of video sequences. The results under several image transitions by using shot fingerprints have shown good precision.

Detection and classification of wipe transitions in sport videos in presence of object motion

Transition detection is the necessary step in retrieval and investigation of videos on the basis of contents. Since last two decades, most of the researchers are involved in developing algorithms for detection of gradual transitions. However, the features of all gradual transitions are different and hence this issue still needs to be addressed precisely. After identifying this issue, an integrated shot boundary detection method is proposed in this paper. Wipe transition is the effect which is mostly used in the video making industries .Since there are different types of wipe transitions, it becomes very difficult to detect such transitions. Due to complexity in detection of wipes due to noise, object and camera motion earlier methods have less focus on this issue. In this paper an efficient wipe detection method is presented which gives better results even in the presence of object and camera movements.

Effective Video Shot Boundary Detection and Keyframe Selection using Soft Computing Techniques

The amount of video data generated and made publicly available has been tremendously increased in today's digital era. Analyzing these huge video repositories require effective and efficient content-based video analysis systems. Shot boundary detection and Keyframe extraction are the two major tasks in video analysis. In this direction, a method for detecting abrupt shot boundaries and extracting representative keyframe from each video shot is proposed. These objectives are achieved by incorporating the concepts of fuzzy sets and intuitionistic fuzzy sets. Shot boundaries are detected using coefficient of correlation on fuzzified frames. Further, probabilistic entropy measures are computed to extract the keyframe within fuzzified frames of a shot. The keyframe representative of a shot is the frame with highest entropy value. To show the efficacy of the proposed methods two benchmark datasets are used (TRECVID and Open Video Project). The proposed methods outperform when compared with some of state-of-the-art shot boundary detection and keyframe extraction methods.

Automatic Detection of Video Shot Boundary in Social Media Using a Hybrid Approach of HLFPN and Keypoint Matching

Shot boundary detection (SBD) is an important and fundamental step in video content analysis such as content-based video indexing, browsing, and retrieval. In this paper, a hybrid SBD method is presented by integrating a high-level fuzzy Petri net (HLFPN) model with keypoint matching. The HLFPN model with histogram difference is executed as a predetection. Next, the speeded-up robust features (SURF) algorithm that is reliably robust to image affine transformation and illumination variation is used to figure out all possible false shots and the gradual transition based on the assumption from the HLFPN model. The top-down design can effectively lower down the computational complexity of SURF algorithm. The proposed approach has increased the precision of SBD and can be applied in different types of videos.

Video retrieval for shot cluster and classification based on key feature set

ABSTRACTThe proposed content-based video retrieval method in this study uses feature extraction, shot boundary detection, shot noise filter, shot cluster, the key feature set of shot extraction and shot classification. Shot boundary detection is conducted using a weight selection genetic algorithm. In order to verify the video retrieval method in this paper, three video databases were used in the experiment. These video databases are used to verify the shot boundary detection, the shot classification and video retrieval of this study. A series of comparisons and analyses were then conducted using the above video databases. The experiment results showed that the shot boundary detection method exhibits better performance than the other compared methods. Moreover, the video retrieval experiment results showed that the proposed method is able to precisely catch a shot in the search video, and do so with reduced video retrieval time.

Shot boundary detection via adaptive low rank and svd-updating

Usually considered as the first step in content-based video retrieval, shot boundary detection (SBD) is crucial to subsequent high-level applications like video summarization. The paper proposes an accurate video shot boundary detection method based on singular value decomposition (SVD), through the use of different mathematical theorems and interpretations. In our contribution, adaptive feature extraction is performed using the Frobenius norm of low rank approximation matrices. Each frame will be mapped into k˜-dimensional vector in the singular space according to each segment. The classification of continuity values is then achieved via double thresholding technique for detecting the hard cuts. For gradual transitions detection, the folding-in technique, known as SVD-updating, is used for the first time in video shot boundary detection. This allows for accurate detection in reduced computational time, as it is no longer necessary to recalculate the whole SVD decomposition for segment correction. Experimental results on four different datasets, including the TRECVid 2001, 2002 and 2005 SBD tasks, show the effectiveness of our detector, which outperforms recent related methods in detecting both hard and gradual transitions.