Shot Boundary Detection Research Articles

An Efficient and Robust Temporal Video Segmentation

Temporal video segmentation is the primary step of content based video retrieval. The whole processes of video management are coming under the focus of content based video retrieval, which includes, video indexing, video retrieval, and video summarization etc. In this paper, we proposed a computationally efficient and discriminating shot boundary detection method, which uses a local feature descriptor named local Contrast and Ordering (LCO) for feature extraction. The results of the experiments, which are conducted on the video dataset TRECVid, analyzed and compared with some existing shot boundary detection methods. The proposed method has given a promising result, even in the cases of illumination changes, rotated images etc.

A Scheme for Shot Detection and Video Retreival using Spatio Temporal Features

There has been a revolution in multimedia with technological advancement. Hence, Video recording has increased in leaps and bounds. Video retrieval from a huge database is cumbersome by the existing text based search since a lot of human effort is involved and the retrieval efficiency is meager as well. In view of the present challenges, video retrieval based on video content prevails over the existing conventional methods. Content implies real video information such as video features. The performance of the Content Based Video Retrieval (CBVR) depends on Feature extraction and similar features matching. Since the selection of features in the existing algorithms is not effective, the retrieval processing time is more and the efficiency is less. Combined features of color and motion have been proposed for feature extraction and Spatio-Temporal Scale Invariant Feature Transform is used for Shot Boundary Detection. Since the characteristic of color feature is visual video content and that of motion feature is temporal content, these two features are significant in effective video retrieval. The performance of the CBVR system has been evaluated on the TRECVID dataset and the retrieved videos reveal the effectiveness of proposed algorithm.

Novel Shot Boundary Detection in News Streams Based on Fuzzy Petri Nets

ABSTRACTWith the advent of a digital era, people have encountered some difficulty in using and absorbing overwhelming information generated by technological advances in multimedia. Thus, the development of video summarization enables people to catch a general idea about videos in a short time. In this paper, we focus on the shot change, a part of the video summarization, to conduct an experimental sample on news programs. Moreover, a high-level fuzzy Petri net model is presented to describe the frame combination which indicates a shot boundary used for a video frame sequence in order to detect both cut transitions and gradual transitions. This study has used feature functions to estimate the direct shot change in consideration of video shot boundary detection which adopts the HLFPN model to find a threshold value. The experimental results manifest that the proposed system saves a lot of time and reduces the occurrence of improper shot changes caused by the motions of objects and cameras when comparing the proposed approach with other existing ones.

A novel video shot boundary detection framework employing DCT and pattern matching

The video Shot Boundary Detection (SBD) is an elementary step in realising a system capability to perform content based video search, structural analysis, data retrieval and video summation. Myriad research works in the past have been reported to construct SBD algorithms. However, the need of an error-free, meticulous and cost-effective SBD technique still persists; for applications viz. apt management, storage, browsing, video indexing and retrieval of multimedia data. This paper is an effort in the same direction with the aim of achieving high execution speed and greater accuracy. The proposed SBD technique in this paper incorporates three steps: (i) Candidate Segment Selection (ii) Cut Transition detection (iii) Gradual Transition detection. This paper adopts pixel based technique with candidate segment selection to speed up the SBD. For Cut Transition detection, the proposed method employs Discrete Cosine Transform (DCT) and for Gradual Transition detection, it employs Image Histogram and Pattern Matching. The comparison of MATLAB simulation results of the proposed SBD technique with those in literature manifest better results in terms of execution speed and accuracy.

Improved Quality Keyframe Selection Method for HD Video

With the widespread use of the Internet, services for providing large-capacity multimedia data such as video-on-demand (VOD) services and video uploading sites have greatly increased. VOD service providers want to be able to provide users with high-quality keyframes of high quality videos within a few minutes after the broadcast ends. However, existing keyframe extraction tends to select keyframes whose quality as a keyframe is insufficiently considered, and it takes a long computation time because it does not consider an HD class image. In this paper, we propose a keyframe selection method that flexibly applies multiple keyframe quality metrics and improves the computation time. The main procedure is as follows. After shot boundary detection is performed, the first frames are extracted as initial keyframes. The user sets evaluation metrics and priorities by considering the genre and attributes of the video. According to the evaluation metrics and the priority, the low-quality keyframe is selected as a replacement target. The replacement target keyframe is replaced with a high-quality frame in the shot. The proposed method was subjectively evaluated by 23 votes. Approximately 45% of the replaced keyframes were improved and about 18% of the replaced keyframes were adversely affected. Also, it took about 10 minutes to complete the summary of one hour video, which resulted in a reduction of more than 44.5% of the execution time.

A secure video watermarking technique based on hyperchaotic Lorentz system

The advancement of multimedia and computer technology has facilitated increasing number of research on digital video watermarking. Video watermarking techniques can be used not only in the protection of multimedia video works’ copyright, but also in the transmission of confidential data. This paper presents a color video watermarking algorithm based on hyperchaotic Lorentz system. Firstly, the color watermark images are scrambled using hyperchaotic Lorentz system to enhance its confidentiality. Secondly, we use shot boundary detection to extract non-motion frames of the video. Then the chaotic sequence is used to determine the specific frames among the non-motion frames. Next, we apply the discrete wavelet transform to specific frames to get the appropriate subbands. And finally the encrypted watermarks are embedded into the selected subbands. The performance of proposed method is evaluated by Peak Signal-to-Noise Ratio, Normalized Correlation and Structural Similarity Index Measure. Experiments showed that the average PSNR and SSIM of watermarked frames are 57.00 dB and 0.99, respectively, which indicate that the proposed method has high imperceptibility. The NC value of 1.00 proves that the watermark can be transmitted without loss under no attacks. And we also tested the robustness and imperceptibility in the presence of various attacks including image processing attacks, geometrical attacks and video attacks. The method we proposed enrich the digital watermarking techniques.

Shot boundary detection based on block-wise principal component analysis

With the rapid development of digital video, shot boundary detection (SBD) has attracted much attention since it is the fundamental preprocessing for video indexing, annotation, retrieval, and other content-based operations. However, most state-of-the-art SBD methods are based on the spatial features of video image, and the overall characteristics of video shots are not fully considered. We propose a feature extraction method based on shot characteristics and a more robust SBD process. First, a video is divided into several segments, the segments containing consecutive video frames inside a shot are considered as training segments and others are called candidate segments. Afterward, using block-wise principal component analysis on the training segments, shot eigenspaces are established. The video frames in the candidate segment are then projected onto the corresponding shot eigenspace to extract the feature vectors. Finally, analysis and pattern matching for feature vectors are performed to extract the video shot boundary. Experiments on TRECVID test data demonstrate that the mean values of F1 in cut transition detection and gradual transition (GT) detection of our method are 0.901 and 0.866, respectively, obviously higher than the values of the compared methods, especially in GT detection, thus providing better accuracy in SBD.

A Proposed Algorithm for Shot Boundary Detection Using Various Features

A Proposed Algorithm for Shot Boundary Detection Using Various Features

A novel shot boundary detection system using hybrid optimization technique

This paper proposes a novel shot boundary detection method which combines the Advantages of Particle Swarm Optimization (PSO) and Gravitational Search Algorithm (GSA) to optimize the weights of the Feed-Forward Neural Network (FNN). To increase the performance of the system, the output of the hybrid technique is again analyzed by forming a Continuity matrix (ϕ). Then an Outlier along with a Continuity matrix is used for extracting a possible set of transition frames. A set of thresholds δ1 and δ2 is selected for classifying abrupt and gradual transitions from the available set of possible transition frames. Experimental results using TRECVid 2001 depicts that PSOGSA outperforms GSA and PSO in-terms of training the feed forward neural network and generating a higher overall F1 score. The proposed system also gives better performance when compared with other latest techniques in-terms of F1 score.

Shot boundary detection based on orthogonal polynomial

Shot boundary detection (SBD) is a substantial step in video content analysis, indexing, retrieval, and summarization. SBD is the process of automatically partitioning video into its basic units, known as shots, through detecting transitions between shots. The design of SBD algorithms developed from simple feature comparison to rigorous probabilistic and using of complex models. Nevertheless, accelerate the detection of transitions with higher accuracy need to be improved. Extensive research has employed orthogonal polynomial (OP) and their moments in computer vision and signal processing owing to their powerful performance in analyzing signals. A new SBD algorithm based on OP has been proposed in this paper. The Features are derived from orthogonal transform domain (moments) to detect the hard transitions in video sequences. Moments are used because of their ability to represent signal (video frame) without information redundancy. These features are the moments of smoothed and gradients of video frames. The moments are computed using a developed OP which is squared Krawtchouk-Tchebichef polynomial. These moments (smoothed and gradients) are fused to form a feature vector. Finally, the support vector machine is utilized to detect hard transitions. In addition, a comparison between the proposed algorithm and other state-of-the-art algorithms is performed to reinforce the capability of the proposed work. The proposed algorithm is examined using three well-known datasets which are TRECVID2005, TRECVID2006, and TRECVID2007. The outcomes of the comparative analysis show the superior performance of the proposed algorithm against other existing algorithms.

Two Stage Shot Boundary Detection via Feature Fusion and Spatial-Temporal Convolutional Neural Networks

Shot boundary detection is essentially to detect the position of frames where the shot changes. It has been actively studied in video analysis and management for convenience, which becomes a key technique with the rapid proliferation of rich and diverse videos. With respect to the complex characteristics of different shots in varying length and content variation property, in this paper we present a two stage method for shot boundary detection (TSSBD) which distinguishes abrupt shot by fusing color histogram and deep features, and locate gradual shot changes with C3D-based deep analysis. Abrupt shot changes are detected firstly as it occurs between two frames, which divides the complete video into segments containing gradual transitions; Over these video segments, gradual shot change detection is implemented using 3D-convolutional neural network, which classifies clips into specific gradual shot change types; Finally, an effective merging strategy is proposed to locate positions of gradual shot transitions. The experimental analysis illustrates that the proposed progressive method is capable of detecting both abrupt shot transitions and gradual shot transitions accurately.

Enhanced Local Directional Pattern based video shot boundary detection and automatic synchronization for STB quality inspection

Enhanced Local Directional Pattern based video shot boundary detection and automatic synchronization for STB quality inspection

ERP/MMR Algorithm for Classifying Topic‐Relevant and Topic‐Irrelevant Visual Shots of Documentary Videos

We propose and evaluate a video summarization method based on a topic relevance model, a maximal marginal relevance (MMR), and discriminant analysis to generate a semantically meaningful video skim. The topic relevance model uses event‐related potential (ERP) components to describe the process of topic relevance judgment. More specifically, the topic relevance model indicates that N400 and P600, which have been successfully applied to the mismatch process of a stimulus and the discourse‐internal reorganization and integration process of a stimulus, respectively, are used for the topic mismatch process of a topic‐irrelevant video shot and the topic formation process of a topic‐relevant video shot. To evaluate our proposed ERP/MMR‐based method, we compared the video skims generated by the ERP/MMR‐based, ERP‐based, and shot boundary detection (SBD) methods with ground truth skims. The results showed that at a significance level of 0.05, the ROUGE‐1 scores of the ERP/MMR method are statistically higher than those of the SBD method, and the diversity scores of the ERP/MMR method are statistically higher than those of the ERP method. This study suggested that the proposed method may be applied to the construction of a video skim without operational intervention, such as the insertion of a black screen between video shots.

A semantic-based video scene segmentation using a deep neural network

Video scene segmentation is very important research in the field of computer vision, because it helps in efficient storage, indexing and retrieval of videos. Achieving this kind of scene segmentation cannot be done by just calculating the similarity of low-level features presented in the video; high-level features should also be considered to achieve a better performance. Even though much research has been conducted on video scene segmentation, most of these studies failed to semantically segment a video into scenes. Thus, in this study, we propose a Deep-learning Semantic-based Scene-segmentation model (called DeepSSS) that considers image captioning to segment a video into scenes semantically. First, the DeepSSS performs shot boundary detection by comparing colour histograms and then employs maximum-entropy-applied keyframe extraction. Second, for semantic analysis, using image captioning that benefits from deep learning generates a semantic text description of the keyframes. Finally, by comparing and analysing the generated texts, it assembles the keyframes into a scene grouped under a semantic narrative. That said, DeepSSS considers both low- and high-level features of videos to achieve a more meaningful scene segmentation. By applying DeepSSS to data sets from MS COCO for caption generation and evaluating its semantic scene-segmentation task results with the data sets from TRECVid 2016, we demonstrate quantitatively that DeepSSS outperforms other existing scene-segmentation methods using shot boundary detection and keyframes. What’s more, the experiments were done by comparing scenes segmented by humans and scene segmented by the DeepSSS. The results verified that the DeepSSS’ segmentation resembled that of humans. This is a new kind of result that was enabled by semantic analysis, which was impossible by just using low-level features of videos.

A vague set approach for identifying shot transition in videos using multiple feature amalgamation

Shot boundary detection (SBD) is the preliminary and most significant step in Content Based Video Retrieval (CBVR). As such the effectiveness of a CBVR system depends heavily on reliable detection of shot boundaries. In this work, a simple yet effective technique for amalgamating several distance features extracted from video frames has been proposed. The aim here is to develop a technique which is able to produce a better distance feature from the existing ones by hybridizing several distance metrics. In the proposed model, any number of distance features can be incorporated and fused together. The resultant feature is not only more robust but also immune to features which are inefficient. Robustness of the proposed method is tested by combining several low performing features with the more efficient ones. Several statistical amalgamation functions are also tested for determining the most efficient one in terms of F1 score. The power of vague sets has been harnessed to detect the shot boundaries effectively using the resultant distance feature. The proposed method is proved to be effective by means of the results obtained, which show that multiple feature amalgamation can lead to a hybrid distance feature which performs better than the best feature incorporated for SBD. The proposed technique is analyzed using ANOVA. A comparison with the other existing methods portray the efficacy of the proposed approach. This method can also be applied for other research problems where several features are to be fused together for producing superior results than the ones obtained by individual methods.

Curvelet transform based feature extraction and selection for multimedia event classification

Multimedia event classification has been one of the major endeavors in video event analysis. For event identification, feature extraction plays a critical role, merely distinguishing the right features becomes a challenging job. So, in this paper, feature extraction and selection for video event detection are proposed. For video event detection or classification, identification of object structure and its motion are a basic needs. So, for object detection, curvelet features are considered, due to its high directional selectivity and high anisotropic properties. Next, an algorithm for Shot Boundary Detection (SBD) called Motion based SBD (MSBD) is proposed to identify the shot boundaries. Also, to make the users to access the queried event in less time, an algorithm for representing few representative frames containing whole video content is proposed. To make an event search efficient, object-based features from dominant features are extracted and to enhance the features much more efficient and dominant, feature selection is performed using the ranking method. Lastly, an SVM classifier with RBF kernel is used for event classification. The proposed work is experimented using Columbia Consumer Video (CCV) dataset and evaluated using mean Average Precision (mAP) and find that, it outperforms various other existing methods.

Generative Model Based Video Shot Boundary Detection for Automated Surveillance

Video shot boundary detection (SBD) or video cut detection is one of the fundamental processes of video-processing with respect to semantic understanding, contextual information accumulation, labeling, content-based information retrieval and many more applications, such as video surveillance and monitoring. In this work, the authors have proposed a generative-model based framework for detecting shot boundaries in between the frames of a video segment. To generate a model of shot-boundaries, the authors have applied the concepts of Support Vector Machine to estimate the distance between any two images, and then, have generated a Gaussian Mixture Model from the estimated distances. Next, a Bayesian Estimation process checks the presence of boundaries in between the images by exploiting the Gaussian Mixture-based boundary model. Further, the authors have used the principles of Compressive Sensing to reduce the overhead of boundary detection process without losing of important information.

Motion and illumination defiant cut detection based on Weber features

The spontaneous proliferation of video data necessitates implementing hierarchical structures for various content management applications. Temporal video segmentation is the key towards such management. To address the problem of temporal segmentation, the current communication exploits the concept of psychological behaviour of the human visual system. Towards this goal an abrupt cut detection scheme has been proposed based on Weber's law which provides a strong spatial correlation among the neighbouring pixels. Thus, the authors provide a robust and unique solution for abrupt shot boundary detection when the frames are affected partially or fully by flashlight, fire and flicker, high motion associated with an object or camera. Further, they have devised a model for generating an automatic threshold, taking into account the statistics of the feature vector which quadrates itself with the variation in the contents of the video. The effectiveness of the proposed framework is validated by exhaustive comparison with few contemporary and recent approaches by using benchmark datasets TRECVID 2001, TRECVID 2002, TRECVID 2007 and some publicly available videos. The results obtained give credence to the remarkable improvement in the performance while preserving a good trade-off between missed hits and false hits as compared to the state-of-the-art methods.

Gradual transition detection in shot boundary using gradual curve point

Video shot boundary detection (SBD) is a fundamental step in automatic video content analysis. Gradual transition is based on the gradual curves characterised by distribution of average edge information in the sequence of frames. An average edge frame is obtained from optimal edge detector. Features are extracted based on the minimum difference between the sequences of frames. The minimum variance in the local frame sequence is chosen as a gradual curve point. Finally, the performance is computed based on the precision (quality) and recall (quantity) value.

Video Searching and Fingerprint Detection by Using the Image Query and PlaceNet-Based Shot Boundary Detection Method

This work presents a novel shot boundary detection (SBD) method based on the Place-centric deep network (PlaceNet), with the aim of using video shots and image queries for video searching (VS) and fingerprint detection. The SBD method has three stages. In the first stage, we employed Local Binary Pattern-Singular Value Decomposition (LBP-SVD) features for candidate shot boundaries selection. In the second stage, we used the PlaceNet to select the shot boundary by semantic labels. In the third stage, we used the Scale-Invariant Feature Transform (SIFT) descriptor to eliminate falsely detected boundaries. The experimental results show that our SBD method is effective on a series of SBD datasets. In addition, video searching experiments are conducted by using one query image instead of video sequences. The results under several image transitions by using shot fingerprints have shown good precision.