Shot Boundary Research Articles

Robust video content analysis schemes for human action recognition.

Action recognition is a challenging time series classification task that has received much attention in the recent past due to its importance in critical applications, such as surveillance, visual behavior study, topic discovery, security, and content retrieval. The main objective of the research is to develop a robust and high-performance human action recognition techniques. A combination of local and holistic feature extraction methods used through analyzing the most effective features to extract to reach the objective, followed by using simple and high-performance machine learning algorithms. This paper presents three robust action recognition techniques based on a series of image analysis methods to detect activities in different scenes. The general scheme architecture consists of shot boundary detection, shot frame rate re-sampling, and compact feature vector extraction. This process is achieved by emphasizing variations and extracting strong patterns in feature vectors before classification. The proposed schemes are tested on datasets with cluttered backgrounds, low- or high-resolution videos, different viewpoints, and different camera motion conditions, namely, the Hollywood-2, KTH, UCF11 (YouTube actions), and Weizmann datasets. The proposed schemes resulted in highly accurate video analysis results compared to those of other works based on four widely used datasets. The First, Second, and Third Schemes provides recognition accuracies of 57.8%, 73.6%, and 52.0% on Hollywood2, 94.5%, 97.0%, and 59.3% on KTH, 94.5%, 95.6%, and 94.2% on UCF11, and 98.9%, 97.8% and 100% on Weizmann. Each of the proposed schemes provides high recognition accuracy compared to other state-of-art methods. Especially, the Second Scheme as it gives excellent comparable results to other benchmarked approaches.

First person video summarization using different graph representations

First-person video summarization has emerged as an important research problem for computer vision and multimedia communities. In this paper, we show how different graph representations can be developed for accurately summarizing first-person (egocentric) videos in a computationally efficient manner. Each frame in a video is first represented as a weighted graph. A shot boundary detection method using graph based mutual information is developed. We next construct a weighted graph for each shot. A representative frame from each shot is selected using a graph centrality measure. A new way of characterizing egocentric video frames using a graph based center-surround model is shown next. Here, each representative frame is modeled as a union of a center region (graph) and a surround region (graph). By exploiting spectral measures of dissimilarity between the two (center and surround) graphs, optimal center and surround regions are determined. Optimal regions for all frames within a shot are kept the same as that of the representative frame. Center-surround differences in entropy and optical flow values along with PHOG (Pyramidal HOG) features are extracted from each frame. All frames in a video are finally represented by another weighted graph, termed as a Video Similarity Graph (VSG). The frames are clustered by applying a Minimum Spanning Tree (MST) based approach with a new measure for inadmissible edges. Frames closest to the centroid of each cluster are captured to build the summary. Experimental evaluation on two benchmark datasets indicate the advantage of the proposed formulation.

A novel bifold-stage shot boundary detection algorithm: invariant to motion and illumination

Shot boundary detection is mainly considered as a stepping stone in the broad arena of content-based video retrieval. Ample systematic investigation has been carried out in the terrain of shot boundary detection. The attainment of shot boundary detection procedures is greatly hindered due to the presence of unforeseen illumination change and motion effects in a video. This work proposes a novel bifold-stage technique to recognize abrupt transition in videos, invariant to motion, and illumination effects. In the first stage, the local ternary patterns feature is used to extract information from each frame in a video. Then, a set of novel adaptive thresholds such as $${\gamma }$$ and $${{{\beta }}}$$ are used to find the possible transition frames. In the confirmation stage, Lab color difference along with an adaptive threshold $${\delta }$$ is used to extract actual transition frames. The experimental result depicts that the motion effect is also scaled down in the initial stage. The Lab color difference passed down in the second stage also handles the illumination and motion effects which are not managed in the initial stage. Experimentation is done using TRECVid 2001 and 2007 standard datasets and palpable that the proposed technique outperforms most of the contemporary shot boundary detection approaches.

Intelligent Sports Video Classification Based on Deep Neural Network (DNN) Algorithm and Transfer Learning.

Traditional text annotation-based video retrieval is done by manually labeling videos with text, which is inefficient and highly subjective and generally cannot accurately describe the meaning of videos. Traditional content-based video retrieval uses convolutional neural networks to extract the underlying feature information of images to build indexes and achieves similarity retrieval of video feature vectors according to certain similarity measure algorithms. In this paper, by studying the characteristics of sports videos, we propose the histogram difference method based on using transfer learning and the four-step method based on block matching for mutation detection and fading detection of video shots, respectively. By adaptive thresholding, regions with large frame difference changes are marked as candidate regions for shots, and then the shot boundaries are determined by mutation detection algorithm. Combined with the characteristics of sports video, this paper proposes a key frame extraction method based on clustering and optical flow analysis, and experimental comparison with the traditional clustering method. In addition, this paper proposes a key frame extraction algorithm based on clustering and optical flow analysis for key frame extraction of sports video. The algorithm effectively removes the redundant frames, and the extracted key frames are more representative. Through extensive experiments, the keyword fuzzy finding algorithm based on improved deep neural network and ontology semantic expansion proposed in this paper shows a more desirable retrieval performance, and it is feasible to use this method for video underlying feature extraction, annotation, and keyword finding, and one of the outstanding features of the algorithm is that it can quickly and effectively retrieve the desired video in a large number of Internet video resources, reducing the false detection rate and leakage rate while improving the fidelity, which basically meets people's daily needs.

Fast Shot Boundary Detection Based on Separable Moments and Support Vector Machine

The large number of visual applications in multimedia sharing websites and social networks contribute to the increasing amounts of multimedia data in cyberspace. Video data is a rich source of information and considered the most demanding in terms of storage space. With the huge development of digital video production, video management becomes a challenging task. Video content analysis (VCA) aims to provide big data solutions by automating the video management. To this end, shot boundary detection (SBD) is considered an essential step in VCA. It aims to partition the video sequence into shots by detecting shot transitions. High computational cost in transition detection is considered a bottleneck for real-time applications. Thus, in this paper, a balance between detection accuracy and speed for SBD is addressed by presenting a new method for fast video processing. The proposed SBD framework is based on the concept of candidate segment selection with frame active area and separable moments. First, for each frame, the active area is selected such that only the informative content is considered. This leads to a reduction in the computational cost and disturbance factors. Second, for each active area, the moments are computed using orthogonal polynomials. Then, an adaptive threshold and inequality criteria are used to eliminate most of the non-transition frames and preserve candidate segments. For further elimination, two rounds of bisection comparisons are applied. As a result, the computational cost is reduced in the subsequent stages. Finally, machine learning statistics based on the support vector machine is implemented to detect the cut transitions. The enhancement of the proposed fast video processing method over existing methods in terms of computational complexity and accuracy is verified. The average improvements in terms of frame percentage and transition accuracy percentage are 1.63% and 2.05%, respectively. Moreover, for the proposed SBD algorithm, a comparative study is performed with state-of-the-art algorithms. The comparison results confirm the superiority of the proposed algorithm in computation time with improvement of over 38%.

Video Index Point Detection and Extraction Framework Using Custom YoloV4 Darknet Object Detection Model

The trend of learning from videos instead of documents has increased. There could be hundreds and thousands of videos on a single topic, with varying degrees of context, content, and depth of the topic. The literature claims that learners are nowadays less interested in viewing a complete video but prefers the topic of their interests. This develops the need for indexing of video lectures. Manual annotation or topic-wise indexing is not new in the case of videos. However, manual indexing is time-consuming due to the length of a typical video lecture and intricate storylines. Automatic indexing and annotation are, therefore, better and efficient solutions. This research aims to identify the need for automatic video indexing for better information retrieval and ease users navigating topics inside a video. The automatically identified topics are referred to as “Index Points.” 137-layer YoloV4 Darknet Neural Network creates a custom object detector model. The model is trained on approximately 6000 video frames and then tested on a suite of 50 videos of around 20 hours of run time. Shot Boundary detection is performed using Structural Similarity fused with a Binary Search Pattern algorithm which outperformed the state-of-the-art SSIM technique by reducing the processing time to approximately 21% and providing around 96% accuracy. Generation of accurate index points in terms of true positives and false negatives is detected through precision, recall, and F1 score, which varies between 60–80% for every video. The results show that the proposed algorithm successfully generates a digital index with reasonable accuracy in topic detection.

Multiview video summarization using video partitioning and clustering

Multiview video summarization plays a crucial role in abstracting essential information form multiple videos of the same location and time. In this paper, we propose a new approach for the multiview summarization. The proposed approach uses the BIRCH clustering algorithm for the first time on the initial set of frames to get rid of the static and redundant. The work presents a new approach for shot boundary detection using frame similarity measures Jaccard and Dice. The algorithm performs effectively synchronized merging of keyframes from all camera-views to obtain the final summary. Extensive experimentation conducted on various datasets suggests that the proposed approach significantly outperforms most of the existing video summarization approaches. To state a few, a 1.5% improvement on video length reduction, 24.28% improvement in compression ratio, and 6.4% improvement in quality assessment ratio is observed on the lobby dataset.

Shot based keyframe extraction using edge-LBP approach

Advancement in technology has led to tremendous increase in the online video content that requires efficient and effective content based video analysis approaches. In this regard, efficient approach for abrupt Shot Boundary Detection (SBD) and keyframe extraction has been presented. The proposed method detects abrupt shots by extracting binarized edge information from frames for texture characterisation using Local Binary Pattern (LBP) method. Further, Euclidean distance has been applied on the histogram features constructed and an adaptive threshold is used to detect abrupt shots. During keyframe extraction phase, magnitude gradient using Sobel operator has been extracted from each frame of the segmented shot. Subsequently, magnitude values are transformed into Z-score which describes the position of each pixelin terms of its distance from themean, when measured in standard deviation units of every frame. Finally, Co-efficient of variation is computed for each frame and the frame possessing the highest value is selected as a keyframe from every shot. Experiments were conducted on TRECVID 2001 dataset to analyze and validate the proposed approach. Experimental result manifest that the proposed SBD and keyframe extraction method outperforms some of the state-of-the-art algorithms with average F1-score of 98.15% and average fidelity measure of 90% respectively.

Enhanced Video Classification System Using a Block-Based Motion Vector

The main objective of this work was to design and implement a support vector machine-based classification system to classify video data into predefined classes. Video data has to be structured and indexed for any video classification methodology. Video structure analysis involves shot boundary detection and keyframe extraction. Shot boundary detection is performed using a two-pass block-based adaptive threshold method. The seek spread strategy is used for keyframe extraction. In most of the video classification methods, selection of features is important. The selected features contribute to the efficiency of the classification system. It is very hard to find out which combination of features is most effective. Feature selection makes relevance to the proposed system. Herein, a support vector machine-based classifier was considered for the classification of video clips. The performance of the proposed system considered six categories of video clips: cartoons, commercials, cricket, football, tennis, and news. When shot level features and keyframe features, along with motion vectors, were used, 86% correct classification was achieved, which was comparable with the existing methods. The research concentrated on feature extraction where combination of selected features was given to a classifier to get the best classification performance.

An Efficient Shot Boundary Detection Using Data-cube Searching Technique

Background:: In the internet era, there is a prime need to access and manage the huge volume of multimedia data in an effective manner. Shot is a sequence of frames captured by a single camera in an uninterrupted space and time. Shot detection is suitable for various applications such that video browsing, video indexing, content based video retrieval and video summarization. Objective:: To detect the shot transitions in the video within a short duration. It compares the visual features of frames like correlation, histogram and texture features only in the candidate region frames instead of comparing the full frames in the video file. Methods: This paper analyses candidate frames by searching the values of frame features which matches with the abrupt detector followed by the correct cut transition frame with in the datacube recursively until it detects the correct transition frame. If they are matched with the gradual detector, then it will give the gradual transition ranges, otherwise the algorithm will compare the frames within the next datacube to detect shot transition. Results:: The total average detection rates of all transitions computed in the proposed Data-cube Search Based Shot Boundary Detection technique are 92.06 for precision, 96.92 for recall and 93.94 for f1 measure and the maximum accurate detection rate. Conclusion:: Proposed method for shot transitions uses correlation value for searching procedure with less computation time than the existing methods which compares every single frame and uses multi features such as color, edge, motion and texture features in wavelet domain.

Video shot boundary detection based on multi-level features collaboration

Video shot boundary detection (SBD) is a basic work of content-based video retrieval and analysis. Various SBD methods have been proposed; however, there exist limitations in the complexity of boundary detection process. In this paper, a simple yet efficient SBD method is proposed, and the aim here is to speed up the boundary detection and simplify the detection process without loss of detection recall and accuracy. In our proposed model, we mainly use a top-down zoom rule, the image color feature, and local descriptors and combine a kind of motion area extraction algorithm to achieve shot boundary detection. Firstly, we select candidate transition segments via color histogram and the speeded-up robust features. Then, we perform cut transition detection through uneven slice matching, pixel difference, and color histogram. Finally, we perform gradual transition detection by the motion area extraction, scale-invariant feature transform, and even slice matching. The experiment is evaluated on the TRECVid2001 and the TRECVid2007 video datasets, and the experimental results show that our proposed method improves the recall, accuracy, and the detection speed, compared with some other related SBD methods.

A Shot boundary Detection Technique based on Visual Colour Information

The paper proposes a novel video segmentation system with maiden application of CIEDE2000 colour-difference and mean luminace pattern. CIEDE2000 colour-difference uses Lab colour space which is a stable and efficient colour space. The main advantage of Lab colour space model is that it can approximate all the available colours perceived by our human eye. CIEDE2000 colour difference is used for detecting abrupt transitions in the video. The novel contribution of the paper is the maiden use of the mean luminance pattern, increasing and decreasing patterns of the mean value of frame luminance, for detecting the gradual transition. The approach is validated on standard databases TRECVid 2001 and 2007 test video database. The performance of the proposed technique is compared with recently reported techniques and found to be superior as compared to other techniques. The accuracy achieved with the proposed method on the standard databases is 95.9% for cut transition, 78.6% for gradual transition and 92.1% overall.

SBD-Duo: a dual stage shot boundary detection technique robust to motion and illumination effect

In this paper, we propose a novel shot boundary detection technique using gradient and colour information. The gradient similarity and luminance distortion are calculated to measure the contrast and structural changes of each frame including luminance changes. In the proposed system, the effects of the changes in luminance and contrast-structure are integrated via an adaptive method to extract the possible transitions using an adaptive threshold across the videos. In the verification part, CIEDE2000 colour-difference values of the possible transition frames are compared for declaration of abrupt and gradual transitions. Our system takes effectively less computational time to detect abrupt and gradual transition for a video as compared with contemporary solutions. Our proposed system also gives dominate the performance as compared with latest techniques in terms of F1 score using TRECVid 2001 and 2007 selected dataset. We have performed a series of rigorous experimentation to validate our claims.

Video shot boundary detection using block based cumulative approach

Video data is becoming an indispensable part of today’s Big Data due to evolution of social web and mobile technology. Content based video analysis has become crucial for video management. Shot boundary detection is one of the most essential task in video content analysis. In view of this, an efficient shot boundary detection approach to detect abrupt and gradual transition in videos is proposed in this work. The approach extracts block based Mean Cumulative Sum Histogram (MCSH) from each edge gradient fuzzified frame as a combination of local and global feature. The relative standard deviation (RSD) statistical measure is applied on the obtained MCSH to detect abrupt and gradual shots in the video. Efficacy of the proposed method is measured by conducting experiments on TRECVID 2001, TRECVID 2007 and VideoSeg datasets. The proposed method shows relatively a good performance when compared to some of the state-of-the-art shot boundary detection approaches.

Eagle Eye CBVR Based on Unique Key Frame Extraction and Deep Belief Neural Network

Great efforts have been paid to develop Content Based Video Retrieval (CBVR) due to the exponential growth of video datasets with several types of data such as visual, audio and metadata. This paper proposes Eagle Eye a new Content Based Video Retrieval Framework through applying new methods for the major three important parts of CBVR. A Subspace Clustering Algorithm is implemented with binary codes that are extracted from different videos. We introduce a Local Histogram based shot boundary detection algorithm to detect shot boundaries. Then Unique Key Frame Summarization algorithm with detected shots is applied. An integrated Local Binary Pattern, Coherence based filter and HOG algorithm is used for the Feature extraction and best features are selected and stored along with the video indices. The extracted features were then utilized to train a classifier. A Deep Belief Neural Network classifier trains each extracted feature of query frame and matches the best features of the query frames with other video features. Then we enhance the video retrieval by introducing video hashing method, combining low level and high-level semantic features for the elimination of repetitive video in the retrieval. We used precision, accuracy, sensitivity and specificity metrics to assess the applicability of the projected technique. Xiph.org and Youtube datasets are used for the valuation analysis. The experimental results show that the Eagle Eye provides better performance and less processing time compared to the other methods.

Improved Fuzzy-Based SVM Classification System Using Feature Extraction for Video Indexing and Retrieval

Various researches have been performed with video abstraction with the constant development of multimedia technology. However, there are some deficiencies that have been encountered in the pre-processing of video frames before attaining classified video archives. To overcome the drawbacks in pre-processing, feature extraction and classification approaches are considered. Here, video indexing has been anticipated with several features’ extraction with dominant frame generation for the input video frame. Fuzzy-based SVM classifier is utilized to categorize frame set into dominant structures. Multi-dimensional Histogram of Oriented Gradients (HOG) and colour feature extraction are used to extract texture features from the video frame. Using the frame sequence, the vector space of structures is captured; dominant frameworks are utilized in video indexing. Shot transitions’ classification is done with a fuzzy system. Experimental outcomes demonstrate that shot boundary detection accuracy increases with an increase in iterations. The simulation was carried out in MATLAB environment. This technique attains an accuracy of about 95.4%, the precision of 100%, and the F1 score of 100% and a recall of 100%. The misclassification rate is 4.6%. The proposed method shows better trade-off than the existing techniques.

Dual Tree Discrete Wavelet Transform and Entropy based Approach for Video Shot Boundary Detection

Shot Boundary detection is one of the most prominent steps for the various video processing techniques like indexing of videos, video retrieval, video summarization, video scene analysis, etc. In this paper, we have projected a useful technique based on dual-tree Discrete Wavelet Transform. The dual-tree structure with selected filters returns the approximate decomposition in an analytical manner. Due to the existence of the two trees, the noise component is characterized precisely. The outcome of the DTDWT is distortion less with normalized localized function and exact reconstruction of the signal. WE have used the entropy function to measure the uncertainty in the random variable. Finally, we evaluated the algorithm using Recall, Precision, and F1 measure. Experimental results demonstrate the efficiency of the given method.

Survey on Shot Boundary Detection Methods

The increase in the growth of multimedia technology leads to an increase in multimedia content in a large amount. Hence it is important to access only interesting video content instead of the whole video. For effective indexing and retrieving the interesting content from the whole video, the Content-Based Video Retrieval (CBVR) is used. Shot boundary detection is one of the most important and necessary steps. It is used to partitioning the video into shots that are necessary for indexing and retrieval of video. Therefore, segmentation plays a significant role in the field of digital image and media processing, computer vision and pattern recognition. In this paper, the recent development for shot boundary detection has been presented.

Detection of shot boundaries and extraction of key frames for video retrieval

Detection of shot boundaries and extraction of key frames for video retrieval

Key Feature Extraction for Video Shot Boundary Detection using CNN

Now days as the progress of digital image technology, video files raise fast, there is a great demand for automatic video semantic study in many scenes, such as video semantic understanding, content-based analysis, video retrieval. Shot boundary detection is an elementary step for video analysis. However, recent methods are time consuming and perform badly in the gradual transition detection. In this paper we have projected a novel approach for video shot boundary detection using CNN which is based on feature extraction. We designed couple of steps to implement this method for automatic video shot boundary detection (VSBD). Primarily features are extracted using H, V&S parameters based on mean log difference along with implementation of histogram distribution function. This feature is given as an input to CNN algorithm which detects shots which is based on probability function. CNN is implemented using convolution and rectifier linear unit activation matrix which is followed after filter application and zero padding. After downsizing the matrix it is given as a input to fully connected layer which indicates shot boundaries comparing the proposed method with CNN method based on GPU the results are encouraging with substantially high values of precision Recall & F1 measures. CNN methods perform moderately better for animated videos while it excels for complex video which is observed in the results.