Video Shot Boundary Detection Research Articles

Video shot-boundary detection: issues, challenges and solutions

The integration of high data transmission rates and the recent digital multimedia technology, paves the way to access a huge amount of video over the internet, in seconds. Additionally, uploading videos to different websites is no more confined to expert software professionals resulting in duplication of video data which led to exorbitant growth of multimedia information in cyberspace in a short span of time. This necessitates the development of efficient data management techniques including storage, searching and annotation mechanism. Automatic shot boundary detection is considered to be the first and foremost step towards such management. It is a booming area of research gaining attention in the domain of image processing, computer vision and pattern recognition. In this review paper, we present a detailed description of the methods and algorithms of shot boundary detection, reported in the last two decades. This review shows that using multiple features performs well in comparison to using only a single feature in the shot boundary detection problem although it leads to higher complexity. The major sources of disturbance in the boundary detection are the sudden illumination variation and presence of high motion in the video. An adaptive threshold outperforms a single global threshold in the boundary detection problem and the threshold requirement can be avoided through learning based strategies at the cost of larger training data and higher computation time. Moreover the present review includes a critical analysis of relative merits and demerits of existing algorithms and finally opens promising research directions in the area.

Video Shot Boundary Detection and Sports Video Classification Algorithm Based on Particle Filter

INTRODUCTION: Sports video is a very important information resource. The classification of sports video with high accuracy can effectively improve the browsing and query effect of users. This project intends to study a motion video classification algorithm based on deep learning particle filter to solve the problems of strong subjectivity and low accuracy of existing motion video classification algorithms. A key box extraction method based on similarity is proposed. The moving video classification algorithm based on deep learning coding model is studied. Examples of various types of sports videos are analyzed. The overall performance of the motion video classification algorithm proposed in this paper is much better than other existing motion video classification algorithms. This algorithm can greatly improve the classification performance of motion video.

Hybridized Neural Network Based Approaches Used for Video Shot Boundary Detection

Hybridized Neural Network Based Approaches Used for Video Shot Boundary Detection

Visual significance model based temporal signature for video shot boundary detection

Visual significance model based temporal signature for video shot boundary detection

Shot Boundary Detection Based on Global Features and the Target Features

Video processing plays an important role in the intelligent monitoring and management system of agricultural information. Video shot boundary detection is the basic symmetry step underlying video processing techniques. According to the current shot boundary detection algorithm, the feature changes between gradual transition frames are difficult to detect, and the misdetection situation is caused by ignoring the attention of the target feature during the feature extraction. A novel symmetry multi-step comparative scheme of shot boundary detection algorithm based on global features and target features is proposed. First, the RGB color histogram features of the video frame are extracted. Second, foreground object detection for the video frames is performed using the Gaussian Mixture Model (GMM), and the scale-invariant features transformation (SIFT) of the foreground targets is extracted. Finally, global features and target features fusion through weights, calculating the difference between adjacent frames across multiple steps, generate a pattern distance map. The pattern distance map of the gradual transition and the cut detection is different; we can judge the gradual transition and the cut detection according to the pattern distance map. Experiments show that the proposed symmetry method improves by about 2% in recall and accuracy compared to other algorithms.

A novel method for video shot boundary detection using CNN-LSTM approach

A novel method for video shot boundary detection using CNN-LSTM approach

The Detection of Video Shot Transitions Based on Primary Segments Using the Adaptive Threshold of Colour-Based Histogram Differences and Candidate Segments Using the SURF Feature Descriptor

Aim: Advancements in multimedia technology have facilitated the uploading and processing of videos with substantial content. Automated tools and techniques help to manage vast volumes of video content. Video shot segmentation is the basic symmetry step underlying video processing techniques such as video indexing, content-based video retrieval, video summarization, and intelligent surveillance. Video shot boundary detection segments a video into temporal segments called shots and identifies the video frame in which a shot change occurs. The two types of shot transitions are cut and gradual. Illumination changes, camera motion, and fast-moving objects in videos reduce the detection accuracy of cut and gradual transitions. Materials and Methods: In this paper, a novel symmetry shot boundary detection system is proposed to maximize detection accuracy by analysing the transition behaviour of a video, segmenting it initially into primary segments and candidate segments by using the colour feature and the local adaptive threshold of each segment. Thereafter, the cut and gradual transitions are fine-tuned from the candidate segment using Speeded-Up Robust Features (SURF) extracted from the boundary frames to reduce the algorithmic complexity. The proposed symmetry method is evaluated using the TRECVID 2001 video dataset, and the results show an increase in detection accuracy. Result: The F1 score obtained for the detection of cut and gradual transitions is 98.7% and 90.8%, respectively. Conclusions: The proposed symmetry method surpasses recent state-of-the-art SBD methods, demonstrating increased accuracy for both cut and gradual transitions in videos.

Hybrid feature extraction and optimized deep convolutional neural network based video shot boundary detection

SummaryThis paper presents a hybrid feature extraction and optimized deep learning model for the effective shot boundary detection (SBD). A cross‐guided bilateral filtering technique is used for pre‐processing. A hybrid fuzzy histogram with dual tree complex wavelet transform (FH‐DTCWT) is developed for the feature extraction to extract the visual features from the pre‐processed frames of the block. Candidate segment selection process is performed to identify the non‐boundary frames from the extracted features to improve the accuracy. Continuity matrix is created using the possible transition frames to verify that the frames are in sequential order without any gap. Classification of transition types are performed by the proposed optimized deep convolutional neural network. Optimized deep learning (DCNN‐RBESO) model is the combination of deep convolutional neural network (DCNN) and Rider bald eagle search optimization (RBESO) algorithm. To update the weight of DCNN model in learning, RBESO is utilized. MATLAB tool is used to develop the proposed SBD model. The experimental analysis is carried out for the proposed model using TREC Video Retrieval Evaluation (TRECVID) and VideoSeg datasets. The proposed model outperformed in the detection of shot boundaries in terms of precision, recall, and F1‐score.

Efficient Detection of Abrupt Transitions Using Statistical Methods

The rapid increment of technological advancements in the field of multimedia stream has piled up data production and consumption in cyberspace in the past two decades. It has prompted a swift increase in data transmission volume and repository size. Video is the most consumed data type on the internet. So, retrieval of selected video clips from the extra-large database of videos is extremely complex. The video shot boundary detection (SBD) is used for retrieval of desired video clips. It also is the fundamental step in video processing, which also was found important for applications like video browsing and indexing. The SBD handles the identification of abrupt and gradual transitions in the video database. In this paper, the statistical methods have implemented to detect the abrupt cuts. This benefitted the advantage of less complexity. The experiments are performed on TRECVID 2007 dataset, and the results obtained with high accuracy in less computational time.

Sports Video Panorama Synthesis Technology Based on Edge Computing and Video Shot Boundary Detection

The development requirements for sports panorama synthesis technology, which rely on modern network technology, abandon traditional basketball training forms, and make effective use of the application and development of video panorama technology in the actual training process, are not only an important response to the current characteristics of students’ physical education learning and physical and mental growth but also an important response to the current characteristics of students’ physical education learning and physical and mental growth. In sports video analysis technology, a sports video panorama is a technical tool that converts an action video into a static action panorama to achieve the effect of action freezing and make overall analysis and mastery of the action easier. Shot segmentation is the foundation of hierarchical video structure, and it necessitates the accurate detection of all types of complex edited shot boundaries, as well as the effective distinction of motion changes in shots, in order to avoid shot boundary recognition being hampered. The synthesis technology for sports video panorama is investigated in this paper using edge computing and video shot boundary detection. After obtaining the boundary feature that describes the video shot, a comparison of this feature with a predetermined threshold value can be used to determine whether there is shot shear.

Video shot boundary detection using hybrid dual tree complex wavelet transform with Walsh Hadamard transform

Video shot boundary detection using hybrid dual tree complex wavelet transform with Walsh Hadamard transform

A novel tensor-information bottleneck method for multi-input single-output applications

Ensuring timeliness and mobility for multimedia computing is a crucial task for wireless communication. Previous algorithms that utilize information channels, such as the information bottleneck method, have shown great performance and efficiency, which guarantees timeliness. However, such methods suit only in handling single variable tasks such as image processing, but are inapplicable to multivariable applications such as video processing. To address this critical shortcoming, we propose a novel tensor information channel which extends the current single-input single-output matrix information channel to a more practical multi-input single-output tensor information channel. In comparison with the classic information channel, our tensor information channel not only performs better in experiments, but also allows for a wider range of practical applications. We further build an innovative tensor-information bottleneck method upon the state-of-the-art information bottleneck method. Experiments on video shot boundary detection are conducted using benchmark data sets to demonstrate the effectiveness of our proposed approach compared with state-of-the-art methods. In specific, our approach yields a 6.2% increase compared with the information channel-based method, and when compared to other state-of-the-art methods, we achieve 0.1%–17.7% performance gains under different experimental configurations.

Video shot boundary detection based on multi-level features collaboration

Video shot boundary detection (SBD) is a basic work of content-based video retrieval and analysis. Various SBD methods have been proposed; however, there exist limitations in the complexity of boundary detection process. In this paper, a simple yet efficient SBD method is proposed, and the aim here is to speed up the boundary detection and simplify the detection process without loss of detection recall and accuracy. In our proposed model, we mainly use a top-down zoom rule, the image color feature, and local descriptors and combine a kind of motion area extraction algorithm to achieve shot boundary detection. Firstly, we select candidate transition segments via color histogram and the speeded-up robust features. Then, we perform cut transition detection through uneven slice matching, pixel difference, and color histogram. Finally, we perform gradual transition detection by the motion area extraction, scale-invariant feature transform, and even slice matching. The experiment is evaluated on the TRECVid2001 and the TRECVid2007 video datasets, and the experimental results show that our proposed method improves the recall, accuracy, and the detection speed, compared with some other related SBD methods.

Video shot boundary detection using block based cumulative approach

Video data is becoming an indispensable part of today’s Big Data due to evolution of social web and mobile technology. Content based video analysis has become crucial for video management. Shot boundary detection is one of the most essential task in video content analysis. In view of this, an efficient shot boundary detection approach to detect abrupt and gradual transition in videos is proposed in this work. The approach extracts block based Mean Cumulative Sum Histogram (MCSH) from each edge gradient fuzzified frame as a combination of local and global feature. The relative standard deviation (RSD) statistical measure is applied on the obtained MCSH to detect abrupt and gradual shots in the video. Efficacy of the proposed method is measured by conducting experiments on TRECVID 2001, TRECVID 2007 and VideoSeg datasets. The proposed method shows relatively a good performance when compared to some of the state-of-the-art shot boundary detection approaches.

Dual Tree Discrete Wavelet Transform and Entropy based Approach for Video Shot Boundary Detection

Shot Boundary detection is one of the most prominent steps for the various video processing techniques like indexing of videos, video retrieval, video summarization, video scene analysis, etc. In this paper, we have projected a useful technique based on dual-tree Discrete Wavelet Transform. The dual-tree structure with selected filters returns the approximate decomposition in an analytical manner. Due to the existence of the two trees, the noise component is characterized precisely. The outcome of the DTDWT is distortion less with normalized localized function and exact reconstruction of the signal. WE have used the entropy function to measure the uncertainty in the random variable. Finally, we evaluated the algorithm using Recall, Precision, and F1 measure. Experimental results demonstrate the efficiency of the given method.

Key Feature Extraction for Video Shot Boundary Detection using CNN

Now days as the progress of digital image technology, video files raise fast, there is a great demand for automatic video semantic study in many scenes, such as video semantic understanding, content-based analysis, video retrieval. Shot boundary detection is an elementary step for video analysis. However, recent methods are time consuming and perform badly in the gradual transition detection. In this paper we have projected a novel approach for video shot boundary detection using CNN which is based on feature extraction. We designed couple of steps to implement this method for automatic video shot boundary detection (VSBD). Primarily features are extracted using H, V&S parameters based on mean log difference along with implementation of histogram distribution function. This feature is given as an input to CNN algorithm which detects shots which is based on probability function. CNN is implemented using convolution and rectifier linear unit activation matrix which is followed after filter application and zero padding. After downsizing the matrix it is given as a input to fully connected layer which indicates shot boundaries comparing the proposed method with CNN method based on GPU the results are encouraging with substantially high values of precision Recall & F1 measures. CNN methods perform moderately better for animated videos while it excels for complex video which is observed in the results.

A new pyramidal opponent color-shape model based video shot boundary detection

Video shot boundary detection (VSBD) is one of the most essential criteria for many intelligent video analysis-related applications, such as video retrieval, indexing, browsing, categorization and summarization. VSBD aims to segment big video data into meaningful fragments known as shots. This paper put forwards a new pyramidal opponent colour-shape (POCS) model which can detect abrupt transition (AT) and gradual transition (GT) simultaneously, even in the presence of illumination changes, huge object movement between frames, and fast camera motion. First, the content of frames in the video subjected to VSBD is represented by the proposed POCS model. Consequently, the temporal nature of the POCS model is subjected to a suitable segment (SS) selection procedure in order to minimize the complexity of VSBD method. The SS from the video frames is examined for transitions within it using a bagged-trees classifier (BTC) learned on a balanced training set via parallel processing. To prove the superiority of the proposed VSBD algorithm, it is evaluated on the TRECVID 2001, TRECVID2007 and VIDEOSEG2004 data sets for classifying the basic units of video according to no transition (NT), AT and GT. The experimental evaluation results in an F1-score of 95.13%, 98.13% and 97.11% on the TRECVID 2001, TRECVID2007 and VIDEOSEG2004 data sets, respectively.

Video Shot Boundary Detection Based on Feature Fusion and Clustering Technique

For the problems of low accuracy and high complexity in detection of gradual shot boundary and long shot, a new video shot boundary detection algorithm based on feature fusion and clustering technique (FFCT) is proposed. In the algorithm, the interval frames of video sequence are selected, converted to gray images and scaled by sampling. With the frames, the speed-up robust features (SURF) and fingerprint features are extracted from non-compressed domain and compressed domain, and then the extracted features are fused. Next, K-means method is used to cluster the fused features, and linear discriminant analysis (LDA) is introduced to map the clusters to realize cohesion within classes and looseness among classes. Finally, the correlation of the feature classes between frames is calculated, and the features in each class are selected through density calculation and matched to realize the coarse detection and fine detection of video shot boundary. In the experiment, compared with the latest representative algorithms, it has the highest accuracy for the proposed algorithm. In particular, the detection of gradual shot boundary and long shot are also more accurate. Meanwhile, the average time consumption is also reduced. The experimental results show that the proposed algorithm has high accuracy and time efficiency, especially for gradual shot boundary and long shot detection.

A Proposed Method for Video's Shot Boundary Detection based on Multiple Features

A Proposed Method for Video's Shot Boundary Detection based on Multiple Features

Novel Shot Boundary Detection in News Streams Based on Fuzzy Petri Nets

ABSTRACTWith the advent of a digital era, people have encountered some difficulty in using and absorbing overwhelming information generated by technological advances in multimedia. Thus, the development of video summarization enables people to catch a general idea about videos in a short time. In this paper, we focus on the shot change, a part of the video summarization, to conduct an experimental sample on news programs. Moreover, a high-level fuzzy Petri net model is presented to describe the frame combination which indicates a shot boundary used for a video frame sequence in order to detect both cut transitions and gradual transitions. This study has used feature functions to estimate the direct shot change in consideration of video shot boundary detection which adopts the HLFPN model to find a threshold value. The experimental results manifest that the proposed system saves a lot of time and reduces the occurrence of improper shot changes caused by the motions of objects and cameras when comparing the proposed approach with other existing ones.