Shot Boundary Detection Techniques Research Articles

Effectiveness of deep learning techniques in TV programs classification: A comparative analysis

In the application areas of streaming, social networks, and video-sharing platforms such as YouTube and Facebook, along with traditional television systems, programs’ classification stands as a pivotal effort in multimedia content management. Despite recent advancements, it remains a scientific challenge for researchers. This paper proposes a novel approach for television monitoring systems and the classification of extended video content. In particular, it presents two distinct techniques for program classification. The first one leverages a framework integrating Structural Similarity Index Measurement and Convolutional Neural Network, which pipelines on stacked frames to classify program initiation, conclusion, and contents. Noteworthy, this versatile method can be seamlessly adapted across various systems. The second analyzed framework implies directly processing optical flow. Building upon a shot-boundary detection technique, it incorporates background subtraction to adaptively discern frame alterations. These alterations are subsequently categorized through the integration of a Transformers network, showcasing a potential advancement in program classification methodology. A comprehensive overview of the promising experimental results yielded by the two techniques is reported. The first technique achieved an accuracy of 95%, while the second one surpassed it with an even higher accuracy of 87% on multiclass classification. These results underscore the effectiveness and reliability of the proposed frameworks, and pave the way for a more efficient and precise content management in the ever-evolving landscape of multimedia platforms and streaming services.

An Efficient Shot Boundary Detection Using Data-cube Searching Technique

Background:: In the internet era, there is a prime need to access and manage the huge volume of multimedia data in an effective manner. Shot is a sequence of frames captured by a single camera in an uninterrupted space and time. Shot detection is suitable for various applications such that video browsing, video indexing, content based video retrieval and video summarization. Objective:: To detect the shot transitions in the video within a short duration. It compares the visual features of frames like correlation, histogram and texture features only in the candidate region frames instead of comparing the full frames in the video file. Methods: This paper analyses candidate frames by searching the values of frame features which matches with the abrupt detector followed by the correct cut transition frame with in the datacube recursively until it detects the correct transition frame. If they are matched with the gradual detector, then it will give the gradual transition ranges, otherwise the algorithm will compare the frames within the next datacube to detect shot transition. Results:: The total average detection rates of all transitions computed in the proposed Data-cube Search Based Shot Boundary Detection technique are 92.06 for precision, 96.92 for recall and 93.94 for f1 measure and the maximum accurate detection rate. Conclusion:: Proposed method for shot transitions uses correlation value for searching procedure with less computation time than the existing methods which compares every single frame and uses multi features such as color, edge, motion and texture features in wavelet domain.

A Shot boundary Detection Technique based on Visual Colour Information

The paper proposes a novel video segmentation system with maiden application of CIEDE2000 colour-difference and mean luminace pattern. CIEDE2000 colour-difference uses Lab colour space which is a stable and efficient colour space. The main advantage of Lab colour space model is that it can approximate all the available colours perceived by our human eye. CIEDE2000 colour difference is used for detecting abrupt transitions in the video. The novel contribution of the paper is the maiden use of the mean luminance pattern, increasing and decreasing patterns of the mean value of frame luminance, for detecting the gradual transition. The approach is validated on standard databases TRECVid 2001 and 2007 test video database. The performance of the proposed technique is compared with recently reported techniques and found to be superior as compared to other techniques. The accuracy achieved with the proposed method on the standard databases is 95.9% for cut transition, 78.6% for gradual transition and 92.1% overall.

SBD-Duo: a dual stage shot boundary detection technique robust to motion and illumination effect

In this paper, we propose a novel shot boundary detection technique using gradient and colour information. The gradient similarity and luminance distortion are calculated to measure the contrast and structural changes of each frame including luminance changes. In the proposed system, the effects of the changes in luminance and contrast-structure are integrated via an adaptive method to extract the possible transitions using an adaptive threshold across the videos. In the verification part, CIEDE2000 colour-difference values of the possible transition frames are compared for declaration of abrupt and gradual transitions. Our system takes effectively less computational time to detect abrupt and gradual transition for a video as compared with contemporary solutions. Our proposed system also gives dominate the performance as compared with latest techniques in terms of F1 score using TRECVid 2001 and 2007 selected dataset. We have performed a series of rigorous experimentation to validate our claims.

A novel automatic shot boundary detection algorithm: robust to illumination and motion effect

Many researches have been done on shot boundary detection, but the performance of shot boundary detection approaches is yet to be addressed for the videos having sudden illumination and object/camera motion effects efficiently. In this paper, a novel dual-stage approach for an abrupt transition detection is proposed which is able to withstand under certain illumination and motion effects. Firstly, an adaptive Wiener filter is applied to the lightness component of the frame to retain some important information on both frequencies and LBP-HF is extracted to reduce the illumination effect. From the experimentation, it is also confirmed that the motion effect is also reduced in the first stage. Secondly, Canny edge difference is used to further remove the illumination and motion effects which are not handled in the first stage. TRECVid 2001 and TRECVid 2007 datasets are applied to analyze and validate our proposed algorithm. Experimental results manifest that the proposed system outperforms the state-of-the-art shot boundary detection techniques.

Video Authentication using Shot Boundary Detection Technique-An Overview

Video is the most challenging and has lots of issues than other type of multimedia data and video authentication is one of the popular issues regarding the video. Technologies of communication nowadays can simplify sharing of multimedia data such as text, images and videos through online or offline and a wide spread of video content its significant to assure the integrity and originality of video information. Recently, a lot of cases have been stated regarding an illegal activity in the video. Such as, video editing or video tampering attacks to ruin the authenticity of the video. With only minor alterations could change the meaning of the video. Video authentication detection is a popular issue and of high importance in various fields such as in forensic investigations, court of law and video surveillance. A lot of techniques are proposed by previous study to assure the originality of video in their own way. In this study, it focuses to detect tampering attacks and alterations in video using shot boundary detection techniques. Shot boundary detection is a very challenging task and it’s commonly used for structured video because these techniques can detect the changes made with normal shot change rate. Meanwhile, this research will emerge to produce new model for video structure detection using shot boundary detection with the histograms feature that is significant to design of powerful tools to detect an originality of video and to locate the tampering attacks in video.

A novel video shot boundary detection framework employing DCT and pattern matching

The video Shot Boundary Detection (SBD) is an elementary step in realising a system capability to perform content based video search, structural analysis, data retrieval and video summation. Myriad research works in the past have been reported to construct SBD algorithms. However, the need of an error-free, meticulous and cost-effective SBD technique still persists; for applications viz. apt management, storage, browsing, video indexing and retrieval of multimedia data. This paper is an effort in the same direction with the aim of achieving high execution speed and greater accuracy. The proposed SBD technique in this paper incorporates three steps: (i) Candidate Segment Selection (ii) Cut Transition detection (iii) Gradual Transition detection. This paper adopts pixel based technique with candidate segment selection to speed up the SBD. For Cut Transition detection, the proposed method employs Discrete Cosine Transform (DCT) and for Gradual Transition detection, it employs Image Histogram and Pattern Matching. The comparison of MATLAB simulation results of the proposed SBD technique with those in literature manifest better results in terms of execution speed and accuracy.

Fast Video Shot Boundary Detection Technique based on Stochastic Model

A video is composed of set of shots, where shot is defined as a sequence of consecutive frames captured by one camera without interruption. In video shot transition could be a prompt (hard cut) or gradual (fade, dissolve, and wipe). Shot boundary detection is an essential component of video processing. These boundaries are utilized on many aspect of video processing such as video indexing, and video in demand. In this paper, the authors proposed a new shot boundary detection algorithm. The proposed algorithm detects all type of shot boundaries in a high accuracy. The algorithm is developed based on a global stochastic model for video stream. The proposed stochastic model utilizes the joined characteristic function and consequently the joined momentum to model the video stream. The proposed algorithm is implemented and tested against different types of categorized videos. The proposed algorithm detects cuts fades, dissolves, and wipes transitions. Experimental results show that the algorithm has high performance. The computed precision and recall rates validated its performance.

Video Shot Boundary Detection Using Normalized Periodogram Distance Metric

Video shot boundary detection is the primary task for content based video management and retrieval system. This paper proposes a shot boundary detection strategy by exploiting the pros of Normalized Periodogram for efficiently representing the content of the video. A Normalized Periodogram based distance metric to detect the key frames using shot boundary, namely Distance- Left-Right (DLR), is addressed, which is computed on a sliding sub-window basis. The DLR sequence is used to detect the suspected shot boundary frames and a transition type detection procedure is adapted to these suspected frames for discriminating the abrupt and gradual transitions. The proposed shot boundary detection methodology yields Precision—95.02%, Recall—93.15% and F1 score—94.07% for cut, Precision—86.57%, Recall—86.67% and F1 score—86.61% for gradual, Precision—90.6%, Recall—90.02% and F1 score—90.3% for overall transitions. Experimental results show that the proposed approach is superior to the recently available shot boundary detection techniques because of its robustness and simplicity, and presents an effective distance metric to detect the shot boundary.

Face Detection using Principal Component Analysis (PCA)

Face Detection makes it possible to use the facial images of a person to authenticate him into secure system, for criminal identification, for passport verification etc. It is done by Principal Component Analysis (PCA).Face images are projected onto a face space that encodes best variation among known face images. The face space is collection of Eigen face. In the algorithm, initially video segmented using shot boundary detection techniques. Specifically, it can detect both the cut and gradual shot transitions in video. For detecting the shot boundary haar wavelet transform is used. In this method, each frame and its haar wavelet transform image is correlated for detection the shot. By setting the threshold of frame correlation shot boundaries can be detected. Video segmentation can be used in various application like video summarization, video search, and video annotation.

Analysis of Popular Video Shot Boundary Detection Techniques in Uncompressed Domain

days there are tremendous amount of videos available on internet. Entertainment video, news video, sports video are accessed by users to fulfill their different needs. Our daily routine systems are also producing huge amount of videos for example surveillance system, shopping malls, home videos etc. These videos need to be accessed for different purposes. Current research topics on video includes video abstraction or summarization, video classification, video annotation, content based video retrieval. In nearly all these application one needs to identify shots and key frames in video which will correctly and briefly indicate the contents of video. This paper compares some of the popular shot boundary detection techniques in uncompressed domain. The merits and demerits of each of the techniques are also discussed. Some experiment done are also discussed.

Shot Boundary Detection for a Video under the Influence of Illumination: A Adaptive Thresholding Approach

Video Shot boundary detection is the process of automatically detecting the boundaries between shots in video. Shot Boundary detection is an important fundamental process in video data access, indexing, search and retrieval. The increased availability and usage of on-line digital video has created a need for automated video content analysis techniques. Detection of gradual transition and elimination of disturbances caused by illumination change is a major challenge in shot boundary detection technique. These disturbances are often mistaken as shot boundaries. It is a crucial task to develop a method that is not only insensitive to illumination change but also sensitive to detect shot change. An algorithm is proposed for shot boundary detection in presence of illumination change. This is very important for accurate detection of shot boundaries and very useful in content based analysis of video. First the algorithm removes illumination change using discrete cosine transform and discrete wavelet transform. Then shot boundaries are detected using normal difference & wavelet difference. A shot boundary is detected when the feature difference shows sharp change greater than threshold. Experimental study is performed on number of videos that include significant illumination change. The performance of proposed algorithm is better as compared to existing techniques

Video Segmentation and Structuring for Indexing Applications

This paper introduces a complete framework for temporal video segmentation. First, a computationally efficient shot extraction method is introduced, which adopts the normalized graph partition approach, enriched with a non-linear, multiresolution filtering of the similarity vectors involved. The shot boundary detection technique proposed yields high precision (90%) and recall (95%) rates, for all types of transitions, both abrupt and gradual. Next, for each detected shot, the authors construct a static storyboard by introducing a leap keyframe extraction method. The video abstraction algorithm is 23% faster than existing techniques for similar performances. Finally, the authors propose a shot grouping strategy that iteratively clusters visually similar shots under a set of temporal constraints. Two different types of visual features are exploited: HSV color histograms and interest points. In both cases, the precision and recall rates present average performances of 86%.

Shot boundary detection in the presence of illumination and motion

Detection of gradual transition and the elimination of disturbances caused by illumination change or fast object and camera motion are the major challenges to the current shot boundary detection techniques. These disturbances are often mistaken as shot boundaries. Therefore, it is a challenging task to develop a method that is not only insensitive to various disturbances but also sensitive enough to capture a shot change. To address these challenges, we propose an algorithm for shot boundary detection in the presence of illumination change, fast object motion, and fast camera motion. This is important for accurate and robust detection of shot boundaries and in turn critical for high-level content-based analysis of video. First, the propose algorithm extracts structure features from each video frame by using dual-tree complex wavelet transform. Then, spatial domain structure similarity is computed between adjacent frames. The declaration of shot boundaries are decided based on carefully chosen thresholds. Experimental study is performed on a number of videos that include significant illumination change and fast motion of camera and objects. The performance comparison of the proposed algorithm with other existing techniques validates its effectiveness in terms of better Recall, Precision, and F1 score.

Automatic video temporal segmentation based on multiple features

This paper investigates automatic video temporal segmentation techniques, also named shot boundary detection (SBD) techniques. Firstly, the existing SBD algorithms are reviewed in detail. Then, a new SBD algorithm is proposed aiming to obtain fast and accurate detection, and its performances are evaluated and compared with existing works. This algorithm computes the frame difference/similarity by such simple features as pixel difference and histogram difference, adopts motion-based difference to resist camera or object movements in the same shot and uses the flash detection to avoid false positives caused by light changes or flashes. The adopted features are computational efficient, and the combination of various features improve the detection accuracy. These properties make the algorithm suitable for real-time applications, such as broadcasted news segmentation.

A compressed-domain approach for shot boundary detection on H.264/AVC bit streams

The amount of digital video content has grown extensively during recent years, resulting in a rising need for the development of systems for automatic indexing, summarization, and semantic analysis. A prerequisite for video content analysis is the ability to discover the temporal structure of a video sequence. In this paper, a novel shot boundary detection technique is introduced that operates completely in the compressed domain using the H.264/AVC video standard. As this specification contains a number of new coding tools, the characteristics of a compressed bit stream are different from prior video specifications. Furthermore, the H.264/AVC specification introduces new coding structures such as hierarchical coding patterns, which can have a major influence on video analysis algorithms. First, a shot boundary detection algorithm is proposed which can be used to segment H.264/AVC bit streams based on temporal dependencies and spatial dissimilarities. This algorithm is further enhanced to exploit hierarchical coding patterns. As these sequences are characterized by a pyramidal structure, only a subset of frames needs to be considered during analysis, allowing the reduction of the computational complexity. Besides the increased efficiency, experimental results also show that the proposed shot boundary detection algorithm achieves a high accuracy.

A STATISTICAL APPROACH TO ROBUST VIDEO TEMPORAL INDEXING AND SEGMENTATION

For video indexing, the problem of video cut detection remains largely an open problem because of the wide nature of transitions that occur in a digital video. This paper describes a shot boundary detection technique, which is an amalgamation of few statistical methods and measures, and robustly detects camera breaks in a full-motion video clip. The proposed algorithm incorporates a weighted histogram, an error-propagation technique for increased robustness, and a curve-fitting technique to extract partitions from the similarity curve for avoiding heuristically chosen threshold value. The algorithm has been validated on many video clips and is shown to give improved results, including for videos with rapid scanning, changes in illumination, fade-ins and fade-outs, and with special effects like dissolve and filters.

Entropy-Based Fade Modeling and Detection

Accurate shot boundary detection techniques have been an important research topic in the last decade. Such interest is motivated by the fact that segmenting a video stream is the first step towards video content analysis and content-based video browsing and retrieval. In this paper, we present a new algorithm mainly focused on the detection of fades by using of a non-common feature in previous work: entropy, a scalar representation of the amount of information of each video frame. A survey of the properties of this feature is first provided where authors show that the pattern this series exhibits when fades occur is clear and consistent. It does not depend on the monochrome color used to fade and, besides, it is able to deal with on-screen texts remaining in the monochrome stage of them. A statistical model based algorithm to detect fades is proposed. Due to the clear pattern shown by fades in the entropy series and the accurate mathematical model used, motion and illumination changes do not severely affect precision as it normally happens with algorithms dealing with the detection of gradual transitions.

A Formal Study of Shot Boundary Detection

This paper conducts a formal study of the shot boundary detection problem. First, a general formal framework of shot boundary detection techniques is proposed. Three critical techniques, i.e., the representation of visual content, the construction of continuity signal and the classification of continuity values, are identified and formulated in the perspective of pattern recognition. Meanwhile, the major challenges to the framework are identified. Second, a comprehensive review of the existing approaches is conducted. The representative approaches are categorized and compared according to their roles in the formal framework. Based on the comparison of the existing approaches, optimal criteria for each module of the framework are discussed, which will provide practical guide for developing novel methods. Third, with all the above issues considered, we present a unified shot boundary detection system based on graph partition model. Extensive experiments are carried out on the platform of TRECVID. The experiments not only verify the optimal criteria discussed above, but also show that the proposed approach is among the best in the evaluation of TRECVID 2005. Finally, we conclude the paper and present some further discussions on what shot boundary detection can learn from other related fields

Scene Change Detection Schemes for Video Indexing in Uncompressed Domain

There are a growing number of applications, which extensively use the visual media. A key requirement in those applications is efficient access to the stored visual information for the purposes of indexing, fast retrieval, and scene analysis. To support this vision, many scene change detection algorithms have been developed for both sudden and gradual scene change in uncompressed domain. This paper presents a comparison of shot boundary detection and classification techniques and their variations including histogram, pixel-differences, likelihood ratio, and motion vector. In addition, the pros and cons of each approach with many clarifying comments are presented.