Scene Change Detection Research Articles

Conversation Disrupts Change Detection in Complex Traffic Scenes

Conversation Disrupts Change Detection in Complex Traffic Scenes

A software-based high-quality MPEG-2 encoder employing scene change detection and adaptive quantization

The paper presents a new software-only MPEG encoder implementation which uses adaptive quantization and scene change detection to enhance image quality. Scene change detection is coupled to the bit-allocation process, providing a more constant image quality over time. Simultaneously, it is used to assign picture coding types to enable easy, lossless cutting at scene change positions in a later editing process. Furthermore, a new fast bit-rate estimation algorithm is proposed, which is accurate enough to avoid macroblock-level rate-control. Although our current implementation concentrates on an MPEG-2 implementation, all concepts are readily applicable to MPEG-4 encoders.

Fast scene change detection for personal video recorder

We propose an efficient scene change detection method for personal video recorder to provide users with several convenient functionalities such as the browsing, fast forward and skip-play of recorded videos. Since most of the current available PVRs have low-powered CPUs, the scene change detector should not be computationally expensive. We implement the scene change detection algorithm using reduced images of only intra pictures in MPEG-2 compressed domain. The experimental results with a commercial PVR demonstrate the efficiency of the proposed method.

Adaptive video-dissolve detection method based on correlation between successive scenes

We establish the importance of correlation between successive scenes in dissolve detection and propose a new adaptive dissolve detection method based on the analysis of a dissolve modeling error, which is the difference between an ideally modeled dissolve curve with no correlation and an actual dissolve curve including correlation. The proposed method consists of two steps. First, candidate dissolve regions are extracted using the characteristics of a downward-convex parabola; then each candidate region is checked using the dissolve modeling error. If the dissolve modeling error for a candidate region is less than a threshold equal to the dissolve modeling error with a target correlation, the candidate region is identified as a dissolve region with a lower correlation than the target correlation. The threshold is determined adaptively from the variance at the start and end of the candidate region and the given target correlation. By considering the correlation between successive scenes, the proposed method is able to function as a semantic scene-change detector. The proposed algorithm was tested on various types of data, and its performance proved to be more accurate and reliable than that of other, commonly used methods.

An Adaptive Video Coding Control Scheme for Real-Time MPEG Applications

This paper proposes a new rate control scheme to increase the coding efficiency for MPEG systems. Instead of using a static group of picture (GOP) structure, we present an adaptive GOP structure that uses more P-and B-frame coding, while the temporal correlation among the video frames maintains high. When there is a scene change, we immediately insert intramode coding to reduce the prediction error. Moreover, an enhanced prediction frame is used to improve the coding quality in the adaptive GOP. This rate control algorithm can both achieve better coding efficiency and solve the scene change problem. Even if the coding bit rate is over the predefined level, this coding scheme does not require re-encoding for real-time systems. Simulations demonstrate that our proposed algorithm can achieve better quality than TM5, and satisfactory reliability for detecting scene changes.

Statistical sequential analysis for real-time video scene change detection on compressed multimedia bitstream

The increased availability and usage of multimedia information have created a critical need for efficient multimedia processing algorithms. These algorithms must offer capabilities related to browsing, indexing, and retrieval of relevant data. A crucial step in multimedia processing is that of reliable video segmentation into visually coherent video shots through scene change detection. Video segmentation enables subsequent processing operations on video shots, such as video indexing, semantic representation, or tracking of selected video information. Since video sequences generally contain both abrupt and gradual scene changes, video segmentation algorithms must be able to detect a large variety of changes. While existing algorithms perform relatively well for detecting abrupt transitions (video cuts), reliable detection of gradual changes is much more difficult. A novel one-pass, real-time approach to video scene change detection based on statistical sequential analysis and operating on a compressed multimedia bitstream is proposed. Our approach models video sequences as stochastic processes, with scene changes being reflected by changes in the characteristics (parameters) of the process. Statistical sequential analysis is used to provide an unified framework for the detection of both abrupt and gradual scene changes.

High accuracy flashlight scene determination for shot boundary detection

Shot boundary detection, or scene change detection, is a technique used in the initial phase of video indexing. One of the problems in the detection is the discrimination of abrupt scene change from flashlight scenes. The usual discriminate method tests the similarity of the frame before and after a suspected flashlight effect. However, the performance of such a technique in discriminating flashlight scene from abrupt scene change can be affected by the scene content. To overcome this, we present a novel method that utilises the edge direction, thereby reducing erroneous matching with increasing dilation radius. This improves the accuracy of similarity testing and reduces the amount of erroneously matched edges by four times. Our experiment in discriminating flashlight effect from abrupt scene change frame pairs shows that our technique produces a perfect detection, which cannot be achieved by normal edge-based detection. Such a contribution is important as it improves the indexing of real life video.

Scene Change Detection Schemes for Video Indexing in Uncompressed Domain

There are a growing number of applications, which extensively use the visual media. A key requirement in those applications is efficient access to the stored visual information for the purposes of indexing, fast retrieval, and scene analysis. To support this vision, many scene change detection algorithms have been developed for both sudden and gradual scene change in uncompressed domain. This paper presents a comparison of shot boundary detection and classification techniques and their variations including histogram, pixel-differences, likelihood ratio, and motion vector. In addition, the pros and cons of each approach with many clarifying comments are presented.

Shot boundary refinement for long transition in digital video sequence

Shot boundary detection, or scene change detection, is a technique used in the initial step of video indexing. With the recent introduction of object-based shot boundary detection technique, the deficiency of traditional detection techniques in handling long transitions has been overcome. However, such a technique merely detects the existence of a transition without giving accurate positions of the transition start and end for long transitions. Shot boundary refinement fills this gap by pinpointing the exact location of the shot boundary regardless of the transition types. This paper presents the theories behind the transition and classifies existing transition types. A novel shot boundary refinement algorithm is then constructed which works independently of the contents and transition length, and is accurate under most transition types. The results of testing using different indicators and different transition types show that the boundaries of long transitions are detected with minimal errors. The success of shot boundary refinement brings us one step closer in developing an accurate, fully automatic indexing system.

A new enhancement method for digital video applications

A new approach for digital video enhancement is proposed in order to achieve optimal picture quality for MPEG encoded video. It takes account of the special characteristics of the encoded video by defining a usefulness metric for enhancement (UME). This metric is calculated based on coding information and local spatial features to quantify how much a pixel can be enhanced without boosting coding artifacts. Motion estimation and scene change detection are integrated in the proposed algorithm to improve the temporal consistency. The idea proposed here can be applied to improve the performance of video enhancement algorithms for digital compressed sources. To demonstrate the potential of the new approach, the UME is applied to a spatial-domain sharpness enhancement algorithm. Experimental results show significant improvement with respect to the algorithm without the UME for various video sequences containing coding artifacts. This new approach can be incorporated into future digital video products (e.g. DVD recorders, set-top boxes, digital TVs, etc) in order to achieve superior picture quality for encoded video.

A hard disk drive embedded digital satellite receiver with scene change detector for video indexing

We present a hard disk drive embedded digital satellite receiver with a scene change detector for video indexing. We implement a scene change detection algorithm in the compressed domain for low computing power systems. This receiver can store, retrieve and manage the broadcast data by implementing an interface between the conventional digital satellite receiver and digital storage media. In addition, this receiver gives users an ability to search the scene change position by implementing a scene change detection algorithm. The detected temporal video segments are stored in the HDD and retrieved when users want them. The user can obtain more information for efficient video retrieval, using this proposed system.

Temporal Segmentation of MPEG Video Streams

Many algorithms for temporal video partitioning rely on the analysis of uncompressed video features. Since the information relevant to the partitioning process can be extracted directly from the MPEG compressed stream, higher efficiency can be achieved utilizing information from the MPEG compressed domain. This paper introduces a real-time algorithm for scene change detection that analyses the statistics of the macroblock features extracted directly from the MPEG stream. A method for extraction of the continuous frame difference that transforms the 3D video stream into a 1D curve is presented. This transform is then further employed to extract temporal units within the analysed video sequence. Results of computer simulations are reported.

On-line knowledge- and rule-based video classification system for video indexing and dissemination

Current information and communication technologies provide the infrastructure to transport bits anywhere, but do not indicate how to easily and precisely access and/or route information at the semantic level. To facilitate intelligent access to the rich multimedia data over the Internet, we develop an on-line knowledge- and rule-based video classification system that supports automatic “indexing” and “filtering” based on the semantic concept hierarchy. This paper investigates the use of video and audio content analysis, feature extraction and clustering techniques for further video semantic concept classification. A supervised rule-based video classification system is proposed using video automatic segmentation, annotation and summarization techniques for seamless information browsing and updating. In the proposed system, a real-time scene-change detection proxy performs an initial video-structuring process by splitting a video clip into scenes. Motional, visual and audio features are extracted in real-time for every detected scene by using on-line feature-extraction proxies. Higher semantics are then derived through a joint use of low-level features along with classification rules in the knowledge base. Classification rules are derived through a supervised learning process that relies on some representative samples from each semantic category. An indexing and filtering process can now be built using the semantic concept hierarchy to personalize multimedia data based on users’ interests. In real-time filtering, multiple video streams are blocked, combined, or sent to certain channels depending on whether or not the video streams are matched with the user's profile. We have extensively experimented and evaluated the classification and filtering techniques using basketball sports video data. In particular, in our experiment, the basketball video structure is examined and categorized into different classes according to distinct motional, visual and audio characteristics features by a rule-based classifier. The concept hierarchy describing the motional/visual/audio feature descriptors and their statistical relationships are reported in this paper along with detailed experimental results using on-line sports videos.

동적 임계값과 컷 프레임 차를 이용한 점진적 전환 검출 기법

내용기반 검색을 위한 비디오 데이터 장면전환 검출에서 점진적인 전환을 검출하는 것은 갑작스런 전환을 검출하는 것에 비해 일반적으로 어려운 문제로 알려져 있다. 본 논문에서는 가변형 동적 임계값과 가장 최근에 검출된 컷 프레임과 현재 프레임간의 특징값 차이인 컷 프레임 차를 이용하여 갑작스런 전환과 점진적인 전환을 찾아내는 기법을 제안한다. 이를 위하여 본 논문에서는 점진적인 전환이 갖는 특성과 수학적 모델을 제시하고 컷 프레임 차를 이용하여 점진적인 전환을 검출할 수 있음을 보인다. 그리고 이를 바탕으로 갑작스런 전환과 점진적인 전환을 함께 검출할 수 있는 방법을 제시한다. 실세계 동영상 데이터를 대상으로 한 실험을 통해 제안하는 기법이 점진적인 전환 효과의 종류에 종속적이지 않으며 적은 연산 비용으로 쉽게 점진적인 전환 유무를 검출 할 수 있음을 보인다. Gradual scene change detection is known as more difficult problem then abrupt scene change detection on video data analysis for contents based retrieval. In this paper, we present a new method for scene change detection far both abrupt and gradual change using the variable dynamic threshold arid cut frame difference (CFD). For this, We present the characteristics arid mathematical models of gradual transitions anti then, how can be detected by the CFD. And also we present new scene change detection algorithm based on cut frame difference. By the experimental result using real world video data indicate that the proposed method detect various scene changes both abrupt and gradual change efficiently without time-consuming computation and any dependency on a kind of gradual change effects.

MPEG으로 압축된 비디오에서 최소 복호화에 의한 빠른 장면전환검출 알고리듬

A scene change detection which involves finding a cut between two consecutive shots is an important step for video indexing and retrieval. This paper proposes an algorithm for fast and accurate detection of abrupt scene changes in an MPEG compressed domain with minimal decoding requirements arid computational effort. The proposed method compares two successive DC images of I-frames for finding the GOP (group of picture) which contain a scene change and uses macroblock-coded type information contained in B-frames to detect the exact frame where the scene change occurred. The experiment results demonstrate that the proposed algorithm has better detection performance, such as precision and recall rate, than the existing method using all DC images. The algorithm has the advantage of speed, simplicity and accuracy. In addition, it requires less amount of storage.

에지 영상의 분산을 이용한 비디오의 점진적 장면전환 검출

A new algorithm for gradual scene change detection in MPEG based frame sequences is proposed in this paper. The proposed algorithm is based on the fact that most of gradual curves can be characterized by variance distributions of edge information in the frame sequences. Average edge frame sequences are obtained by performing "sober" edge detection. Features are extracted by comparing variances with those of local blocks in the average edge frames. Those features are further processed by the opening operation to obtain smoothing variance curves. The lowest variance in the local frame sequences is chosen as a gradual detection point. Experimental results show that the proposed method provides 85% precision and 86% recall rate fur gradual scene changes.

Eminent scene change detection algorithm and new indexing method for video retrieval on MPEG sequences

We propose an eminent scene change detection (SCD) algorithm, and also present a new video-indexing method for fast content- based browsing and retrieval in video databases. The first step in our approach is to extract representative frames by using the proposed SCD algorithm from the Motion Picture Experts Group (MPEG) video sequence. Thereafter, we perform the video indexing by applying the rosette pattern (RP) to the extracted representative frames, and retrieve them. A principal cause of these processing procedures is that the performance of video indexing and retrieval system depends on the SCD. And how accurately SCD performs as well as a preprocessing are some of the considerations that we must take into account. Therefore, we classify every possible scene change that may happen within the video into three types: abrupt, gradual, and flashlight. And we detect every kind of scene change. Our SCD algorithm is better than the conventional ones in terms of SCD performance and we demonstrate its superiority by experiments. We also extract representative frames that represent series of detected scenes by the proposed accumulative histogram intersection measure (AHIM). Finally, we apply the RP to extracted representative frames for indexing, we can remarkably reduce the number of pixels (NOP) required to index and excellently retrieve the video scene.

Motion-Based Video Representation for Scene Change Detection

In this paper, we present a new framework to automatically group similar shots into one scene, where a scene is generally referred to as a group of shots taken place in the same site. Two major components in this framework are based on the motion characterization and background segmentation. The former component leads to an effective video representation scheme by adaptively selecting and forming keyframes. The later is considered novel in that background reconstruction is incorporated into the detection of scene change. These two components, combined with the color histogram intersection, establish our basic concept on assessing the similarity of scenes.

Monitoring human behavior from video taken in an office environment

Abstract In this paper, we describe a system which automatically recognizes human actions from video sequences taken of a room. These actions include entering a room, using a computer terminal, opening a cabinet, picking up a phone, etc. Our system recognizes these actions by using prior knowledge about the layout of the room. In our system, action recognition is modeled by a state machine, which consists of ‘states’ and ‘transitions’ between states. The transitions from different states can be made based on a position of a person, scene change detection, or an object being tracked. In addition to generating textual description of recognized actions, the system is able to generate a set of key frames from video sequences, which is essentially content-based video compression. The system has been tested on several video sequences and has performed well. A representative set of results is presented in this paper. The ideas presented in this system are applicable to automated security.

Constrained bit-rate control for very low bit-rate streaming-video applications

We propose a low-complexity frame-layer bit-rate control algorithm for very low bit-rate streaming-video applications with causal one-pass processing. Rate control is achieved by jointly adapting the frame rate and quantization step-size. Constraint coding is introduced, which forces the encoder to operate within a subset of operating points on a 2-D grid. The control parameters are also constrained to change in a gradual fashion, allowing bits to be saved in easy scenes so that more bits can be used for difficult scenes. A simple scene-change detector is used to insert intra frames at scene-change boundaries. The proposed algorithm is implemented in H.263 and is compared with the case of fixed control parameters and a conventional off-line bit-rate control based on adapting the quantization step-size. It is shown that the proposed algorithm codes more frames over a given time interval while achieving average PSNR gains over 1 dB and reducing bit-rate fluctuations. Results are confirmed by subjective tests, which show that our bit-rate control consistently provides improved visual quality.